Survey experiments on policymakers' perspectives towards evaluation and experimentation
How to cite
Fuller, Rob; Phipps, James; Brackin, Maria. 2025. "Survey experiments on policymakers' perspectives towards evaluation and experimentation." AEA RCT Registry. AEARCTR-0016824. https://doi.org/10.1257/rct.16824-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
The Innovation Growth Lab (IGL) and Technopolis Group will conduct an online survey of policymakers and policy practitioners across Europe, to assess their current level of understanding of policy experimentation, the drivers of and barriers to use of experimentation, and their views on the potential for a community of practice.
The survey incorporates four randomised experiments:
- A conjoint (discrete-choice) experiment designed to elicit respondents’ willingness to engage in randomised experimentation, and which conditions are conducive to this.
- An A/B test to determine whether respondents’ perceptions of their understanding of experimentation is affected by whether they are prompted with a particularly rigorous definition of what is meant by experimentation.
- An A/B test to examine whether respondents are more supportive of policy experimentation if they are first told of the results of a previous survey in which a majority of the general public in several European countries expressed generally supportive views about experimentation.
- An A/B test to examine whether respondents have different reactions to the terms “randomised experimentation” or “randomised controlled trials (RCTs)”, when asked whether there should be more or less of this type of experiment in research and innovation policy.
Intervention
The interventions will involve adjustments to the wording of several different questions in the survey – including a conjoint experiment and three A/B tests. Further details are in the attached statistical analysis plan.
Experimental design
The structure of the conjoint (discrete-choice) experiment follows the approach set out by Hainmueller, Hopkins & Yamamoto (2014).
Respondents will be presented with two scenarios for a programme evaluation, each of which consists of a random assortment of the pre-defined levels of each of the six attributes. They will be asked which of the two scenarios they prefer. Each respondent will be asked to make five such pairwise comparisons.
In the A/B tests, each respondent will be randomly assigned to one of two arms. The arms are set out in the attached analysis plan.
Design details
Cited by
No citation data available for this registration yet.