Question Order and Expectation Formation: Experimental Evidence from a Business Survey
How to cite
Wohlrabe, Klaus; Demmelhuber, Katrin. 2026. "Question Order and Expectation Formation: Experimental Evidence from a Business Survey." AEA RCT Registry. AEARCTR-0018942. https://doi.org/10.1257/rct.18942-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Survey-based measures of firm expectations are key inputs to leading economic indicators, yet expectations must be elicited through survey instruments whose design may itself shape responses. This study provides randomized experimental evidence on whether the position of expectation questions within a business-survey questionnaire affects measured expectations. Firms in the ifo Business Survey are randomly assigned, within strata defined by firm size and sector, to one of three arms that differ only in question order. In the control arm (A), expectation questions follow a block of neutral filler questions. In treatment arm B, expectation questions follow a block of substantive firm-related questions on operating costs, skilled-labor availability, and competitive intensity. In treatment arm C, the same expectation questions are asked at the very beginning of the questionnaire, before any other firm-related content. Question wording and response scales are identical across arms; only block order differs. The primary estimand is the pure question-order effect, the difference in expectations between arms B and C. We analyze three outcomes: qualitative expectations about the firm's own business situation, qualitative expectations about the overall German economy, and a quantitative turnover expectation. Secondary contrasts against the control arm (B vs. A, C vs. A) separate content-specific priming from positioning/attention effects.
Intervention
The intervention is a manipulation of question order within the questionnaire of the ifo Business Survey; the informational content, question wording, and response scales are held identical across all respondents. Firms are randomly assigned, within strata defined by firm size and sector, to one of three arms that differ only in the sequence of three question blocks: an expectation block (qualitative expectations about the firm's own business situation and about the German economy, plus a quantitative turnover expectation), a substantive attribute block (operating costs, skilled-labor availability, and competitive intensity, each with a retrospective and a forward-looking item), and a neutral control block matched to the attribute block in number of items, response format, and cognitive demands but covering topics unlikely to prime business-cycle considerations (digitalization of administrative and billing processes, company-provided training, and predictability of operational processes).
In the control arm (A), respondents answer the neutral filler block first and the expectation block afterwards. In treatment arm B, respondents answer the substantive attribute block first and the expectation block at the end, so that they reflect on concrete business conditions before reporting expectations. In treatment arm C, respondents answer the expectation block at the very beginning of the questionnaire, before any other firm-related content, with the attribute block following afterwards. Within each block, the display order of topics is randomized to avoid within-block ordering effects, while the internal order of the three expectation items is fixed across all arms. The contrast between arms B and C isolates the pure question-order effect, since both answer the same questions in reverse order; the control arm serves as a diagnostic benchmark for separating content-specific priming from positioning and attention effects.
Experimental design
We implement a three-arm, between-subjects randomized survey experiment embedded in the regular ifo Business Survey. Respondents are randomly assigned, within strata defined by firm size and sector, to one of three arms that differ only in the order in which three question blocks are presented: a block of expectation questions, a block of substantive firm-related attribute questions, and a neutral filler block. Question content, wording, and response scales are identical across arms; only the block order varies. In the control arm, the neutral filler block precedes the expectation block; in the first treatment arm, the attribute block precedes the expectation block (expectations at the end); in the second treatment arm, the expectation block comes first, before any other firm-related content. Within each block the order of topics is randomized, while the internal order of the expectation items is fixed across arms.
The primary comparison is between the two treatment arms, which answer the same questions in reverse order and thereby isolate the effect of the position of the expectation block. Comparisons of each treatment arm against the control arm serve diagnostic purposes, helping to separate content-specific priming from positioning or attention effects. Outcomes are the reported expectations, analyzed separately for each expectation dimension; treatment effects are estimated as differences in mean outcomes across arms.
Design details
Cited by
No citation data available for this registration yet.