Setting Expectations: Randomizing the Structure of Quarterly Reviews

How to cite

Saouma, Richard; Sandvik, Jason. 2026. "Setting Expectations: Randomizing the Structure of Quarterly Reviews." AEA RCT Registry. AEARCTR-0019320. https://doi.org/10.1257/rct.19320-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Richard Saouma* , Jason Sandvik

* Primary investigator

Registry ID
AEARCTR-0019320
DOI
https://doi.org/10.1257/rct.19320-1.0
First registered
Last updated
Status
In development
Trial dates
2026-04-03 – 2027-12-31
Countries
Private
JEL codes
M12, M54, M50, J24, J28, D23
Secondary IDs
MSU IRB STUDY00011827 -- FWA00004556
Registry record
http://www.socialscienceregistry.org/trials/19320

Abstract

The partner organization is a U.S. credit union of roughly 1,250 employees that conducts quarterly manager–employee performance and development reviews through its HR platform in conjunction with one-on-one meetings. This study is a manager-clustered randomized field experiment testing whether restructuring these reviews around substantially more detailed documentation improves employee outcomes. Treated managers and their teams follow an enhanced protocol prompting both parties to document expectations, progress against competencies and goals, and development discussion in greater detail than the business-as-usual process; control teams continue unchanged.

Randomization was conducted at the team level — a manager together with all direct reports — so each manager delivers the same version of the review to every one of their direct reports. Of 152 teams, 76 are assigned to the enhanced protocol and 76 to business-as-usual, balanced within division on age, tenure, gender, function, and job level; matched to the most recent roster these teams comprise 1,207 regular employees (interns excluded) (597 treated, 610 control). A short survey attached to each review supplies the primary and some secondary outcomes. Two pre-treatment waves are complete; two post-treatment waves follow the intervention, giving a difference-in-differences design layered on the randomization.

The primary outcome is employee-reported expectations clarity: i.e., whether the most recent review strengthened the employee's understanding of what is expected of them. Secondary outcomes are role satisfaction, one-year retention intent, recognition adequacy, survey completion, employee–manager performance-rating calibration, and voluntary attrition. Analysis is intention-to-treat by assigned arm, using a linear probability model (LPM) with standard errors clustered at the team level and two-sided tests.

Intervention

Under the business-as-usual process, each quarterly review is organized around a small set of open-text reflections and the manager's response, with light documentation of the underlying performance and development content. The intervention keeps the quarterly cadence but restructures the pre-review prompts so that managers and employees document the components of the review in substantially greater detail: the expectations set for the period, progress against competencies and goals, development and career discussion, and the link between individual and organizational objectives. Manager- and employee-facing prompts differ. Because the enhanced documentation is embedded in the managerial review process rather than offered as an optional survey, take-up among treated units is expected to be high. All active employees on a treated team receive the treated condition.

Experimental design

Two-arm, parallel, cluster-randomized controlled trial using a difference-in-differences model. The unit of randomization is the team: a manager together with all direct reports, which prevents within-manager contamination. 152 teams were randomized 76/76, balanced within the institution's 11 divisions; matched to the most recent roster these teams comprise 1,207 regular employees (interns excluded) (mean team size 8.0, range 1–19); realized arms are balanced on age, tenure, gender, member-facing versus back-office status, job level, and pay type. Each wave is defined by the review cycle it covers rather than by a calendar cut-off.

Design details

Randomization method
Computer-generated (Stata) pseudo-random assignment executed prior to launch from the organization's roster as finalized at randomization, balanced within division.
Randomization unit
Team — a manager and all of their direct reports.
Clusters
152 teams: 76 treatment, 76 control
Observations
1,207 regular employees (interns excluded), as matched to the most recent roster; the operative headcount at each wave is the roster in force at that wave. The per-wave analysis sample is the subset completing the outcome item; response rate is the binding constraint on power.
Arms
Treatment — enhanced protocol: 76 teams and 597 employees Control — business as usual: 76 teas and 610 employees
Primary outcomes
Employee-reported expectations clarity, measured by the review-linked survey item: “Following your most recent quarterly review, which statement best represents your sentiment regarding what is expected of you at [organization]?” (stronger / unchanged / weaker understanding). The registered primary is the binary indicator: stronger versus not stronger.
Secondary outcomes
● Role satisfaction (0–10) ● One-year retention intent (0–10) ● Recognition adequacy (adequately recognized / recognized but not adequately / not recognized) ● Survey completion ● Employee–manager rating calibration — employee's predicted manager rating vs. the manager's survey rating ● Voluntary attrition — voluntary separations, through 1 Mar 2027 ● Exploratory: Gallup Q12 group-level engagement, especially Q01 (“I know what is expected of me at work”); and internal coherence between self-rated and predicted manager performance ratings

Cited by

No citation data available for this registration yet.

Keywords