Complementarities of AI-Enabled and Human Recommendation to Jobseekers: Experimental Evidence from Kenya

How to cite

Baier, Jasmin. 2026. "Complementarities of AI-Enabled and Human Recommendation to Jobseekers: Experimental Evidence from Kenya." AEA RCT Registry. AEARCTR-0018341. https://doi.org/10.1257/rct.18341-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Jasmin Baier*

* Primary investigator

Registry ID
AEARCTR-0018341
DOI
https://doi.org/10.1257/rct.18341-1.0
First registered
Last updated
Status
On going
Trial dates
2026-04-01 – 2027-07-31
Countries
Kenya (Nairobi, Mombasa, Kitui, Kilifi)
JEL codes
J64, O15, O33, D83, J68, C93, D91, J24, O12
Registry record
http://www.socialscienceregistry.org/trials/18341

Abstract

Career guidance is one of the few scalable tools that targets the information frictions behind occupational mismatch in low- and middle-income countries, yet its welfare value depends on whether it works by informing jobseekers or by persuading them to act. We run a randomized controlled trial with about 4,000 jobseekers in coastal Kenya that compares human-only, AI-only, and hybrid AI-plus-human career guidance, and cross-randomizes whether AI support stops at a recommendation or adds persuasion and action support. Primary outcomes are employment and earnings, match quality, and persistence; we anchor short-run measures to long-run welfare with a surrogate index estimated in external panel data. The design uses AI as a research instrument, fixing the informativeness of advice and the strength of persuasion separately across arms in a way human counselling cannot. We ask whether AI substitutes for or complements human guidance, and whether guidance works by informing or by persuading.

Intervention

Participants are offered structured career guidance, delivered through one of three channels depending on random assignment: a trained near-peer human mentor, an AI-enabled career-guidance tool (Compass), or a combination of the two. A control group receives no additional personalized guidance during the study period.

Human mentors are drawn from Swahilipot Hub and the National Council of Churches of Kenya, and are recruited from the same coastal communities as participants. They provide one-on-one counselling: eliciting skills and preferences, discussing options, recommending a path, and supporting the jobseeker to act on it. Compass (developed by Tabiya) elicits a jobseeker's skills and preferences through a structured conversation, translates informal experience into a structured skill profile, and recommends jobs and career paths matched both to the jobseeker and to local labour demand.

Within the AI-supported arms, the design also varies whether AI support stops once a recommendation has been delivered (information only) or continues with persuasion and action support (planning, encouragement, and follow-through). This separates the informational content of guidance from its persuasive, action-shifting content.

Experimental design

Individually randomized controlled trial with approximately 4,000 jobseekers. Six operational arms pool into four primary categories forming a 2×2 factorial (AI access × human counselling): Control, Human-only, AI-only, and AI+Human. Within the AI-supported arms, a secondary cross-randomization varies whether AI support stops at a recommendation (information only) or adds persuasion/action support. Primary estimands are intention-to-treat effects of the four pooled categories together with the factorial main effects (of AI and of human counselling) and their interaction (the substitutes-vs-complements test). The timing of AI introduction is separately randomized at the mentor level to diagnose counsellor learning and spillovers.

Design details

Randomization method
Randomization is carried out by a reproducible, seeded computer script, run after baseline data collection. The procedure combines blocking with rerandomization for covariate balance (Morgan and Rubin 2012, 2015) and is implemented batch-by-batch as recruitment proceeds, conditioning each batch on all previously randomized participants via cumulative-deficit apportionment of arm totals. The script blocks on geography (county cell), gender, and baseline labour-market state, then rerandomizes on pre-specified baseline predictors of outcomes and attrition, accepting an assignment only if it passes a calibrated, tiered Mahalanobis balance criterion. Confirmatory inference replays this exact mechanism (randomization inference).
Randomization unit
The individual jobseeker. A separate, secondary randomization is conducted at the mentor level (timing of AI introduction: early vs. late, 1:1), used only to diagnose counsellor learning and spillovers, not to define the primary treatment effects.
Clusters
Individual randomization (not clustered): approximately 4,000 individuals. Guidance in the human-contact arms is delivered by approximately 80 near-peer mentors; inference clusters at the counsellor × session level. The secondary mentor-level randomization (AI-introduction timing) involves approximately 80 mentor clusters.
Observations
Approximately 4,000 jobseekers.
Arms
Approximately 4,000 jobseekers, assigned across six operational arms in a 2:1:1:2:1:1 ratio: - T0 Control: 1,000 - T1-R (AI-only, recommendation): 500 - T1-P (AI-only, recommendation + persuasion): 500 - T2-R (AI+Human, recommendation): 500 - T2-P (AI+Human, recommendation + persuasion): 500 - T3 Human-only: 1,000 Pooled into four primary categories of ~1,000 each: Control; Human-only (T3); AI-only (T1-R + T1-P); AI+Human (T2-R + T2-P).
Primary outcomes
Primary outcomes are pre-specified in families defined by domain and by measurement wave, across two waves. (Yet to be refined) Short-run primary (midline, ~4 weeks): 1. Information uptake and decision quality: preference clarity and belief accuracy (pre/post preference elicitation; calibration of subjective probabilities; choice uncertainty / second-order beliefs), consideration-set breadth and calibration, dominated-option avoidance, and search targeting (a search-focus / Herfindahl measure over occupations considered and applied to). 2. Job search and advice take-up: job-search effort and application yield, active vs. passive search (on/off-path search-help decision), information-bundle choice, implementation of the recommended path and action-plan completion, and the subjective probability of pursuing the recommended occupation. Longer-run primary (endline, ~6-9 months): 3. Economic opportunity: employment or income-generating activity (incl. a count of IGAs), earnings, hours worked, number of income sources, job applications, interviews, offers, and participation in training. 4. Match quality and persistence: skill-occupation alignment (KeSCO 2- and 3-digit), targeting of search toward a coherent occupation or sector, occupational/sectoral persistence over time, tenure and work-history length, and work satisfaction. 5. Welfare and subjective well-being: consumption, savings, perceived agency, and life satisfaction. Long-run targets: occupation/industry persistence (2-digit ISIC) and total real earnings ~10 years out, via a pre-specified surrogate index (Kenya Life Panel Survey). The two short-run families map to the two channels the design separates: information (uptake, decision quality, search focus) and persuasion (take-up, effort). They are primary both because the intervention targets them most directly (proximal behavioural responses) and because they identify the mechanism (RQ2). The endline families test whether these proximal effects translate into distal labour-market and welfare gains.
Secondary outcomes
(Yet to be refined) Additional midline mechanism measures, each its own family: incentivized revealed preferences and WTP (counselling-format WTP; stepping-stone vignette; stated usefulness and a recommendation-regret index) ; beliefs and expectations, extended (top-option subjective probabilities; pre/post preference-shift magnitude); decision-process robustness (stated-chosen consistency; choice stability under re-elicitation); time use and substitution (displaced leisure); agency and discouragement (locus of control, self-efficacy, fatigue, aspirations, controllability slope). Secondary treatment estimands: CACE; recommendation-vs-persuasion contrasts (T1-P vs. T1-R; T2-P vs. T2-R); "any-AI vs. control" and "any-human vs. control" pooled policy contrasts; counsellor-side mechanism outcomes.
Sponsors
Name: J-PAL Location: United States Url: https://www.povertyactionlab.org/initiative/project-ai-evidence-rfp
Partners
Name: Swahilipot Hub Partner_type: ngo Url: https://swahilipothub.co.ke/ Name: NCCK Kenya Partner_type: ngo Url: https://swahilipothub.co.ke/ Name: Tabiya Partner_type: ngo Url: https://tabiya.org/

Cited by

No citation data available for this registration yet.

Keywords