Using Informational Videos to Elicit Participation in Online Survey Research: Evidence from a Randomized Controlled Trial

How to cite

Pedersen, Mogens Jin; Pontoppidan, Maiken. 2017. "Using Informational Videos to Elicit Participation in Online Survey Research: Evidence from a Randomized Controlled Trial." AEA RCT Registry. AEARCTR-0002541. https://doi.org/10.1257/rct.2541-2.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Mogens Jin Pedersen* , Maiken Pontoppidan

* Primary investigator

Registry ID
AEARCTR-0002541
DOI
https://doi.org/10.1257/rct.2541-2.1
First registered
Last updated
Status
Completed
Trial dates
2017-11-01 – 2017-12-31
Countries
Denmark ()
Registry record
http://www.socialscienceregistry.org/trials/2541

Abstract

Survey research is a widely used method for collecting data about a population of interest. A high survey response rate mitigates concerns about non-response bias and is thus a significant component for the validity of survey-based findings. At the same time, response rates to surveys have been in general decline in recent decades, and survey data are increasingly collected online using web-based questionnaires. Although online surveys comprise a relatively fast and inexpensive way to collect information from people, this development amplifies the problem of low response rates: The response rates in online surveys are typically lower than those in surveys using other data collection methods (i.e., postal questionnaires and face-to-face interviews).

Identifying the strategies that investigators may use to maximize survey response rates in online surveys is thus important. While survey researchers have examined a range of strategies for increasing survey response rates, one particular approach appears to be understudied: The use of a video in the survey invitation email.

This study examines the effects of two video treatments on participation in a nation-wide online survey among Danish parents. During a trial period of two months, a sample of Danish parents to a child (aged 9 months, 24 months, or 36 months) receive an email invitation to participate in an online survey. Both parents of 15,000 children—5,000 in each child age group—will be invited to participate (30,000 parents). Clustered at the child level, each parent couple is randomly assigned to either a control group (no video) or one of two treatment groups (standard or enhanced video). Parents in the treatment groups receive the same email invitation as those in the control group—with one important exception: The email includes a link to a short informational video. The video shows an investigator explaining the purpose of the survey and the prosocial value of the individual’s participation, thus seeking to elicit participation by personifying the task of survey response and increasing perceived task significance. The two video treatments differ in terms of visual content: A “standard” video treatment simply shows the principal survey investigator (low cost), whereas the “enhanced” video treatment involves a set of cover shots meant to supplement and visualize the audio content (high cost). We examine and compare the effects of the video treatments on the survey response rate, response speed, survey dropout rate, survey completion time, survey response patterns, and participation in subsequent survey waves. We also test for heterogeneous effects by socioeconomic characteristics (parent’s sex, ethnicity, age, and educational attainment; child’s sex, age, and “at risk” status).

Intervention

The interventions consist of two video treatments embedded into a survey invitation email. Both videos show an investigator explaining the purpose of the survey and the prosocial value of the individual’s participation, thus seeking to elicit participation by personifying the task of survey response and increasing perceived task significance.

Treatment 1 (standard video): A video showing the principal survey investigator. Video production costs are relatively low.

Treatment 2 (enhanced video): A video containing the same audio content as Treatment 1, but involving a set of cover shot that supplements and visualizes the audio content. While video production costs are higher than those for Treatment 1, the video is expected to be more visually appealing to the audience.

Experimental design

The study involves a three-arm, parallel cluster randomized controlled trial carried out on a nationwide sample comprising both parents of a total of 15,000 children in Denmark (30,000 parents).

The trial period is two months. Each month, the independent organization Statistics Denmark draws a sample of 7,500 children having turned 9, 24, or 36 months in the previous month (2,500 children in each child age group). Each child age group will comprise a random sample of 1,500 “at risk” children and a random sample of 1,000 children from the remaining population.

“At risk” children are defined by the characteristics of the mother—i.e., whether the child’s mother satisfies one of the following four criteria at the time of the child’s birth: (1) age 21 or less, (2) lower secondary education as the highest completed education, (3) fit to work but unemployed for one year or more during last three years, or (4) not cohabiting with the child’s father.

Clustered at the child level, each eligible parent couple is randomly assigned to either a control group (no video) or one of two treatment groups (Treatment 1 or 2).

Design details

Randomization method
Randomization is carried out by simple randomization by computer (randomization based on a single sequence of random assignments). Stratified by trial month, child age group, and child status (“at risk or not), Statistics Denmark will randomly assign one third of eligible (parent couples to one of three conditions: (1) no video (Control), (2) standard video (Treatment 1), or (3) enhanced video (Treatment 2).
Randomization unit
Stratified by trial month, child age group, and child status (“at risk or not), the individual child is the randomization unit. For each child, both parents receive identical treatment.
Clusters
15,000 children/parent couples
Observations
30,000 parents
Arms
C: 5,000 child/parent couples (10,000 parents) T1: 5,000 child/parent couples (10,000 parents) T2: 5,000 child/parent couples (10,000 parents) Sample size analysis (power = .8 and two-sided alpha level at .05) suggests this sample size (n = 15,000) will allow for identification of effects of size .05 or larger (Cohen’s d).
Primary outcomes
The survey response rate (parent level), current and subsequent waves.
Secondary outcomes
Response speed (time from the survey invitation email is sent out to survey completion) Survey dropout rate (proportion of respondent beginning but not completing the survey) Survey completion time (time from beginning the survey to completing the survey) Survey response patterns (number of “do not know” or valid item responses)

Papers from this trial

  1. Mogens Jin Pedersen, Anders Bo Bojesen, Signe Boe Rayce, Maiken Pontoppidan, Using Informational Video to Elicit Participation in Online Survey Research: A Randomized Controlled Trial, International Journal of Public Opinion Research, Volume 33, Issue 4, Winter 2021, Pages 946–959, https://doi.org/10.1093/ijpor/edaa023 https://academic.oup.com/ijpor/article-abstract/33/4/946/6006402

Cited by

No citation data available for this registration yet.

Keywords