The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment

How to cite

Karlan, Dean; Wood, Daniel H.. 2017. "The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment." AEA RCT Registry. AEARCTR-0001849. https://doi.org/10.1257/rct.1849-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Dean Karlan* , Daniel H. Wood

* Primary investigator

Registry ID
AEARCTR-0001849
DOI
https://doi.org/10.1257/rct.1849-1.0
First registered
Last updated
Status
Completed
Trial dates
2007-06-01 – 2014-04-01
Countries
United States of America (North America)
JEL codes
D64, H41, L31, O12
Registry record
http://www.socialscienceregistry.org/trials/1849

Abstract

We test how donors respond to new information about a charity's effectiveness. Freedom from Hunger implemented a test of its direct marketing solicitations, varying letters by whether they include a discussion of their program's impact as measured by scientific research. The base script, used for both treatment and control, included a standard qualitative story about an individual beneficiary. Adding scientific impact information has no effect on average likelihood of giving or average gift amount. However, we find important heterogeneity: large prior donors are both more likely to give and also give more, whereas small prior donors are less likely to give. This patter is consistent with two different types of donors: warm glow donors who respond negatively to analytical effectiveness information, and altruism donors who respond positively to such information.

Intervention

The researchers conducted a randomized controlled trial to test how charity donors respond to information about a charity's effectiveness. The RCT was conducted in two waves in June 2007 and October 2008. Partnering with Freedom from Hunger, the researchers randomly assigned the recipients of a direct-mail solicitation campaign to receive a donation renewal letter which featured an emotional appeal (control group), or a donation renewal letter which included the same emotional appeal as well as evidence from a randomized trial on the charity's positive impact (treatment group).

The researchers developed a model in which charitable donors have two motivations for donating to charity: altruism, whereby the donor gains utility by social welfare gain, and warm-glow, whereby the donor gains utility from the act of donating. In this model, donation size can serve as a proxy for distinguishing altruistic donors from warm-glow donors. In addition, altruistic donors should respond favorably to information about a charity's effectiveness, while warm-glow donors may find the emotional appeal of donating lessened by the additional information.

The researchers found no effects on average likelihood to donate or on amount donated. However, the average effect masked heterogeneous effects on large donors (>$100) versus small donors (<$100). Large donors exposed to treatment were both more likely to donate and donated more, whereas small donors exposed to treatment were less likely to donate and donated less on average.

Experimental design

In the first wave, mail solicitations were sent to 16,889 individuals who had donated to FFH in the previous three years. Recipients were randomized into a control group of 11,258 and a treatment group of 5,631. The control group was divided into two subgroups of 5,628 and 5,630. Randomization was stratified by the most recent donation year and whether the last donation was more or less than $100. The control group consisting of 5,628 recipients received a standard donation renewal letter, while the control group consisting of 5,630 recipients received a standard renewal letter with an additional insert describing the emotional story of a participant in one of FFH's programs in Bolivia. The treatment group received the standard donation renewal letter and the same insert, except the final paragraph was replaced with one detailing the positive impact of the program as demonstrated by "rigorous scientific methodologies".

In the second wave, mail solicitations were sent to 17,784 individuals who had donated to FFH in the previous three years. Recipients were randomized into a control group of 5,960 and two treatment groups of 5,903 and 5,921. Randomization was stratified by whether the last donation was more or less than $100, treatment assignment in the first wave, and whether the recipient had donated in 2008. The control group received the standard donation renewal letter with a personal story and emotional appeal. The treatment group of 5,903 received the same renewal letter, except additional paragraphs were added which described the positive impact of the programs. The treatment group of 5,921 received the same renewal letter, but the information about the positive impacts of the programs explicitly cited Yale-affiliated researchers.

Design details

Randomization method
Randomization done in office by computer
Randomization unit
First wave: randomized at individual level, stratified by prior giving amount and recency of donation Second wave: randomized at individual level, stratified by prior giving amount, whether they donated the previous year, and treatment status in first wave
Clusters
First wave: 16,889 individuals Second wave: 17,784 individuals
Observations
First wave: 16,889 individuals Second wave: 17,784 individuals
Arms
First wave: 5,631 individuals in treatment group; 11,258 individuals in control group Second wave: 5,903 individuals in first treatment group; 5,921 individuals in second treatment group; 5,960 individuals in control group
Primary outcomes
Making any donation (within 5 months of receiving the mailer); Donation amount (within 5 months of receiving the mailer)

Papers from this trial

  1. Karlan, Dean, and Daniel H. Wood. "The Effect of Effectiveness: Donor Response to Aid Effectiveness in a Direct Mail Fundraising Experiment." Working Paper, June 2015. http://www.nber.org/papers/w20047

Cited by

No citation data available for this registration yet.

Keywords