The Impact of Personalization and SMS Reminders on the Uptake of Customer Satisfaction Surveys in Government Services Centers
How to cite
Osseiran, Ali; Saleh, Nabil; Makki, Fadi. 2024. "The Impact of Personalization and SMS Reminders on the Uptake of Customer Satisfaction Surveys in Government Services Centers." AEA RCT Registry. AEARCTR-0014592. https://doi.org/10.1257/rct.14592-2.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
Customer satisfaction surveys are essential tools for government agencies, providing a cost-effective and efficient method for gathering large-scale feedback on service quality. These surveys help identify customer pain points and enhance the overall user experience. However, low response rates present a significant challenge, impacting both the quality of the data collected and the depth of actionable insights that can be derived. An analysis of over 400,000 web-based surveys conducted across seven Government Service Centers (GSCs) from July 2023 to July 2024 illustrated this issue, revealing a low completion rate of approximately 9%. Over 80% of these responses were received within the first hour of sending the SMS invite.
To address this challenge, a randomized controlled trial (RCT) was implemented to test the effectiveness of behavioral insights in increasing survey participation rates. The intervention employed personalized SMS invitations and timely reminders to encourage survey completion following visits to the GSCs. The study consisted of four treatment arms: a control group receiving the original generic SMS, a group receiving personalized SMS invites, another group receiving the original SMS with timely reminders, and a final group that combined both personalized invites and reminders.
The intervention will run for two weeks (10 working days) across four Government Service Centers (GSCs) serving approximately 1,500 customers daily (60% of whom are first-time visitors). It aims to test and develop a scalable cost-effective solution for enhancing customer feedback mechanisms in public service environments.
Intervention
The intervention involved sending personalized SMS invitations to customers shortly after they received a service at one of four participating GSCs, inviting them to complete a web-based survey rating their experience at the center. The messages were customized to make them more relevant and engaging, and were sent in both Arabic and English to accommodate language preferences.
The intervention also included sending reminders the day after the initial SMS invite, scheduled for 7 PM. These reminders were sent to customers who had not yet completed the survey, prompting them again to participate. Like the original invitation, the reminders were personalized and delivered in both Arabic and English.
Treatment Arms
The trial utilized a randomized controlled design comprising four treatment arms:
- Control Group: Received the original generic SMS invitation.
- Personalized SMS Group: Received personalized SMS invitations tailored to their specific interactions with the government service.
- Original SMS + Reminder Group: Received the original SMS invitation along with a follow-up reminder sent the following day at 7 PM.
- Personalized SMS + Reminder Group: Received both the personalized SMS invitations and a follow-up reminder.
Experimental design
The study employed a randomized controlled trial (RCT) with a factorial design to evaluate the effectiveness of the personalized SMS invite and follow-up reminder in increasing participation rates in the customer satisfaction surveys. This design allowed for assessing the effect of the generic and personalized invites separately as well as in combination with the follow-up reminders.
Key Components of the Experimental Design
Treatment Arms:
The trial comprised four treatment arms:
- Control Group: Received the original generic SMS invitation.
- Personalized SMS Group: Received personalized SMS invitations tailored to their specific interactions with the government service.
- Original SMS + Reminder Group: Received the original SMS invitation along with a follow-up reminder sent the following day at 7 PM.
- Personalized SMS + Reminder Group: Received both the personalized SMS invitations and a follow-up reminder.
Randomization:
Customers visiting one of the four participating GSCs were randomly assigned to one of four intervention arms. This randomization was achieved through a simple randomization method, whereby each customer was assigned a random number between 1 and 4 upon checking into a GSC to request a government service. Each number corresponded to a specific treatment arm.
To ensure consistency in assignment, customers were tracked using the local mobile numbers provided during check-in. This tracking mechanism allowed for repeat visitors to remain in the same intervention group throughout the trial period.
Sample Size:
The study aimed to include approximately 10,000 customers, with each treatment arm consisting of about 2,500 customers. This sample size was deemed sufficient to detect meaningful differences of 2 percentage points in response rates between the groups.
The trial anticipated collecting around 15,000 survey observations, which included responses from first-time visitors as well as repeat customers.
Duration of the Trial:
The intervention will be implemented over a two-week period, with data collection concluding 10 days afterward, providing sufficient data to achieve the desired statistical power.
Data Collection:
Surveys were administered through web-based links sent via SMS. Each customer received a unique link to participate in the survey, which expired 10 days after the invitation was sent.
Outcome Measures:
The primary outcome measure was the completion rate of the customer satisfaction surveys.
Design details
Cited by
No citation data available for this registration yet.