Robo-Advising Meets Large Language Models: Educating Investors on Alpha and Beta of Mutual Funds and Stocks
How to cite
Lu, Fangzhou. 2025. "Robo-Advising Meets Large Language Models: Educating Investors on Alpha and Beta of Mutual Funds and Stocks." AEA RCT Registry. AEARCTR-0015716. https://doi.org/10.1257/rct.15716-1.0
Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.
* Primary investigator
Abstract
We will utilize a robo-advising setup to investigate the impact of financial literacy on investors, specifically focusing on the concepts of beta and alpha. Partnering with a large brokerage firm, we will study the effects of educating investors about beta and alpha in mutual funds and stocks. To enhance investors’ understanding, we will integrate a back-end language model (ChatGPT/DeepSeek/Doubao/Kimi) system with the robo-advisor. We aim to determine whether learning about beta influences investors to prefer investing in ETFs or index funds rather than directly in active mutual funds or individual stocks. We will also examine whether a one-click automatic enrollment mechanism has a larger impact compared to a self-assembled portfolio, and if this gap is smaller for investors supported by the language model. Additionally, we will explore whether the language model-based chatbot prompts investors to pursue international diversification even when these options are not initially recommended by the robo-advisors but are still available. Our goal is to assess the long-term improvement in financial literacy and the potential benefits for investors, such as increased returns and gains.
Intervention
To investigate the impact of financial literacy on investor behavior and performance, we are conducting a Randomized Controlled Trial (RCT) involving 28,819 investors who expressed interest in participating. We randomly selected 14,000 of these investors to form the treatment group, while the remaining 14,819 investors constitute the control group. Both groups are similar in terms of investment portfolio size and personal characteristics, ensuring a balanced and fair comparison.
In the treatment group, investors receive targeted financial education on the concepts of beta and alpha through our robo-advising platform. The education module provides detailed explanations, examples, and personalized insights into how these metrics influence investment risk and performance. Investors in the control group continue to receive standard robo-advisory services without the additional financial literacy component. Moreover, a subsample of the treatment group is further enhanced with a back-end Large Language Model (LLM) system or other follow-up assistance for interactive learning.
The robo-advisor begins by evaluating the investor’s current portfolio holdings, systematically assessing each mutual fund or stock by calculating key metrics such as alpha and beta. It performs a multi-factor analysis using a comprehensive set of factors, including size (SMB), value (HML), and momentum (MOM), among others. Statistical measures, such as t-statistics, are computed to determine the significance of alpha, distinguishing genuine manager skill from random fluctuations. The robo-advisor aggregates these metrics to provide an overall assessment of the portfolio, benchmarking against market indices to provide context on the portfolio’s performance and volatility.
Based on this detailed analysis, the robo-advisor generates personalized recommendations for the investor. These recommendations aim to optimize the portfolio by suggesting actions such as reallocating funds from underperforming assets, maintaining positions in high-performing assets, and introducing low-cost, diversified investment vehicles like ETFs or index funds. The ultimate goal is to help the investor achieve better returns, manage risk more effectively, and minimize investment costs.
Experimental design
Experimental Design (Public) This study is a Randomized Controlled Trial (RCT) conducted in collaboration with one of the largest brokerage firms in China. The objective is to investigate the impact of financial literacy on investor behavior and performance. The trial involves 28,819 investors who expressed interest in participating. The experimental design is as follows:
Participant Allocation: We randomly selected 14,000 of these investors to form the treatment group, while the remaining 14,819 investors constitute the control group. The randomization was performed using a computer algorithm to ensure transparency and reproducibility.
Interventions: Treatment Group: Investors in the treatment group receive targeted financial education on the concepts of beta and alpha through our robo-advising platform. The education module includes detailed explanations, examples, and personalized insights into how these metrics influence investment risk and performance. This group is further divided into four subgroups to examine the heterogeneity of the intervention’s effects:
ChatGPT/DeepSeek (LLM) Chatbot: Receives financial education and personalized advice through an LLM-powered chatbot.
Without Chatbot: Receives financial education through static content such as articles and videos, without interactive chatbot support.
Human Chat Assistant: Has access to human financial advisors who provide real-time assistance and explanations via chat.
Normal Chatbot: Uses a standard rule-based chatbot that provides predefined responses and guidance based on a fixed set of rules.
Additionally, each subgroup is divided into two groups:
Automatic Execution: This group benefits from automatic execution or one-click execution of the recommended portfolio.
DIY (Do It Yourself): This group can only execute the recommended portfolio by doing it themselves.
Control Group: Investors in the control group continue to receive standard robo-advisory services without the additional financial literacy component.
Assessment and Monitoring: The robo-advisor begins by evaluating the investor’s current portfolio holdings, systematically assessing each mutual fund or stock by calculating key metrics such as alpha and beta. It performs a multi-factor analysis using a comprehensive set of factors, including size (SMB), value (HML), and momentum (MOM), among others. Statistical measures, such as t-statistics, are computed to determine the significance of alpha.
Based on this detailed analysis, the robo-advisor generates personalized recommendations for the investor. These recommendations aim to optimize the portfolio by suggesting actions such as reallocating funds from underperforming assets, maintaining positions in high-performing assets, and introducing low-cost, diversified investment vehicles like ETFs or index funds.
Outcome Measurement: Throughout the study period, we closely monitor the investment decisions and performance outcomes of all subgroups in the treatment group, as well as the control group. The key outcome variables include changes in portfolio composition, trading activity, investment performance, risk management, financial literacy improvement, behavioral changes, and international diversification.
This experimental design allows us to rigorously evaluate the causal impact of financial literacy on investor behavior and performance, providing valuable insights into how educational interventions can enhance investment outcomes. The results of this trial will contribute to the broader understanding of financial literacy's role in investment decision-making and its potential benefits for investors.
Design details
Cited by
No citation data available for this registration yet.