Metacognitive awareness and academic performance: Evidence from randomized controlled trials

How to cite

Viceisza, Angelino; Apperson, Jarod; Blankson, A. Nayena; Guillory, Jimmeka; Jackson, Francesina. 2019. "Metacognitive awareness and academic performance: Evidence from randomized controlled trials." AEA RCT Registry. AEARCTR-0004840. https://doi.org/10.1257/rct.4840-1.1

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Angelino Viceisza* , Jarod Apperson , A. Nayena Blankson , Jimmeka Guillory , Francesina Jackson

* Primary investigator

Registry ID
AEARCTR-0004840
DOI
https://doi.org/10.1257/rct.4840-1.1
First registered
Last updated
Status
On going
Trial dates
2016-01-01 – 2027-05-31
Countries
Private
JEL codes
A22, C93, D91, I23, J0
Registry record
http://www.socialscienceregistry.org/trials/4840

Abstract

Enhancing student awareness of their own thinking and learning could lead to improvements in academic success and persistence in their courses of study. This project assesses whether (1) faculty/peer-tutors respond to metacognitive training and (2) there is an impact of metacognitive teaching methods in the classroom on a sample of female students of African descent at Spelman College by means of two randomized controlled trials (RCTs) over the course of three years. The first RCT assesses whether (1) faculty respond to metacognitive training and (2) there is an impact of metacognitive teaching methods in the classroom during academic years (AY) 2016-17 and 2017-18. The second RCT continues the former intervention while overlaying metacognitive strategies via peer-tutoring during AY 2018-19. In other words, RCT 2 implements a (2x2) design that enables us to test whether metacognitive strategies communicated in the classroom are complementary to those disseminated through peer-tutoring.

This registry/submission (October 2019) comprises the following main outcomes: (1) GPA excluding the course where the intervention is taking place; (2) a metacognitive outcome based on (a) the generalized version of the Motivated Strategies for Learning Questionnaire (MSLQ), (b) the components of the Metacognition Awareness Index (MAI) that assess metacognitive knowledge, and (c) the components of the MAI that assess regulation; and (3) persistence/retention. It is being submitted in order to analyze one set of research questions (as outlined in the funded proposal) in time for the Spelman FitW dissemination conference, which will take place on November 2, 2019. See https://www.spelman.edu/fitw.

A future update/edit to this registry/submission will comprise a separate set of research questions to be analyzed after the conference.

Intervention

1. Intervention strategies that are targeted at first year programs and peer-tutoring services could make significant impact on college student outcomes. Therefore, we designed RCTs that would target both first year programs and peer-tutoring. We assess the impact of metacognitive training on faculty/peer-tutors and in turn, metacognitive teaching (RCT 1) and peer-tutoring (RCT 2) on students enrolled in a signature first year course -- African Diaspora and the World (ADW) -- at Spelman College.

2. ADW is a two-semester course that all first-year students at Spelman must take. The aim of the course is for students to examine major themes associated with the African Diaspora within a global context and from perspectives that are both interdisciplinary and gender-informed. Faculty share a syllabus, assign common readings, administer common assessments (e.g., map quiz and a final exam), and give common assignments (e.g., reading logs, museum audio narrative, and an artistic/performance group project). The course introduces complex concepts that are new to many students. Students are required to read academically difficult materials that often challenge their current perspectives or worldview. Yet, many students find the course interesting because it places them, as Black women, at the center of the content they are studying.

3. Metacognitive instruction is a low-cost, potentially high-impact intervention that requires no change to campus infrastructure, curriculum or degree programs. It involves a number of different strategies. The main strategy used in this intervention is reciprocal teaching. Faculty/peer-tutors were trained on metacognition. In turn, while reading, the instructor/peer-tutor teaches students how to (1) generate questions, (2) summarize in their own words, (3) clarify word meanings, and (4) predict what might appear next. Instructors/peer tutors use a scaffolding approach. That is, they initially model the four-stage process, with students gradually assuming greater responsibility, culminating with a dialogue between the instructor and student, and student and student.

Experimental design

1. The first RCT, which is implemented during AY 2016-17 and AY 2017-18, is focused on assessing whether (1) faculty respond to metacognitive training and (2) there is an impact of metacognitive teaching methods in the classroom. That is, ADW faculty are randomly assigned to treatment (T) and control (C) conditions. T faculty receive metacognitive training at the beginning of each AY with periodic "reminders" (e.g., in ADW faculty meetings) while C faculty receive placebo training. The latter focuses on aspects such as use of classroom technologies and learning management software (e.g., Moodle) as well as dealing with student pushback. The two cohorts of first year students for AYs 2016-17 and 2017-18 are randomly assigned to T or C conditions, which in turn implies that they are assigned to T or C classrooms, pegged to T or C faculty.

2. The second RCT, which is implemented during AY 2018-19, is focused on assessing the impact of metacognitive instruction in the classroom and/or via peer-tutoring. That is, we implement a (2x2) design by overlaying the peer-tutor conditions on top of the faculty conditions. Peer-tutors are T or C based on their original assignment as a student in RCT 1 (thus receiving "booster" training depending on their condition) and randomly assigned to a T or C faculty member. This leads to four conditions: T-T, T-C, C-T, or C-C, where the first letter represents the faculty's condition and the second represents the peer-tutor's condition. The cohort of first year students for AY 2018-19 is randomly assigned to one of the above four conditions, which in turn implies that they are assigned to T or C classrooms, pegged to T or C faculty, and an associated weekly peer-tutoring session that is either T or C. Formally, the peer-tutors were referred to as peer recitation facilitators in order to reduce the potential stigma associated with "tutoring".

3. Each cohort of first year students includes a small number of transfer, non-traditional, and repeat students. These students are also assigned to T and C conditions (as explained above); however, they are not supposed to be in the same classroom or peer-tutoring session as a "standard" first year student. This is for two reasons. First, the intervention is intended to test the potential impact of MC on a typical incoming student. Second, since we are planning to exclude atypical students from the analysis, What Works Clearinghouse guidelines indicate that they should be assigned to separate classrooms; otherwise, that could be counted as attrition. That is, while also assigned randomly, atypical students are "outside" of the RCTs in question from a data analysis standpoint.

Design details

Randomization method
Randomization is done in office by a computer. It is typically done by external evaluators.
Randomization unit
RCT 1: Student-level and classroom/faculty-level. RCT 2: Student-level and classroom/faculty-level.
Clusters
RCT 1: 22 classrooms (11 T and 11 C) for each cohort. So, there are a total of 22 T and 22 C clusters across two cohorts (although there are "repeat" T and C faculty across the two cohorts). See "Power calculation" below. Each AY, two additional T and two additional C classrooms are reserved for the atypical students described previously. RCT 2: 22 classrooms (6 CC, 5 CT, 7 TC, 4 TT) for AY 2018-19, where the first letter refers to the faculty condition and the second letter refers to the peer tutor condition. In addition, 1 CC classroom, 1 CT classroom, and two TT classrooms were reserved for the atypical students described previously.
Observations
RCT 1: 800-900 students across two cohorts. RCT 2: 400-500 students.
Arms
See "Planned Number of Clusters".
Primary outcomes
(1) GPA excluding the course where the intervention is taking place; (2) a metacognitive outcome based on (a) the generalized version of the MSLQ, (b) the components of the MAI that assess metacognitive knowledge, and (c) the components of the MAI that assess regulation; and (3) persistence/retention.
Secondary outcomes
A second set of research questions will be registered after the November 2 conference, see https://www.spelman.edu/fitw. At that point, we will discuss additional outcome measures.

Cited by

No citation data available for this registration yet.

Keywords