Assessing the effectiveness of significant team-based learning application exercises relative to traditional practice problems in TBL principles of microeconomics classrooms

How to cite

Silz-Carson, Katherine; Ruder, Philip; Maier, Mark; Simkins, Scott. 2019. "Assessing the effectiveness of significant team-based learning application exercises relative to traditional practice problems in TBL principles of microeconomics classrooms." AEA RCT Registry. AEARCTR-0005225. https://doi.org/10.1257/rct.5225-1.5

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Katherine Silz-Carson* , Philip Ruder , Mark Maier , Scott Simkins

* Primary investigator

Registry ID
AEARCTR-0005225
DOI
https://doi.org/10.1257/rct.5225-1.5
First registered
Last updated
Status
Completed
Trial dates
2019-09-02 – 2020-04-30
Countries
United States of America (Multiple locations across the U.S.)
JEL codes
A2
Registry record
http://www.socialscienceregistry.org/trials/5225

Abstract

The purpose of this study is to assess, using a multi-site randomized controlled trial, whether in-class exercises designed based on the "4-S" principles of the team-based learning pedagogy are more effective at developing student learning in college principles of microeconomics classes than traditional principles of microeconomics practice problems in those classes. Assignment to treatment and control exercises is randomized at the site and module level, so that all students, sites and instructors are exposed to both the treatment and the control. Effectiveness is measured via a low-stakes end-of-block assessment that is common for each module across all sites.

Intervention

Interventions are treatment and control versions of application exercises (AEs) to be used in four modules (supply & demand, firm costs & competitive markets, imperfect competition, and externalities) of principles of microeconomics courses at each study site. Treatment AEs employ the "4-S" (exercises involve a Significant problem, all teams work on the Same problem, all teams are required to make a Specific choice, and all teams report Simultaneously) methodology that is part of the Team-Based Learning (TBL) pedagogy. Control AEs are standard end-of-chapter problem-type exercises that appear in most principles of microeconomics textbooks. All treatment AEs are available for use at: https://serc.carleton.edu/econ/tbl-econ/activities.html. (There are other AEs posted on this site besides the four used for this study.) For each module, participating sections of principles of microeconomics courses will incorporate the following common materials: either a treatment or control in-class exercise and a module post-test. Treatment and control modules are randomly assigned at the site-module level, so that all students at each site are exposed to two treatment modules and two control modules.

Experimental design

Each site is randomly assigned to two treatment modules and two control modules. At each site, class proceeds as it normally does except that for the experimental modules, students complete either the treatment or control exercise (depending on the site's random assignment for that module) and take a common low-stakes assessment of their learning for that module. Student's answers to each question on the assessment are recorded for analysis.

Design details

Randomization method
Assignment to treatment and control is conditionally random. Conditions to be satisfied are: (1) Each site has two treatment modules and two control modules (2) All sections at a site have the same assignment (either treatment or control) for a given module Given these constraints and the fact that there are four modules, there are six possible patterns to use to randomly assign sites to a set of treatment and control modules: TTCC, CCTT, TCTC, CTCT, TCCT, CTTC. The randomization procedures was as follows: (1) Sites are matched together so that the number of students enrolled at one site is roughly equal to the number of students enrolled at another site. For example, if one site has a projected enrollment of 35 students, a second site has a projected enrollment of 37 students, and a third site has a project enrollment of 70 students, then the matching algorithm would place sites 1 and 2 into the opposite randomization pattern as site 3. (2) One half of each site match is randomly assigned a pattern via a computerized random number draw that simulates the roll of a 6-sided die. Once one half of the match is assigned a pattern, the other half's randomization pattern is set.
Randomization unit
Unit of randomization is a site. Random assignment is to a pattern of 2 treatment and 2 control modules for that site.
Clusters
Clustering is at the section level (some sites have multiple sections). Original plan was to have 8 sites and 13 total clusters. We lost one site with two sections part-way through the semester (after randomization was completed). Thus, the final number of sites is 7, with 11 total clusters.
Observations
An observation is a student-module combination. The original plan with 8 sites resulted in 1560 student-module combinations. After losing a site, it is expected that approximately 1400 student-module combinations remain.
Arms
Of the 1400 expected student-module combinations, approximately 700 observations will be treatment student-module combinations, and 700 will be control student-module combinations.
Primary outcomes
Each student's standardized score on each of 4 module assessment quizzes
Secondary outcomes
(1) Fraction of members of students from under-represented groups who participated in this study who either (1) take additional courses in economics; or (b) declare economics relative to the overall fractions for each study site. (2) The distribution of correct vs. incorrect answers on each question on the four common assessment instruments.
Sponsors
Name: National Science Foundation Location: Washington, DC Url: http://www.nsf.gov

Papers from this trial

  1. Carson, K.S., H. Adams, J. Gonzalez-Ramirez, C. Heinicke, J. Latham, M. Maier, C.L. Malakar, P. Ruder, and S.P. Simkins (2021), "Challenges and lessons: design and implementation of a multi-site evaluation of team-based learning," Journal of Economic Education 52(3), 241-248. https://doi.org/10.1080/00220485.2021.1925185
  2. Silz Carson, Katherine and González-Ramírez, Jimena and Heinicke, Craig and Maier, Mark H. and Ruder, Philip J. and Simkins, Scott P. and Adams, Hiuko and Latham, James "Michael" and Malakar, C. Lucy, Results of a Randomized Evaluation of Team-Based Learning Exercises. Available at SSRN: https://ssrn.com/abstract=4896585 or http://dx.doi.org/10.2139/ssrn.4896585 https://doi.org/10.1016/j.iree.2025.100316

Cited by

No citation data available for this registration yet.

Keywords