Culturally and context specific student assessment

How to cite

Dobrescu, Loretti; Holden, Richard; Motta, Alberto; Piccoli, Adrian; Roberts, Philip; Walker, Sarah. 2021. "Culturally and context specific student assessment." AEA RCT Registry. AEARCTR-0007309. https://doi.org/10.1257/rct.7309-2.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Loretti Dobrescu* , Richard Holden , Alberto Motta , Adrian Piccoli , Philip Roberts , Sarah Walker

* Primary investigator

Registry ID
AEARCTR-0007309
DOI
https://doi.org/10.1257/rct.7309-2.0
First registered
Last updated
Status
Completed
Trial dates
2021-03-15 – 2021-06-30
Countries
Australia (New South Wales)
Registry record
http://www.socialscienceregistry.org/trials/7309

Abstract

This project explores the effectiveness of culturally and contextually relevant assessment tasks in primary and secondary education.

Intervention

We estimate the effect of adapting and deploying a universal test similar to the National Assessment Program – Literacy and Numeracy (NAPLAN) test to children from different cultures and circumstances.

The sample will include a minimum of (i) 240 Aboriginal and Torres Strait Islander (ATSI) students currently in year 6 (11-12 years old) and their class peers, and (ii) 240 ATSI students currently in year 8 (14-15 year old) and their class peers. In each group, half of the student will be randomly assigned into the group receiving the standard test, whilst the other will receive the contextualized test.

The goal of this project is to explore whether students are more responsive and engaged in assessment tasks if they are locally contextualized and, importantly, whether students perform better on standardized assessments if these are made relevant to their own backgrounds and cultural competencies. We are additionally investigating the mechanism behind these effects.

Experimental design

Important details:
• There are 2 experimental groups: one control and one treatment.
• In control ("Standard Test"), students will receive a standard numeracy and reading test (in the same style as NAPLAN).
• In treatment ("Culturally Appropriate Test"), students will receive a revised version of the standard test that will take into account their context and cultural background.
• There are no differences in the educational content of the two tests (just merely in their form) and each of the numeracy and reading tasks will take 50 minutes regardless of their (standard or contextualised) form.

Design details

Randomization method
Randomization will be carried out by a computer.
Randomization unit
Randomization will be assigned at the individual (student) level.
Clusters
NA
Observations
The analysis involving main outcomes (1) is based on (i) 240 ATSI students currently in year 6 and 500 of their class peers, and (ii) 240 ATSI students currently in year 8 and 500 of their class peers. Depending on the effort indicator mentioned in main outcomes (2-3), we will have the same number of observations as above.
Arms
We have one treatment ("Culturally Appropriate Test") and one control (“Standard Test”), across 4 groups: 1. ATSI students in year 6: 120 treatment and 120 control 2. ATSI students in year 8: 120 treatment and 120 control 3. Peers of ATSI students in year 6: 250 treatment and 250 control 4. Peers of ATSI students in year 8: 250 treatment and 250 control
Primary outcomes
1. Differences in test performance between students who undertook the standard tests compared to those who undertook the contextualized tests. 2. Differences in perceived test effort between students who undertook the standard tests compared to those who undertook the contextualized tests. 3. Differences in perceived test relevance between students who undertook the standard tests compared to those who undertook the contextualized tests.

Cited by

No citation data available for this registration yet.

Keywords