Risk Management Software and Customs Enforcement in Pakistan

How to cite

Best, Michael; Hayat, Faraz; Dobermann, Tim. 2026. "Risk Management Software and Customs Enforcement in Pakistan." AEA RCT Registry. AEARCTR-0017801. https://doi.org/10.1257/rct.17801-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Michael Best* , Faraz Hayat , Tim Dobermann

* Primary investigator

Registry ID
AEARCTR-0017801
DOI
https://doi.org/10.1257/rct.17801-1.0
First registered
Last updated
Status
On going
Trial dates
2026-01-25 – 2026-03-31
Countries
Pakistan ()
Registry record
http://www.socialscienceregistry.org/trials/17801

Abstract

The Federal Board of Revenue (FBR) of Pakistan is evaluating a new risk management software (RMS2), as a potential enhancement to its existing Risk Management System (RMS1) for customs clearance. RMS classifies Goods Declarations (GDs) into color-coded risk channels—Red (high risk), Yellow (medium risk), and Green (low risk)—which determine the level of documentary review and physical inspection. This randomized controlled trial assesses whether RMS2 improves the identification of high-risk imports and supports better decision-making by customs officers.
The study has two primary objectives: first, to compare the accuracy of RMS2’s risk-scoring algorithm with that of RMS1 in identifying risky GDs; and second, to evaluate whether providing officers with additional risk-related information through the RMS2 user interface enhances officer performance. To achieve these aims, the study implements two interventions. In the first, GDs are independently scored by both RMS2 and RMS1, and a stratified random sample is assigned to a forced-red regime to generate ground-truth inspection outcomes while keeping assessment officers blinded to original risk signals. In the second intervention, assessment officers are randomly assigned to either receive access to the RMS2 interface or continue under standard procedures.
The study will run for approximately 60 days and cover about 9,000 GDs. Outcomes will inform FBR’s procurement decision and provide evidence on the operational value of integrating RMS2 into customs risk management processes.

Experimental design

The study uses two linked experimental components to evaluate the performance of the new RMS2 customs risk management system and its effect on assessment officers’ decisions. First, to create reliable “ground-truth” outcomes for comparing RMS2 with the existing FBR Risk Management System (RMS1), a randomly selected subset of Goods Declarations (GDs) is assigned to a forced-Red channel. All GDs are initially risk-scored and classified by RMS1 as Red, Yellow, or Green. Within each classification and risk score range, GDs are randomly selected for mandatory full documentary review and physical inspection, regardless of their original channel. This forced-Red process generates verified inspection results that enable a direct ex-post comparison of how accurately RMS2 and RMS1 identify high-risk filings, while all other GDs proceed under standard business-as-usual procedures.

Second, to measure the effect of providing officers with additional risk information, assessment officers are randomly assigned to either a control group or a treatment group. Control officers follow existing procedures without access to the RMS2’s interface and channel assignments, while treated officers receive full access to RMS2’s channel assignments and detailed features such as is there a predicted hs code discrepancy, is there an origin country discrepancy, and is there a valuation discrepancy, and if so, then how much. Outcomes are compared across forced-Red and business-as-usual cases, allowing the study to isolate improvements in risk targeting, the informational effect of RMS2 on officer decision-making, and the overall operational impact of integrating RMS2 into customs processes.

Design details

Randomization method
The UI was randomized in an office on a computer by us. The "forced-red" sample of goods declarations is randomized in the scheduler system.
Randomization unit
officer for the UI intervention. Good Declaration for the "forced-red" sample for measurement.
Clusters
8400 GDs for the "forced-red" sample. 96 officers for the UI intervention.
Observations
8400 GDs
Arms
48 treated officers and 48 controls. approximately 4200 GDs per treatment arm.
Primary outcomes
1) Algorithm Performance: To determine if RMS2 is more effective at identifying high-risk Goods Declarations (GDs) for inspection than FBR's current Risk Management System (RMS1). 2) Information Effect: To measure whether providing customs officers with access to the full RMS2 User Interface (UI) leads to better decision-making and improved outcomes.
Sponsors
Name: International Growth Centre Location: London Url: https://www.theigc.org
Partners
Name: Federal Board of Revenue Partner_type: government Url: https://www.fbr.gov.pk

Cited by

No citation data available for this registration yet.

Keywords