What do Editors Maximize? A Survey on Paper Quality

How to cite

DellaVigna, Stefano; Card, David. 2016. "What do Editors Maximize? A Survey on Paper Quality." AEA RCT Registry. AEARCTR-0001669. https://doi.org/10.1257/rct.1669-1.0

Download: BibTeX · RIS — for reference managers without a browser translator. Zotero and most others pick up this page's citation metadata directly.

Stefano DellaVigna* , David Card

* Primary investigator

Registry ID
AEARCTR-0001669
DOI
https://doi.org/10.1257/rct.1669-1.0
First registered
Last updated
Status
In development
Trial dates
2016-09-29 – 2017-03-15
Countries
Private
Registry record
http://www.socialscienceregistry.org/trials/1669

Abstract

Publications in top scientific journals play a critical role in the careers of scientists. Yet, remarkably little is known about how editors choose which submissions to publish. We provide evidence on this decision-making process using anonymized data on all submissions over eight years to four leading economics journals: the Journal of the European Economics Association, the Quarterly Journal of Economics, the Review of Economic Studies, and the Review of Economics and Statistics. The data set contains information on characteristics of the papers (and their authors), referee recommendations, and the editorial decision. The manuscripts are matched to Google Scholar citations. We compare the findings in the four journals to the predictions of a simple descriptive model in which editors aim to maximize the citations of published articles. To gather some additional insights, we conduct a survey of faculty and PhD students in economics, asking them to compare pairs of papers in their field of expertise. This survey is the focus of this pre-registration. The pairs of papers evaluated in the survey are selected so both papers are in the same field and both were published in a top journal in the same year between 1999 and 2012. We ask the respondents to compare papers on four features: novelty, exposition, rigor, and importance of contribution. We also provide respondents with the actual Google Scholar citations for the paper and ask them about what they think would be the appropriate citations, given their judgment. This additional evaluation us to return to alternative interpretations for the findings on editorial choices.

Intervention

There is no intervention. This pre-registration is for a survey that will collect an assessment of quality of pairs of published papers.

Experimental design

This pre-registration is for a survey that will collect an assessment of quality of pairs of published papers. We will use this information to complement the analysis of editorial choices in a data set of 4 high-impact economic journals, as described in detail in the pre-analysis plan.

Design details

Randomization method
This is a survey, in which randomization does not play a key role. Nonetheless, we randomize the order of some questions and which paper is presented first in a pair of papers.
Randomization unit
The paper-pair level.
Clusters
There will be 60 pairs of papers.
Observations
We intend to collect evaluations for 50-100 respondents, depending on the response rate to the survey.
Arms
There is only one arm.
Primary outcomes
The outcomes are the collected valuations of paper quality, as specified in the pre-analysis plan.

Cited by

No citation data available for this registration yet.

Keywords