Difference between revisions of "Pre-Analysis Plan"

Jump to: navigation, search
Line 3: Line 3:


== Read First ==
== Read First ==
* include here key points you want to make sure all readers understand


== Guidelines ==
== Guidelines ==
While most economics journals do not currently require PAPs as a condition for publication, researchers may choose to produce a PAP prior to data analysis to: (i) increase the credibility of their findings; and (ii) help researchers finetune their analysis strategy.


===Subsection 1===
While PAPs provide the benefit of potentially reducing the prevalence of spurious results, this comes at the cost of tying researcher hands more formally to ex ante analysis plans that may limit the potential of exploratory learning. Benjamin Olken provides a summary of the costs and benefits associated with fully pre-specifying the analysis for a development economics RCT [https://www.aeaweb.org/articles?id=10.1257/jep.29.3.61]. He notes that "forcing all papers to be fully pre-specified from start to end would likely results in simpler papers, which could potentially lose some of the nuance of current work", but that "in many contexts, pre-specification of one (or a few) key primary outcome variables, statistical specifications, and control variables offers a number of advantages".
===Subsection 2===
===Subsection 3===


== Back to Parent ==
== Back to Parent ==
This article is part of the topic [[Data Analysis]]
This article is part of the topic [[Research Ethics]]




== Additional Resources ==
== Additional Resources ==
* Promises and Perils of Pre-Analysis Plans http://economics.mit.edu/files/10654
* BITTS: [http://scholar.harvard.edu/files/alejandro_ganimian/files/pre-analysis_plan_template_0.pdf Template for Pre-analysis plans]
*Development Impact Blog: [http://blogs.worldbank.org/impactevaluations/a-pre-analysis-plan-checklist Pre-analysis plan checklist]
*Olken, Benjamin A.. 2015. "[https://www.aeaweb.org/articles?id=10.1257/jep.29.3.61 Promises and Perils of Pre-analysis Plans]." Journal of Economic Perspectives, 29(3): 61-80.
*Olken, Benjamin A.. 2015. "[https://www.aeaweb.org/articles?id=10.1257/jep.29.3.61 Promises and Perils of Pre-analysis Plans]." Journal of Economic Perspectives, 29(3): 61-80.
DOI: 10.1257/jep.29.3.61
DOI: 10.1257/jep.29.3.61
* [https://www.bitss.org/wp-content/uploads/2015/12/Pre-Analysis-Plan-Template.pdf Pre-Analysis Plan Template]
* [http://blogs.worldbank.org/impactevaluations/a-pre-analysis-plan-checklist Pre-analysis plan checklist from Development Impact Blog]


[[Category: Data Analysis]] [[Category: Reproducible Research]]
[[Category: Research Ethics]]

Revision as of 23:40, 9 February 2018

A pre-analysis plan (PAP) lays out how the researcher will analyze data, at the design stage of an impact evaluation. The objective of a PAP is to prevent data mining and specification searching.


Read First

Guidelines

While most economics journals do not currently require PAPs as a condition for publication, researchers may choose to produce a PAP prior to data analysis to: (i) increase the credibility of their findings; and (ii) help researchers finetune their analysis strategy.

While PAPs provide the benefit of potentially reducing the prevalence of spurious results, this comes at the cost of tying researcher hands more formally to ex ante analysis plans that may limit the potential of exploratory learning. Benjamin Olken provides a summary of the costs and benefits associated with fully pre-specifying the analysis for a development economics RCT [1]. He notes that "forcing all papers to be fully pre-specified from start to end would likely results in simpler papers, which could potentially lose some of the nuance of current work", but that "in many contexts, pre-specification of one (or a few) key primary outcome variables, statistical specifications, and control variables offers a number of advantages".

Back to Parent

This article is part of the topic Research Ethics


Additional Resources

DOI: 10.1257/jep.29.3.61