Difference between revisions of "Minimum Detectable Effect"

Jump to: navigation, search
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Read First ==
The minimum detectable effect is the effect size set by the researcher that an '''impact evaluation''' is designed to estimate for a given level of significance. The minimum detectable effect is a critical input for [[Power Calculations | power calculations]] and is closely related to '''power''', [[Sampling|sample size]], and [[Survey Budget | survey and project budgets]]. The page provides an overview of minimum detectable effect and provides points to consider when choosing it.
<onlyinclude>
Minimum Detectable Effect Size (MDES) is a critical part of power calculations.  
</onlyinclude>


== Guidelines ==
==Read First==
We do not know in advance the effect of our policy. We want to design a precise way of measuring it. But precision is not cheap: need cost-benefit analysis to decide. We need to identify the smallest program effect size that it would be useful to detect, i.e. the smallest effect for which we would be able to say with statistical confidence that the program effect is statistically different from zero.  
*The minimum detectable effect is an important input in [[Power Calculations | power calculations]].
*The minimum detectable effect can be considered the level of impact below which a program will be considered unsuccessful.  
*When setting the minimum detectable effect, look at studies on similar programs to understand the potential impact size of the program.
*Set the minimum detectable effect conservatively given sufficient resources.


===Subsection 1===
==Overview==
===Subsection 2===
The minimum detectable effect answers the following question:
===Subsection 3===
 
Given that I only have budget to [[Sampling|sample]] x households, what is the minimum effect size that I will be able to distinguish from a null effect?
 
Any level of impact below the minimum detectable effect will not be detected. As such, the minimum detectable effect can also be considered the level of impact below which a program will be considered unsuccessful.
 
==Considerations==
To set the minimum detectable effect, consider the change in outcomes that would justify the investment in an intervention. What is a policy relevant-impact? What are the policy implications? Consider results from studies on similar programs: these may give insight into the potential impact size.
 
While it is ideal to set the minimum detectable effect conservatively at a small figure, consider that, all else held constant, detecting a smaller minimum detectable effect requires a larger [[Sampling|sample size]]. Thus, factors like the [[Survey Budget | survey budget]] may also influence the researcher’s choice for the minimum detectable effect.


== Back to Parent ==
== Back to Parent ==
This article is part of the topic [[Sampling & Power Calculations]]
This article is part of the topic [[Sampling & Power Calculations]]


== Additional Resources ==
== Additional Resources ==
Please add here related articles, with a brief description and link.  
*JPAL’s [https://www.povertyactionlab.org/sites/default/files/L2-Power%20Calculation.pdf Power Calculation] slides.
 
*JPAL’s [https://www.povertyactionlab.org/sites/default/files/resources/2017.01.11-The-Danger-of-Underpowered-Evaluations.pdf The Danger of Underpowered Evaluations]
*DIME Analytics' presentations on randomization [https://github.com/worldbank/DIME-Resources/blob/master/stata1-5-randomization.pdf 1] and [https://github.com/worldbank/DIME-Resources/blob/master/stata2-5-randomization.pdf 2], which cover minimum detectable effect
[[Category: Sampling & Power Calculations]]
[[Category: Sampling & Power Calculations]]

Latest revision as of 18:54, 9 August 2023

The minimum detectable effect is the effect size set by the researcher that an impact evaluation is designed to estimate for a given level of significance. The minimum detectable effect is a critical input for power calculations and is closely related to power, sample size, and survey and project budgets. The page provides an overview of minimum detectable effect and provides points to consider when choosing it.

Read First

  • The minimum detectable effect is an important input in power calculations.
  • The minimum detectable effect can be considered the level of impact below which a program will be considered unsuccessful.
  • When setting the minimum detectable effect, look at studies on similar programs to understand the potential impact size of the program.
  • Set the minimum detectable effect conservatively given sufficient resources.

Overview

The minimum detectable effect answers the following question:

Given that I only have budget to sample x households, what is the minimum effect size that I will be able to distinguish from a null effect?

Any level of impact below the minimum detectable effect will not be detected. As such, the minimum detectable effect can also be considered the level of impact below which a program will be considered unsuccessful.

Considerations

To set the minimum detectable effect, consider the change in outcomes that would justify the investment in an intervention. What is a policy relevant-impact? What are the policy implications? Consider results from studies on similar programs: these may give insight into the potential impact size.

While it is ideal to set the minimum detectable effect conservatively at a small figure, consider that, all else held constant, detecting a smaller minimum detectable effect requires a larger sample size. Thus, factors like the survey budget may also influence the researcher’s choice for the minimum detectable effect.

Back to Parent

This article is part of the topic Sampling & Power Calculations

Additional Resources