Difference between revisions of "Minimum Detectable Effect"
| (8 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
The minimum detectable effect is the effect size set by the researcher that an '''impact evaluation''' is designed to estimate for a given level of significance. The minimum detectable effect is a critical input for [[Power Calculations | power calculations]] and is closely related to '''power''', [[Sampling|sample size]], and [[Survey Budget | survey and project budgets]]. The page provides an overview of minimum detectable effect and provides points to consider when choosing it. | |||
== | ==Read First== | ||
*The minimum detectable effect is an important input in [[Power Calculations | power calculations]]. | |||
*The minimum detectable effect can be considered the level of impact below which a program will be considered unsuccessful. | |||
*When setting the minimum detectable effect, look at studies on similar programs to understand the potential impact size of the program. | |||
*Set the minimum detectable effect conservatively given sufficient resources. | |||
== | ==Overview== | ||
== | The minimum detectable effect answers the following question: | ||
Given that I only have budget to [[Sampling|sample]] x households, what is the minimum effect size that I will be able to distinguish from a null effect? | |||
Any level of impact below the minimum detectable effect will not be detected. As such, the minimum detectable effect can also be considered the level of impact below which a program will be considered unsuccessful. | |||
==Considerations== | |||
To set the minimum detectable effect, consider the change in outcomes that would justify the investment in an intervention. What is a policy relevant-impact? What are the policy implications? Consider results from studies on similar programs: these may give insight into the potential impact size. | |||
While it is ideal to set the minimum detectable effect conservatively at a small figure, consider that, all else held constant, detecting a smaller minimum detectable effect requires a larger [[Sampling|sample size]]. Thus, factors like the [[Survey Budget | survey budget]] may also influence the researcher’s choice for the minimum detectable effect. | |||
== Back to Parent == | == Back to Parent == | ||
This article is part of the topic [[Sampling & Power Calculations]] | This article is part of the topic [[Sampling & Power Calculations]] | ||
== Additional Resources == | == Additional Resources == | ||
*JPAL’s [https://www.povertyactionlab.org/sites/default/files/L2-Power%20Calculation.pdf Power Calculation] slides. | |||
*JPAL’s [https://www.povertyactionlab.org/sites/default/files/resources/2017.01.11-The-Danger-of-Underpowered-Evaluations.pdf The Danger of Underpowered Evaluations] | |||
*DIME Analytics' presentations on randomization [https://github.com/worldbank/DIME-Resources/blob/master/stata1-5-randomization.pdf 1] and [https://github.com/worldbank/DIME-Resources/blob/master/stata2-5-randomization.pdf 2], which cover minimum detectable effect | |||
[[Category: Sampling & Power Calculations]] | [[Category: Sampling & Power Calculations]] | ||
Latest revision as of 18:54, 9 August 2023
The minimum detectable effect is the effect size set by the researcher that an impact evaluation is designed to estimate for a given level of significance. The minimum detectable effect is a critical input for power calculations and is closely related to power, sample size, and survey and project budgets. The page provides an overview of minimum detectable effect and provides points to consider when choosing it.
Read First
- The minimum detectable effect is an important input in power calculations.
- The minimum detectable effect can be considered the level of impact below which a program will be considered unsuccessful.
- When setting the minimum detectable effect, look at studies on similar programs to understand the potential impact size of the program.
- Set the minimum detectable effect conservatively given sufficient resources.
Overview
The minimum detectable effect answers the following question:
Given that I only have budget to sample x households, what is the minimum effect size that I will be able to distinguish from a null effect?
Any level of impact below the minimum detectable effect will not be detected. As such, the minimum detectable effect can also be considered the level of impact below which a program will be considered unsuccessful.
Considerations
To set the minimum detectable effect, consider the change in outcomes that would justify the investment in an intervention. What is a policy relevant-impact? What are the policy implications? Consider results from studies on similar programs: these may give insight into the potential impact size.
While it is ideal to set the minimum detectable effect conservatively at a small figure, consider that, all else held constant, detecting a smaller minimum detectable effect requires a larger sample size. Thus, factors like the survey budget may also influence the researcher’s choice for the minimum detectable effect.
Back to Parent
This article is part of the topic Sampling & Power Calculations
Additional Resources
- JPAL’s Power Calculation slides.
- JPAL’s The Danger of Underpowered Evaluations
- DIME Analytics' presentations on randomization 1 and 2, which cover minimum detectable effect