Difference between revisions of "Power Calculations in Optimal Design"
Maria jones (talk | contribs) |
|||
| (10 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
Optimal Design is free software designed by University of Michigan. It provides a useful platform on which researchers can visualize the relationship between different elements of the [[Sampling|sample]] size formula when conducting [[Power Calculations | power calculations]] during the research design stage. This page provides a general overview of and additional resources for Optimal Design. | |||
== Read First == | == Read First == | ||
* | *Access Optimal Design [https://sites.google.com/site/optimaldesignsoftware/home here]. | ||
*For [[Reproducible Research | reproducibility]], DIME recommends conducting power calculations in [[Power Calculations in Stata | Stata]] and using Optimal Design as a compliment for visualization. | |||
*For more information on the key parameters of power calculations, see [[Power Calculations|power calculations]]. | |||
== Overview == | |||
== | Optimal Design creates graphs to visualize trade-offs and the relationships between the various components of [[Power Calculations|power calculations]]. It can compare, for example, power versus [[Sampling|sample size]] for a given effect or effect size versus '''sample size''' for a given desired '''power'''. While Optimal Design can aid researcher understanding and decision-making during '''power calculations''', the software has a few shortcomings. First, it is not replicable. Second, it cannot calculate '''power''' for an individual-level [[Randomization|randomization]] with a binary outcome. Third, it assumes equal mean and variance for '''treatment''' and '''control groups'''. For an [[Randomized Control Trials | RCT]] this is generally okay. Fourth, it assumes an equal split of '''sample size''' or cluster number between '''treatment''' and '''control groups'''. For researchers who wish to fix the size of the '''treatment group''', say because of budget constraints, and then calculate '''control group''' size, this feature is limiting. In general, [[Power Calculations in Stata | Stata]] offers much greater flexibility in '''power and sampling calculations'''. DIME Analytics strongly recommends conducting '''power calculations in Stata''' and using Optimal Design as a compliment for visualization and general understanding. | ||
== Back to Parent == | == Back to Parent == | ||
This article is part of the topic [[Sampling & Power Calculations]] | This article is part of the topic [[Sampling & Power Calculations]] | ||
== Additional Resources == | == Additional Resources == | ||
* | *Poverty Action Lab’s [https://www.ilo.org/wcmsp5/groups/public/@ed_emp/documents/presentation/wcms_419000.pdf Exercise: How to do Power Calculations in Optimal Design Software] | ||
* Berk Ozler’s [http://blogs.worldbank.org/impactevaluations/power-calculations-what-software-should-i-use Power Calculations: What software should I use?] via The World Bank's Development Impact blog | |||
*DIME Analytics guidelines on survey sampling and power calculations [https://github.com/worldbank/DIME-Resources/blob/master/survey-sampling-1.pdf 1] and [https://github.com/worldbank/DIME-Resources/blob/master/survey-sampling-2.pdf 2] | |||
* Andrew Gelman’s [http://andrewgelman.com/2017/03/03/yes-makes-sense-design-analysis-power-calculations-data-collected/ Why it makes sense to revisit power calculations after data has been collected] | |||
*JPAL’s [https://www.povertyactionlab.org/sites/default/files/resources/2017.01.11-The-Danger-of-Underpowered-Evaluations.pdf The Danger of Underpowered Evaluations] | |||
[[Category: Sampling & Power Calculations]] | [[Category: Sampling & Power Calculations]] | ||
[[Category: Reproducible Research]] | |||
Latest revision as of 13:28, 14 August 2023
Optimal Design is free software designed by University of Michigan. It provides a useful platform on which researchers can visualize the relationship between different elements of the sample size formula when conducting power calculations during the research design stage. This page provides a general overview of and additional resources for Optimal Design.
Read First
- Access Optimal Design here.
- For reproducibility, DIME recommends conducting power calculations in Stata and using Optimal Design as a compliment for visualization.
- For more information on the key parameters of power calculations, see power calculations.
Overview
Optimal Design creates graphs to visualize trade-offs and the relationships between the various components of power calculations. It can compare, for example, power versus sample size for a given effect or effect size versus sample size for a given desired power. While Optimal Design can aid researcher understanding and decision-making during power calculations, the software has a few shortcomings. First, it is not replicable. Second, it cannot calculate power for an individual-level randomization with a binary outcome. Third, it assumes equal mean and variance for treatment and control groups. For an RCT this is generally okay. Fourth, it assumes an equal split of sample size or cluster number between treatment and control groups. For researchers who wish to fix the size of the treatment group, say because of budget constraints, and then calculate control group size, this feature is limiting. In general, Stata offers much greater flexibility in power and sampling calculations. DIME Analytics strongly recommends conducting power calculations in Stata and using Optimal Design as a compliment for visualization and general understanding.
Back to Parent
This article is part of the topic Sampling & Power Calculations
Additional Resources
- Poverty Action Lab’s Exercise: How to do Power Calculations in Optimal Design Software
- Berk Ozler’s Power Calculations: What software should I use? via The World Bank's Development Impact blog
- DIME Analytics guidelines on survey sampling and power calculations 1 and 2
- Andrew Gelman’s Why it makes sense to revisit power calculations after data has been collected
- JPAL’s The Danger of Underpowered Evaluations