Difference between revisions of "Cost-effectiveness Analysis"

Jump to: navigation, search
Line 1: Line 1:
<onlyinclude>
<onlyinclude>
'''Cost-effectiveness Analysis''' is the economic analysis of the costs and benefits of an impact evaluation project.  
A cost-effectiveness analysis (CEA) compares the cost and effectiveness of a given program to determine whether the value of an intervention justifies its cost. When to use. What this page tells you.  
 
</onlyinclude>
</onlyinclude>
== Read First ==
== Read First ==


*Cost-effectiveness analysis should be included in project design to compare alternative projects.
*A cost-effectiveness analysis should be included in project design to compare alternative projects.
*point 1
*point 2
 
==Overview==


== Guidelines ==
A CEA summarizes the results of a program as the ratio of costs to monetized effects. It measures the cost for a given level of effectiveness (i.e. the cost to increase skilled birth attendance by 50 percent) or visa versa (i.e the percentage gain in skilled birth attendance induced by spending $100). The analysis must accurately reflect the program based on costs and impacts actually observed.
===What is Cost-effective Analysis (CEA)?===


CEA summarizes the results of complex program as ratio of costs to effects. It measures the cost for a given level of effectiveness, for example the cost to increase skilled birth attendance by 50 percent or vice versa, for example the percentage gain in skilled birth attendance induced by spending $100. The analysis must accurately reflect the program based on costs and impacts actually observed. It is useful for comparing multiple interventions with common outcome(s) if the costs and benefits are computed using similar methodology for all programs. It is useful for policymakers to compare programs when they are primarily concerned about one outcome of interest (e.g. increasing skilled birth attendance, not maternal and child health broadly)
A CEA is useful for policymakers looking to compare programs when they are primarily concerned about one outcome of interest (i.e. increasing skilled birth attendance rather than maternal and child health broadly). It can be used to compare multiple interventions with common outcome(s) if the costs and benefits are computed using similar methodology for all programs.


====Difference between Cost-effective Analysis (CEA) and Cost-benefit Analysis (CBA)====
== Cost-effectiveness analysis (CEA) vs. cost-benefit analysis (CBA)==


CBA compares the monetary value of benefits against costs and hence calculates the ratio of all benefits to the costs of a program. CBA can be useful to determine whether a program is worth the investment. It can also allow comparison across vastly different interventions, for example education versus agriculture. However, CBA requires a number of strong assumptions about the monetary value of all the different benefits, including the lifetime benefits of an intervention.
CBA calculates the monetary ratio of all benefits to all costs of a program. CBA can help to determine whether a program is worth the investment. It can also allow comparison across vastly different interventions (i.e. education versus agriculture). However, CBA requires a number of strong assumptions about the monetary value of all the different benefits, including the lifetime benefits of an intervention. In contrast, CEA is a transparent, simple, and objective measurement that enables comparison of programs with common outcome(s) of interest. However, the implicit assumption is a common post-intervention trajectory.


'''Cost-benefit Analysis Formula'''
'''Cost-benefit Analysis Formula'''
[[File:CBA Formula.jpg]]
[[File:CBA Formula.jpg]]
CEA is transparent, simple, objective and enables comparison of programs with common outcome(s) of interest. However, the implicit assumption is a common post-intervention trajectory.


'''Cost-effectiveness Analysis Formula'''
'''Cost-effectiveness Analysis Formula'''
[[File:CEA Formula.jpg]]
[[File:CEA Formula.jpg]]


====Use of Cost-effective Analysis in Impact Evaluations====
== Considerations for cost calculations ==  
There are very few papers that have undertaken comparative cost‐effectiveness of different programs.
Most studies don’t conduct cost analysis. Of the 77 RCTs, 56% reported zero data on incremental costs (McEwan 2014).


Studies that have incorporated CEA include:
*Use program financial records. Typically, begin calculating costs by referring to program budgets, which provide a list of all relevant activities involved in implementing the program. Consider, however, that budgets are forward-looking estimates of true costs. Thus, it’s better to use actual program financial records on expenditures for calculations. However, this is not commonly done.
*Kremer, Miguel and Thornton (2005): Incentives to learn
*Banerjee, Cole, Duflo, Linden (2007): Remedial education vs computer-assisted learning
However, these were done for the same country / organization


====Challenges of Conducting Cost-effective Analysis====
*Interview program and field staff to get key information including unit cost data like allocation of staff time across activities and wages for staff at various levels of implementation team. Note that using average wages makes calculations less sensitive.
*Difficult to get data on costs, especially from other authors (this is where Field Co-ordinators come in; they have an advantage in collecting local costs)
*Difficult to compare studies ‐ exchange rates, inflation, discount rates, differing factor endowments, efficiency of implementers (CEA are sensitive to place, scale, errors in estimates)
*Absence of publication incentives to do CEA: trade-off between exposing program effectiveness and cost‐effectiveness


====='''Quantifying Impacts'''=====
*Model program costs at the margin. You want to capture marginal costs of new activities initiated as part of the interventions. For example, to calculate the costs of a school meals program, you may need the cost of new kitchen, cooks, implements, and food. However, you should not include the cost of school administration or buildings as these would exist whether or not the intervention was implemented.


*Spillover effects
*Consider depreciation. Account for how to value new assets or equipment obtained over the program implementation period.
**We need to include spillover effects on indirect beneficiaries when quantifying impacts for CEA


*Impacts on multiple outcomes
*Consider the cost of user time. Include the costs of participation in program (i.e. using local wages).
**Because CEA measures cost of obtaining impacts on a single outcome, aggregating cost-effectiveness across multiple outcomes is difficult


*Insignificant results on outcomes
*Differentiate between pilot costs and scale-up costs.
**It is important to think about how these should be interpreted for CEA and how to present such results with costs together in publications


====='''Calculating Costs'''=====
*Include spillover effects on indirect beneficiaries when quantifying impacts


*Activity-based costing
==Challenges to conducting CEA==
–usually start with program budgets to get a list of all relevant activities involved in implementing the program
•budgets are forward-looking estimates of true costs; better to update with actual program financial records on expenditures, but this is not commonly done
*It is difficult to get data on program costs, especially from other authors. This is where Field Coordinators come in, as they have an advantage in collecting local costs.
•need to account for how to value new assets/equipment obtained over the program implementation period; depreciation
* Since CEAs are sensitive to place, scale, and errors in estimates, it is difficult to compare studies. Exchange rates, inflation rates, discount rates, differing factor endowments, and the efficiency of implementors all affect CEA results and make valid comparisons between studies or programs difficult.
–interview program staff and field staff to get key information
*Because CEA measures cost of obtaining impacts on a single outcome, aggregating cost-effectiveness across multiple outcomes is difficult
•unit cost data: e.g., wages for staff at various levels of implementation team, using average wages makes this less sensitive
•allocation of staff time across activities


*Modeling program costs at the margin
==Use in impact evaluations==
–want to capture marginal costs of new activities initiated as part of the interventions
–Ex: school meals program: cost of new kitchen, cooks, implements, food, but NOT cost of school administration or buildings


•Find comparative costs to frame the cost analysis
Very few papers have undertaken comparative cost‐effectiveness of different programs.
A [http://academics.wellesley.edu/Economics/mcewan/PDF/meta.pdf meta-analysis] of randomized experiments finds that of the 77 RCTs gathered, 56% don’t report any data on incremental costs. Studies that have incorporated CEA Kremer, Miguel and Thornton’s [https://dash.harvard.edu/bitstream/handle/1/3716457/kremer_incentiveslearn.pdf?sequence=2 Incentives to Learn] and Banerjee, Cole, Duflo, and Linden’s  [https://economics.mit.edu/files/804 Remedying Education].


*Costs of user time
Many researchers have no incentive to conduct a CEA. There is sometimes a trade-off between exposing program effectiveness and cost-effectiveness. For those that do conduct CEAs, it is unclear how to report insignificant results on outcomes. It is important to think about how insignificant CEA results should be interpreted and presented in such cases.
Gather costs of participation in program, including cost of time (e.g., using local wages)
 
*Differentiate between Pilot Costs and Scale-up Costs


== Back to Parent ==
== Back to Parent ==
Line 80: Line 65:
*Comparative Cost-Effectiveness Analysis to Inform Policy in Developing Countries, by Dhaliwal et al (2011)
*Comparative Cost-Effectiveness Analysis to Inform Policy in Developing Countries, by Dhaliwal et al (2011)
*Cost-Effectiveness Measurement in Development: Accounting for Local Costs & Noisy Impacts, by Evans & Popova (2014)
*Cost-Effectiveness Measurement in Development: Accounting for Local Costs & Noisy Impacts, by Evans & Popova (2014)


Book  
Book  
*Cost-Effectiveness Analysis: Methods & Applications, by McEwan & Levin (2000)
*Cost-Effectiveness Analysis: Methods & Applications, by McEwan & Levin (2000)


Other Resources
Other Resources
Line 92: Line 75:
**Examples
**Examples
**Templates
**Templates


*Poverty Action Lab
*Poverty Action Lab
**Examples for student participation and student learning
**Examples for student participation and student learning
**What does this look like in Excel?
**What does this look like in Excel?

Revision as of 16:43, 12 April 2019

A cost-effectiveness analysis (CEA) compares the cost and effectiveness of a given program to determine whether the value of an intervention justifies its cost. When to use. What this page tells you.


Read First

  • A cost-effectiveness analysis should be included in project design to compare alternative projects.
  • point 1
  • point 2

Overview

A CEA summarizes the results of a program as the ratio of costs to monetized effects. It measures the cost for a given level of effectiveness (i.e. the cost to increase skilled birth attendance by 50 percent) or visa versa (i.e the percentage gain in skilled birth attendance induced by spending $100). The analysis must accurately reflect the program based on costs and impacts actually observed.

A CEA is useful for policymakers looking to compare programs when they are primarily concerned about one outcome of interest (i.e. increasing skilled birth attendance rather than maternal and child health broadly). It can be used to compare multiple interventions with common outcome(s) if the costs and benefits are computed using similar methodology for all programs.

Cost-effectiveness analysis (CEA) vs. cost-benefit analysis (CBA)

CBA calculates the monetary ratio of all benefits to all costs of a program. CBA can help to determine whether a program is worth the investment. It can also allow comparison across vastly different interventions (i.e. education versus agriculture). However, CBA requires a number of strong assumptions about the monetary value of all the different benefits, including the lifetime benefits of an intervention. In contrast, CEA is a transparent, simple, and objective measurement that enables comparison of programs with common outcome(s) of interest. However, the implicit assumption is a common post-intervention trajectory.

Cost-benefit Analysis Formula CBA Formula.jpg

Cost-effectiveness Analysis Formula CEA Formula.jpg

Considerations for cost calculations

  • Use program financial records. Typically, begin calculating costs by referring to program budgets, which provide a list of all relevant activities involved in implementing the program. Consider, however, that budgets are forward-looking estimates of true costs. Thus, it’s better to use actual program financial records on expenditures for calculations. However, this is not commonly done.
  • Interview program and field staff to get key information including unit cost data like allocation of staff time across activities and wages for staff at various levels of implementation team. Note that using average wages makes calculations less sensitive.
  • Model program costs at the margin. You want to capture marginal costs of new activities initiated as part of the interventions. For example, to calculate the costs of a school meals program, you may need the cost of new kitchen, cooks, implements, and food. However, you should not include the cost of school administration or buildings as these would exist whether or not the intervention was implemented.
  • Consider depreciation. Account for how to value new assets or equipment obtained over the program implementation period.
  • Consider the cost of user time. Include the costs of participation in program (i.e. using local wages).
  • Differentiate between pilot costs and scale-up costs.
  • Include spillover effects on indirect beneficiaries when quantifying impacts

Challenges to conducting CEA

  • It is difficult to get data on program costs, especially from other authors. This is where Field Coordinators come in, as they have an advantage in collecting local costs.
  • Since CEAs are sensitive to place, scale, and errors in estimates, it is difficult to compare studies. Exchange rates, inflation rates, discount rates, differing factor endowments, and the efficiency of implementors all affect CEA results and make valid comparisons between studies or programs difficult.
  • Because CEA measures cost of obtaining impacts on a single outcome, aggregating cost-effectiveness across multiple outcomes is difficult

Use in impact evaluations

Very few papers have undertaken comparative cost‐effectiveness of different programs. A meta-analysis of randomized experiments finds that of the 77 RCTs gathered, 56% don’t report any data on incremental costs. Studies that have incorporated CEA Kremer, Miguel and Thornton’s Incentives to Learn and Banerjee, Cole, Duflo, and Linden’s Remedying Education.

Many researchers have no incentive to conduct a CEA. There is sometimes a trade-off between exposing program effectiveness and cost-effectiveness. For those that do conduct CEAs, it is unclear how to report insignificant results on outcomes. It is important to think about how insignificant CEA results should be interpreted and presented in such cases.

Back to Parent

This article is part of the topic Cost-effectiveness Analysis

Additional Resources

Papers

  • Cost-Effectiveness Analysis of Education & Health Interventions in Developing Countries, by McEwan (2012)
  • Comparative Cost-Effectiveness Analysis to Inform Policy in Developing Countries, by Dhaliwal et al (2011)
  • Cost-Effectiveness Measurement in Development: Accounting for Local Costs & Noisy Impacts, by Evans & Popova (2014)

Book

  • Cost-Effectiveness Analysis: Methods & Applications, by McEwan & Levin (2000)

Other Resources

  • IDB Evaluation Hub: Cost Benefit and Cost-Effectiveness
    • Background materials
    • Terms of reference
    • Examples
    • Templates
  • Poverty Action Lab
    • Examples for student participation and student learning
    • What does this look like in Excel?