Difference between revisions of "Quasi-Experimental Methods"

Jump to: navigation, search
Line 1: Line 1:
<span style="font-size:150%">
== Introduction ==
<span style="color:#ff0000"> '''NOTE: this article is only a template. Please add content!''' </span>
</span>
 
 
add introductory 1-2 sentences here
 


 
When it is not possible to use [[Experimental Methods]], researchers turn to other methods that attempt to simulate conditions where exposure to a treatment effect is uncorrelated with other characteristics. Unlike in experimental research methods, the investigator does not have direct control over the exposure. "Natural experiments", such as [[Regression Discontinuity | regression discontinuity]] designs and [[Event Study | event studies]], identify existing circumstances where assignment of treatment has an exploitable element of randomness. In other cases, researchers attempt to simulate an experimental counterfactual by constructing a control group that is as similar as possible to the treatment group, using techniques such as [[Propensity Score Matching|propensity score matching]].
== Read First ==
* include here key points you want to make sure all readers understand




== Guidelines ==
== Guidelines ==
* organize information on the topic into subsections. for each subsection, include a brief description / overview, with links to articles that provide details
=== The Power of Quasi-Experimental Methods ===
===Subsection 1===
===Subsection 2===
===Subsection 3===
 
== Back to Parent ==
This article is part of the topic [[*topic name, as listed on main page*]]
 
 
== Additional Resources ==
* list here other articles related to this topic, with a brief description and link
 
[[Category: *category name* ]]
 
== Introduction ==
 
Like [[Experimental Methods]], quasi-experimental research methods are intended to satisfy the conditions where a researcher can construct a model where exposure to a treatment effect is uncorrelated with other characteristics. Unlike in experimental research methods, the investigator does not have direct control over the exposure, and instead uses "natural experiments" such as [[Regression Discontinuity | regression discontinuity]] designs in [[Event Study | event studies]] to identify existing circumstances where random assignment of some treatment took place.
 
== The Power of Quasi-Experimental Methods ==
 
Like experimental methods, quasi-experimental methods aim to solve the problems of [[confoundedness]] and [[reverse or simultaneous causality]] in identifying the marginal effect of a change in condition. Unlike [[Experimental Methods | experimental methods]], quasi-experiments tend to be based on policy changes or natural events that were not actually randomly assigned, but occurred in such a way that they can be considered "[[as good as randomly assigned]]" relative to some of the outcomes under consideration. Two examples illustrate these differences well.
Like experimental methods, quasi-experimental methods aim to solve the problems of [[confoundedness]] and [[reverse or simultaneous causality]] in identifying the marginal effect of a change in condition. Unlike [[Experimental Methods | experimental methods]], quasi-experiments tend to be based on policy changes or natural events that were not actually randomly assigned, but occurred in such a way that they can be considered "[[as good as randomly assigned]]" relative to some of the outcomes under consideration. Two examples illustrate these differences well.


Line 43: Line 16:
This is because the local magnitude may be correlated with other varying characteristics (such as soil quality or hilliness) that also contribute to the outcomes (such as family income) and therefore re-introduce the confounding problem by including a non-random component that is correlated to a key characteristic. Distance from a randomly placed disaster, though a less precise measure of "exposure", is critically uncorrelated with almost any imaginable outcome under the correct assumption (unlike, for example, distance to a volcano, or to a tsunami-affected shoreline).
This is because the local magnitude may be correlated with other varying characteristics (such as soil quality or hilliness) that also contribute to the outcomes (such as family income) and therefore re-introduce the confounding problem by including a non-random component that is correlated to a key characteristic. Distance from a randomly placed disaster, though a less precise measure of "exposure", is critically uncorrelated with almost any imaginable outcome under the correct assumption (unlike, for example, distance to a volcano, or to a tsunami-affected shoreline).


== Common Types of Non-Experimental Methods ==
 
 
== Additional Resources ==
* list here other articles related to this topic, with a brief description and link
 
[[Category: Quasi-Experimental Methods ]]

Revision as of 16:23, 9 February 2018

Introduction

When it is not possible to use Experimental Methods, researchers turn to other methods that attempt to simulate conditions where exposure to a treatment effect is uncorrelated with other characteristics. Unlike in experimental research methods, the investigator does not have direct control over the exposure. "Natural experiments", such as regression discontinuity designs and event studies, identify existing circumstances where assignment of treatment has an exploitable element of randomness. In other cases, researchers attempt to simulate an experimental counterfactual by constructing a control group that is as similar as possible to the treatment group, using techniques such as propensity score matching.


Guidelines

The Power of Quasi-Experimental Methods

Like experimental methods, quasi-experimental methods aim to solve the problems of confoundedness and reverse or simultaneous causality in identifying the marginal effect of a change in condition. Unlike experimental methods, quasi-experiments tend to be based on policy changes or natural events that were not actually randomly assigned, but occurred in such a way that they can be considered "as good as randomly assigned" relative to some of the outcomes under consideration. Two examples illustrate these differences well.

First, consider a time-based shock that can potentially apply to all people participating in a given market, such as joining a job guarantee program. Regression Discontinuity and Event Study designs are natural choices in such a situation, but comparing across time (ie, before and after the join date) at the individual level is inappropriate, since individuals may have joined the program due to other factors that would have affected their income, such as job market seasonality, business failure, or family changes. This type of reverse causality confounds the effect of the estimate because the true counterfactual cannot be estimated.

Instead, it is necessary to use a population-level cutoff such as the implementation of the program in an area differences-in-differences or staged rollout designs, or, if only one area is implementing the policy, synthetic control methods or the assumption that events occurring on either side of the policy cutoff could not manipulate on which side of the cutoff they occurred (the running variable).

Second, consider a natural disaster that has a various impact across space, such as an earthquake with a random fault/epicenter activation. Assuming that there was no way for households to foresee the timing or location of the earthquake, and that there was no serious differential attrition on critical dimensions, it can be used as a natural experiment on outcomes later in life for affected individuals. However, for the input component to be truly exogenous, something like the distance to the faultline should be used as the treatment variable - even though estimates of local magnitude may be available.

This is because the local magnitude may be correlated with other varying characteristics (such as soil quality or hilliness) that also contribute to the outcomes (such as family income) and therefore re-introduce the confounding problem by including a non-random component that is correlated to a key characteristic. Distance from a randomly placed disaster, though a less precise measure of "exposure", is critically uncorrelated with almost any imaginable outcome under the correct assumption (unlike, for example, distance to a volcano, or to a tsunami-affected shoreline).


Additional Resources

  • list here other articles related to this topic, with a brief description and link