Difference between revisions of "Data Analysis"
| (7 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
Data analysis | Data analysis is the process of exploring and describing trends and results from data. Data analysis typically occurs in two stages: exploratory analysis and final analysis. This page provides guidance on how to organize analysis files and output results in an orderly and [[Reproducible Research | reproducible]] manner. | ||
==Read First== | ==Read First== | ||
* | * Always [[Data Cleaning | clean]] data before conducting data analysis. | ||
* Place exploratory analysis files in a separate, well-organized "exploratory" folder | * Place exploratory analysis files in a separate, well-organized "exploratory" folder; place the final analysis file in the "final" analysis folder. | ||
* Create tables and figures via [[Reproducible Research|replicable]] '''do-files''' in such a way that the result files are ordered, named, placed, and formatted appropriately. | |||
* | |||
==Exploratory vs. Final Analysis== | |||
Exploratory analysis focuses on producing easily understood summaries of the trends in the data so that researchers can begin to outline reports, [[Publishing Data|publications]], presentations, and summaries. Final analysis is the fine-tuned culmination of exploratory analysis and requires re-written code appropriate for public release with the results. | |||
== Organizing Analysis Files == | |||
Place exploratory analysis programs in an "exploratory" folder, separated according to topic. When folder syncing over [https://www.dropbox.com Dropbox] or [[Getting Started with GitHub|Github]], separating these files by function (rather than combining them into a single "analysis" file) allows multiple researchers to work simultaneously and modularly. | |||
When the final analysis workflow is agreed upon for a given publication or other output, a final analysis file should be collated for that output only in the "final analysis" folder. This allows selective reuse of the code from the exploratory analyses, in preparation for the final release of the code if required. This allows any collaborator, referee, or replicator to access only the code used to prepare the final outputs and [[Reproducible Research | reproduce]] them exactly. | |||
== Implementing Analysis == | == Implementing Analysis == | ||
Below follows a sampling of specific analytical methods. | |||
*[[Spatial Analysis |Spatial/GIS Analysis]] uses geospatial data to explore relationships mediated by proximity or connectiveness. This can be descriptive (i.e. map illustrations) or informative (i.e. distance to and quality of the nearest road). | *[[Spatial Analysis |Spatial/GIS Analysis]] uses geospatial data to explore relationships mediated by proximity or connectiveness. This can be descriptive (i.e. map illustrations) or informative (i.e. distance to and quality of the nearest road). | ||
*[[Randomization Inference | Randomization inference]] techniques replace the "normal" p-values from regression analyses with values based on the treatment assignment methodology. They are generally recommended for reporting in experiments whose estimates are of randomly assigned treatment controlled by the implementer and researcher. | *[[Randomization Inference | Randomization inference]] techniques replace the "normal" p-values from regression analyses with values based on the '''treatment assignment''' methodology. They are generally recommended for reporting in experiments whose estimates are of [[Randomization|randomly]] assigned '''treatment''' controlled by the implementer and researcher. | ||
*[[Cost-effectiveness Analysis | Cost-effectiveness analysis]] compares the cost and effectiveness per unit of a given program to determine whether the value of an intervention justifies its cost. | *[[Cost-effectiveness Analysis | Cost-effectiveness analysis]] compares the cost and effectiveness per unit of a given program to determine whether the value of an intervention justifies its cost. | ||
*[[Regression Discontinuity|Regression discontinuity]] analysis is a [[Quasi-Experimental Methods | quasi-experimental]] impact evaluation design which attempts to find the causal effects of interventions by assigning a threshold (cutoff point) above and below which the treatment is assigned | *[[Regression Discontinuity|Regression discontinuity]] analysis is a [[Quasi-Experimental Methods | quasi-experimental]] '''impact evaluation''' design which attempts to find the causal effects of interventions by assigning a threshold (cutoff point) above and below which the '''treatment''' is assigned. | ||
*[[Propensity Score Matching | Propensity score matching]] is another quasi-experimental impact evaluation technique to estimate the effects of a treatment by matching control group participants to treatment group participants based on propensity score | *[[Propensity Score Matching | Propensity score matching]] is another '''quasi-experimental impact evaluation''' technique to estimate the effects of a '''treatment''' by matching control group participants to '''treatment group''' participants based on '''propensity score'''. | ||
*Heterogeneous Effects Analysis uses an [https://web.stanford.edu/~jgrimmer/het.pdf ensemble of methods] to understand how effects vary across sub-populations | *Heterogeneous Effects Analysis uses an [https://web.stanford.edu/~jgrimmer/het.pdf ensemble of methods] to understand how effects vary across sub-populations | ||
*[[Data visualization]] is a critical step in effectively communicating your research results. | *[[Data visualization]] is a critical step in effectively communicating your research results. | ||
The [[Stata Coding Practices | Stata]] cheat sheet on [http://geocenter.github.io/StataTraining/pdf/StataCheatSheet_analysis_201615_June-REV.pdf data analysis] gives guidelines on relevant '''Stata''' code for analysis. The cheat sheet on [http://geocenter.github.io/StataTraining/pdf/StataCheatSheet_programming15_2016_June_TE-REV.pdf Stata programming] is a good resource for more advanced analytical tasks in '''Stata'''. | |||
== Outputting Analytical Results == | == Outputting Analytical Results == | ||
Since the final analysis do-files are intended to be fully replicable | Since the final analysis '''do-files''' are intended to be fully [[Reproducible Research|replicable]] and the code itself is considered a vital, shareable output, all tables and figures should be created in such a way that the files are ordered, [[Naming Conventions | named]], placed, and formatted appropriately. Running the analysis '''do-file''' should result in only necessary files in the "outputs" folder, with names like "figure_1.png", "table_1.xlsx", and so on. | ||
For some applications (i.e. creating internal presentations or simple Word reports), file types like PNG and XLSX are sufficiently functional. For larger projects with multiple collaborators, particularly when syncing over a [ | For some applications (i.e. creating internal presentations or simple Word reports), file types like PNG and XLSX are sufficiently functional. For larger projects with multiple collaborators, particularly when syncing over a [[Getting Started with GitHub|GitHub]] service, plaintext file types such as EPS, CSV, and TEX will be the preferred formats. Tables and figures should at minimum be produced by these files such that no further mathematical calculations are required. They should furthermore be organized and formatted as nearly to the [[Publishing Data|published]] versions as possible. It is typically easy to do this with figures by using an appropriate <code>graph export</code> command in [[Stata Coding Practices|Stata]] or the equivalent. [https://www.latex-project.org LaTeX] is a particularly powerful tool for doing this with tables. DIME provides several guides on both processes. See [[Exporting Analysis |exporting analysis results]] for more details and more resources. | ||
== Additional Resources == | == Additional Resources == | ||
*Angrist and Pischke's [https://www.researchgate.net/publication/51992844_Mostly_Harmless_Econometrics_An_Empiricist's_Companion Mostly Harmless Econometrics: An Empiricist's Companion] | |||
[[Category: Data Analysis ]] | [[Category: Data Analysis ]] | ||
Latest revision as of 18:27, 9 August 2023
Data analysis is the process of exploring and describing trends and results from data. Data analysis typically occurs in two stages: exploratory analysis and final analysis. This page provides guidance on how to organize analysis files and output results in an orderly and reproducible manner.
Read First
- Always clean data before conducting data analysis.
- Place exploratory analysis files in a separate, well-organized "exploratory" folder; place the final analysis file in the "final" analysis folder.
- Create tables and figures via replicable do-files in such a way that the result files are ordered, named, placed, and formatted appropriately.
Exploratory vs. Final Analysis
Exploratory analysis focuses on producing easily understood summaries of the trends in the data so that researchers can begin to outline reports, publications, presentations, and summaries. Final analysis is the fine-tuned culmination of exploratory analysis and requires re-written code appropriate for public release with the results.
Organizing Analysis Files
Place exploratory analysis programs in an "exploratory" folder, separated according to topic. When folder syncing over Dropbox or Github, separating these files by function (rather than combining them into a single "analysis" file) allows multiple researchers to work simultaneously and modularly.
When the final analysis workflow is agreed upon for a given publication or other output, a final analysis file should be collated for that output only in the "final analysis" folder. This allows selective reuse of the code from the exploratory analyses, in preparation for the final release of the code if required. This allows any collaborator, referee, or replicator to access only the code used to prepare the final outputs and reproduce them exactly.
Implementing Analysis
Below follows a sampling of specific analytical methods.
- Spatial/GIS Analysis uses geospatial data to explore relationships mediated by proximity or connectiveness. This can be descriptive (i.e. map illustrations) or informative (i.e. distance to and quality of the nearest road).
- Randomization inference techniques replace the "normal" p-values from regression analyses with values based on the treatment assignment methodology. They are generally recommended for reporting in experiments whose estimates are of randomly assigned treatment controlled by the implementer and researcher.
- Cost-effectiveness analysis compares the cost and effectiveness per unit of a given program to determine whether the value of an intervention justifies its cost.
- Regression discontinuity analysis is a quasi-experimental impact evaluation design which attempts to find the causal effects of interventions by assigning a threshold (cutoff point) above and below which the treatment is assigned.
- Propensity score matching is another quasi-experimental impact evaluation technique to estimate the effects of a treatment by matching control group participants to treatment group participants based on propensity score.
- Heterogeneous Effects Analysis uses an ensemble of methods to understand how effects vary across sub-populations
- Data visualization is a critical step in effectively communicating your research results.
The Stata cheat sheet on data analysis gives guidelines on relevant Stata code for analysis. The cheat sheet on Stata programming is a good resource for more advanced analytical tasks in Stata.
Outputting Analytical Results
Since the final analysis do-files are intended to be fully replicable and the code itself is considered a vital, shareable output, all tables and figures should be created in such a way that the files are ordered, named, placed, and formatted appropriately. Running the analysis do-file should result in only necessary files in the "outputs" folder, with names like "figure_1.png", "table_1.xlsx", and so on.
For some applications (i.e. creating internal presentations or simple Word reports), file types like PNG and XLSX are sufficiently functional. For larger projects with multiple collaborators, particularly when syncing over a GitHub service, plaintext file types such as EPS, CSV, and TEX will be the preferred formats. Tables and figures should at minimum be produced by these files such that no further mathematical calculations are required. They should furthermore be organized and formatted as nearly to the published versions as possible. It is typically easy to do this with figures by using an appropriate graph export command in Stata or the equivalent. LaTeX is a particularly powerful tool for doing this with tables. DIME provides several guides on both processes. See exporting analysis results for more details and more resources.
Additional Resources
- Angrist and Pischke's Mostly Harmless Econometrics: An Empiricist's Companion