|(49 intermediate revisions by 4 users not shown)|
== iefieldkit =
iefieldkit provides a set of commands that enable a reproducible primary data collection and cleaning workflow. The package is developed to facilitate a workflow including (1) data collection (in particular using opendatakit.org, more specifically surveycto. com); (2) basic data cleaning, such as labeling and recoding; (3) reconciling survey rounds; (4) preparing codebooks to document data sets. iefieldkit was developed to standardize and simplify best practices for high-quality primary data collection across the World Bank' s Development Research Group Impact Evaluations team (DIME). The commands can also be used independently, and are developed to be applicable to many other contexts as well. See https://github. com/worldbank/iefieldkit for more details, or read the DIME Wiki entries for:
commands that and more . , to and primary data collection'. and are .
- [[ ieduplicates]]
- [[ iecodebook]]
The iefieldkit package is a set of commands designed to simplify a series of tedious and repetitive tasks for Stata users who are in the process of collecting primary survey data in the field. This package currently supports three major components of that workflow: survey design; survey completion; and data cleaning and survey harmonization.
This of .
One of the most important developments in economics research over the past two decades has been the rise of empirical data collection, especially with unique primary datasets collected by the researchers themselves. The authors of iefieldkit have supported the implementation of a wide range of primary data collection in fields including agriculture, health, energy and environment, edutainment, financial and private sector development, fragility, conflict, and violence, gender, governance, and transport. They have developed workflows to support general best practices for data collection, and as a rule develop new packages only when they fill an essential gap in Stata functionality. The packages here are a first attempt to provide Stata-based tools for managing the primary data collection process using native tools from start to finish.
Specifically, iefieldkit performs three essential tasks. Before data collection occurs, iefieldkit allows for rapid error-checking of ODK-based electronic surveys, including best practices for SurveyCTO-styled forms. This ensures that data, once collected, will import in Stata-friendly formats -- such as avoiding name conflicts and ensuring compliant variable naming and labelling. While data collection is ongoing, ieduplicates and iecompdup provide a workflow for detecting and resolving duplicate entries in the dataset, ensuring that the final survey dataset will be a correct record of the survey sample to merge onto the master sampling database. Finally, once data collection is complete, the iecodebook commands provide a workflow for rapidly cleaning, harmonizing, and documenting datasets.
All three commands utilize spreadsheet-based workflows so that their inputs and outputs are significantly more human-readable than Stata dofiles completing the same tasks would be, and these tasks can be supported and reviewed by personnel who specialize in field work rather than code tools. The increasing diversity and specialization of research teams has made accessibility to non-Stata-proficient personnel an essential component of data management workflows, and the iefieldkit package takes this development seriously. The code is also open-source and available for public contribution and comment on GitHub at https://github.com/worldbank/iefieldkit .
Primary data collection and cleaning involve highly repetitive but extremely important processes that contribute to high quality reproducible research. DIME Analytics has developed
iefieldkit as a package in Stata to standardize and simplify best practices involved in primary data collection.
iefieldkit consists of commands that automate: error-checking for electronic Open Data Kit (ODK)-based survey modules; duplicate checking and resolution; data cleaning and survey harmonization; and codebook creation.
One of the most important developments in economics over the past two decades has been the rise of empirical research, through primary as well as secondary data collection. The authors of
iefieldkit have developed the package to support data collection by researchers directly in a wide range of fields like agriculture, health, energy and environment, transport, financial and private sector development, gender, governance, and fragility, conflict and violence (FCV).
iefieldkit therefore supports general best practices in primary data collection from start to finish:
These four commands in this package make sure that inputs and outputs are significantly more human-readable by working with spreadsheets instead of Stata do-files. In doing so, they allow field personnel who do not specialize in code tools to understand and review the tasks involved in primary data collection.
iefieldkit thus recognizes the vital role played by field personnel in supporting data management and data cleaning even if they are not proficient in Stata.
Before Data Collection
In Open Data Kit (ODK)-based electronic survey kits, including SurveyCTO, survey forms (or questionnaires) are typically built in Excel using a specialized structured syntax. Before the research team starts with field data collection, they can use
ietestform to test Open Data Kit (ODK)-based electronic survey forms for common errors, as well as best practices for SurveyCTO-based forms.
Most ODK servers, including SurveyCTO servers, have a built-in test feature that tests the ODK syntax of a form when it is uploaded by the research team.
ietestform complements these built-in tests to ensure that the collected data is in a format that is easily readable in Stata, and warns users who use practices we have learnt are prone to data quality errors.
During Data Collection
While data collection is ongoing,
iecompdup allow researchers to test for, and resolve duplicate
entries in the dataset. The commands combine four key tasks to deal with duplicate ID values:
- Identifying duplicate entries.
- Comparing observations with the same ID value.
- Tracking and documenting any changes made to the identifying variable.
- Applying the necessary corrections to the data.
Together these commands ensure that the collected data will be a correct record of the sample, and can be merged with the master database. Both these commands were previously part of the
ietoolkit package, but have now been moved to
After Data Collection
After data collection is complete,
iecodebook allows the research team to automate the repetitive tasks involved in cleaning data before it can be analyzed. As the name suggests, the
iecodebook command is structured around Excel-based codebooks, which allows researchers to perform and document data cleaning tasks in Excel itself, instead of do-files. Therefore, codebooks allow researchers to document the cleaned data in a format that is both human and machine-readable.
iecodebook implements this through 4 subcommands:
iecodebook apply applies rename, recode, and/or label commands to a large number of variables in the dataset.
iecodebook append harmonizes two or more datasets, and appends them. That is, it allows two or more datasets to have the same variable names, labels, and value labels.
iecodebook export creates an Excel codebook that describes the current dataset. It can also produce an exportable version of the dataset which only contains the variables used in a particular do-file.
iecodebook template creates an Excel template that describes the current or targeted dataset(s), and prepares the codebook for the other subcommands in
Click here to see pages that link to this topic.
This page is part of the topic Stata coding practices.