Difference between revisions of "Back Checks"
Maria jones (talk | contribs) |
|||
| Line 1: | Line 1: | ||
Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, | Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, a randomly-selected subset of households are re-interviewed with a very short questionnaire to verify and determine the legitimacy of key data collected in the actual survey. | ||
==Read First == | ==Read First == | ||
Back checks are an important tool to detect fraud, i.e. enumerators sitting under a tree and filling out questionnaires themselves, and to assess the accuracy of the data collected. Back checks can be conducted by in-person visits or phone calls. A complementary approach to in-person back checks is to do [[Random Audio Audits]]. | |||
Back checks are | |||
==Best Practices during back checks == | ==Best Practices during back checks == | ||
Here are some of the best practices for back checks: | |||
Here are some of the best practices | * Aim to back check at least 10% of the total observations | ||
* | * Back checks should also be front-heavy i.e. majority of them occurring in the first few days / weeks of data collection. This helps find whether the questionnaire/enumerators are doing their jobs well and can be remedied through training/replacement. | ||
*Back checks should also be front-heavy i.e. majority of them occurring | *The back check sample should be stratified across survey teams/surveyors. | ||
*The back | * The back checks should be done in person by an independent third party. | ||
* It is important that enumerators do not know what questions will be audited. to that end, many people randomly select a small number of questions from the survey instrument to back check, and change the back check form regularly during data collection. | |||
* The back checks should be done in person by an independent third party. | |||
==How to Select Back Check Questions == | ==How to Select Back Check Questions == | ||
Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality, i.e. questionnaire language, surveyor performance, | Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality, i.e. questionnaire language, surveyor performance, survey fraud, etc. Some of the questions that should be asked during a back check are as follows: | ||
To test for | * To test for translation issues, back check questions which can be interpreted differently by different surveyors. * | ||
* To test whether enumerators are falsifying data to shorten interviews, back check questions that determine repeated sections of the questionnaire. For example, if there is a long series of questions about household members, verify the correct number of household members. If an agricultural survey asks for production information by plot, verify the number of plots. | |||
* To test for fraud, check simply that an enumerator visited the household and conducted an interview with the correct respondent | |||
==A framework for back checks from Innovations for Poverty Action== | ==A framework for back checks from Innovations for Poverty Action== | ||
| Line 57: | Line 51: | ||
== Back to Parent == | == Back to Parent == | ||
This article is part of the topic | This article is part of the topic [[Field Management]]. | ||
== Additional Resources == | == Additional Resources == | ||
* World Health Organization's [http://unstats.un.org/unsd/hhsurveys/pdf/Chapter_10.pdf '''Quality Assurance in Surveys: standards, guidelines, and procedures''']. This chapter provides, in detail, the approach and methodology on quality control during surveys. | * World Health Organization's [http://unstats.un.org/unsd/hhsurveys/pdf/Chapter_10.pdf '''Quality Assurance in Surveys: standards, guidelines, and procedures''']. This chapter provides, in detail, the approach and methodology on quality control during surveys. | ||
[[Category: | [[Category: Field Management ]] | ||
Revision as of 15:53, 9 February 2018
Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, a randomly-selected subset of households are re-interviewed with a very short questionnaire to verify and determine the legitimacy of key data collected in the actual survey.
Read First
Back checks are an important tool to detect fraud, i.e. enumerators sitting under a tree and filling out questionnaires themselves, and to assess the accuracy of the data collected. Back checks can be conducted by in-person visits or phone calls. A complementary approach to in-person back checks is to do Random Audio Audits.
Best Practices during back checks
Here are some of the best practices for back checks:
- Aim to back check at least 10% of the total observations
- Back checks should also be front-heavy i.e. majority of them occurring in the first few days / weeks of data collection. This helps find whether the questionnaire/enumerators are doing their jobs well and can be remedied through training/replacement.
- The back check sample should be stratified across survey teams/surveyors.
- The back checks should be done in person by an independent third party.
- It is important that enumerators do not know what questions will be audited. to that end, many people randomly select a small number of questions from the survey instrument to back check, and change the back check form regularly during data collection.
How to Select Back Check Questions
Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality, i.e. questionnaire language, surveyor performance, survey fraud, etc. Some of the questions that should be asked during a back check are as follows:
- To test for translation issues, back check questions which can be interpreted differently by different surveyors. *
- To test whether enumerators are falsifying data to shorten interviews, back check questions that determine repeated sections of the questionnaire. For example, if there is a long series of questions about household members, verify the correct number of household members. If an agricultural survey asks for production information by plot, verify the number of plots.
- To test for fraud, check simply that an enumerator visited the household and conducted an interview with the correct respondent
A framework for back checks from Innovations for Poverty Action
The following framework for back checks has been developed by Innovation for Poverty Action.
- Identifying Respondents and Interview Information
- - Check if we have the right person
- - Check if they interview took place and when did it take place.
- Type 1 Variables
- - Straightforward questions where we expect no variation.
- - For example - education level, marital status, occupation, has children or not, etc.
- Type 2 Variables
- - Questions where we expect capable enumerators to get the true answer.
- Type 3 Variables
- - Questions that we expect to be difficult. We back check these questions to understand if they were correctly interpreted in the field.
The total duration of the back checks should be around 10-15 minutes.
Comparing Back Checks to Actual Survey Data
After completing a back check, you can now compare the data obtained from the back check to your actual survey data. This can be done by using the Stata command bcstats developed by Innovations for Poverty Action. This command compares the back check data and the survey data, and produces a data set of the comparisons between the two data sets. The command also completes enumerator checks and stability checks for variables.
The steps are as follows:
ssc install bcstats .
bcstats, surveydata(filename) bcdata(filename) id(varlist) [options]
To learn about the options for bcstats and survey back checks, please type help bcstats on Stata after installing the command.
Back to Parent
This article is part of the topic Field Management.
Additional Resources
- World Health Organization's Quality Assurance in Surveys: standards, guidelines, and procedures. This chapter provides, in detail, the approach and methodology on quality control during surveys.