Difference between revisions of "Back Checks"

Jump to: navigation, search
Line 1: Line 1:
Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, certain households are re-interviewed for certain questions to verify and determine the legitimacy of the data collected in the actual survey.  
+
Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, a randomly-selected subset of households are re-interviewed with a very short questionnaire to verify and determine the legitimacy of key data collected in the actual survey.  
  
 
==Read First ==
 
==Read First ==
 
+
Back checks are an important tool to detect fraud, i.e. enumerators sitting under a tree and filling out questionnaires themselves, and to assess the accuracy of the data collected. Back checks can be conducted by in-person visits or phone calls. A complementary approach to in-person back checks is to do [[Random Audio Audits]].
 
 
== Purpose==
 
Back checks are done to monitor the quality of the field work. This gives us valuable information on whether the questionnaire accurately captures the key outcomes of the study, and on whether enumerators are performing as well as expected. A complementary approach to in-person back checks is to do [[Random Audio Audits]].
 
  
 
==Best Practices during back checks ==  
 
==Best Practices during back checks ==  
 
+
Here are some of the best practices for back checks:
Here are some of the best practices that should be done while performing back checks:
+
* Aim to back check at least 10% of the total observations
*The number of back checks that can be done depends on the budget of the survey team. The survey team should aim for at least 10% of the total observations.
+
* Back checks should also be front-heavy i.e. majority of them occurring in the first few days / weeks of data collection. This helps find whether the questionnaire/enumerators are doing their jobs well and can be remedied through training/replacement.   
*Back checks should also be front-heavy i.e. majority of them occurring early in the survey. This helps find whether the questionnaire/enumerators are doing their jobs well and can be remedied through training/replacement.   
+
*The back check sample should be stratified across survey teams/surveyors.  
*The back checks should include surveys done by all the survey teams/surveyors. Households should be selected randomly from these teams.
+
* The back checks should be done in person by an independent third party.  
*The back check sample should be proportional in terms of respondent selection with the actual survey.
+
* It is important that enumerators do not know what questions will be audited. to that end, many people randomly select a small number of questions from the survey instrument to back check, and change the back check form regularly during data collection.
* The back checks should be done in person by an independent third party. In some contexts (where mobile phone penetration is very high), doing back checks by phone is possible.
 
  
 
==How to Select Back Check Questions ==
 
==How to Select Back Check Questions ==
Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality, i.e. questionnaire language, surveyor performance, [[Computer-Assisted_Personal_Interviews_(CAPI)|CAPI]] errors etc.  Some of the questions that should be asked during a back check are as follows:  
+
Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality, i.e. questionnaire language, surveyor performance, survey fraud, etc.  Some of the questions that should be asked during a back check are as follows:  
 
 
To test for questionnaire language, back checks can be done on questions which can be interpreted differently by different surveyors. Asking questions that can be interpreted different during the survey and the back check provides the survey team with the knowledge on whether or not the surveyor is interpreting a question correctly.
 
 
 
Testing surveyor performance can be done using questions which cannot have different answers at different times. Simple questions like the age of the respondent, or the number of member in the households are questions who should not differ between the survey and the back check.
 
  
To test for [[Computer-Assisted_Personal_Interviews_(CAPI)|CAPI]] errors, question sections with complex skips can be tested.
+
* To test for translation issues, back check questions which can be interpreted differently by different surveyors. *
 +
* To test whether enumerators are falsifying data to shorten interviews, back check questions that determine repeated sections of the questionnaire. For example, if there is a long series of questions about household members, verify the correct number of household members. If an agricultural survey asks for production information by plot, verify the number of plots.
 +
* To test for fraud, check simply that an enumerator visited the household and conducted an interview with the correct respondent
  
 
==A framework for back checks from Innovations for Poverty Action==
 
==A framework for back checks from Innovations for Poverty Action==
Line 57: Line 51:
  
 
== Back to Parent ==
 
== Back to Parent ==
This article is part of the topic, [[Monitoring Data Quality]].
+
This article is part of the topic [[Field Management]].
  
 
== Additional Resources ==
 
== Additional Resources ==
 
* World Health Organization's  [http://unstats.un.org/unsd/hhsurveys/pdf/Chapter_10.pdf '''Quality Assurance in Surveys: standards, guidelines, and procedures''']. This chapter provides, in detail,  the approach and methodology on quality control during surveys.
 
* World Health Organization's  [http://unstats.un.org/unsd/hhsurveys/pdf/Chapter_10.pdf '''Quality Assurance in Surveys: standards, guidelines, and procedures''']. This chapter provides, in detail,  the approach and methodology on quality control during surveys.
  
[[Category: Data Quality ]]
+
[[Category: Field Management ]]

Revision as of 15:53, 9 February 2018

Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, a randomly-selected subset of households are re-interviewed with a very short questionnaire to verify and determine the legitimacy of key data collected in the actual survey.

Read First

Back checks are an important tool to detect fraud, i.e. enumerators sitting under a tree and filling out questionnaires themselves, and to assess the accuracy of the data collected. Back checks can be conducted by in-person visits or phone calls. A complementary approach to in-person back checks is to do Random Audio Audits.

Best Practices during back checks

Here are some of the best practices for back checks:

  • Aim to back check at least 10% of the total observations
  • Back checks should also be front-heavy i.e. majority of them occurring in the first few days / weeks of data collection. This helps find whether the questionnaire/enumerators are doing their jobs well and can be remedied through training/replacement.
  • The back check sample should be stratified across survey teams/surveyors.
  • The back checks should be done in person by an independent third party.
  • It is important that enumerators do not know what questions will be audited. to that end, many people randomly select a small number of questions from the survey instrument to back check, and change the back check form regularly during data collection.

How to Select Back Check Questions

Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality, i.e. questionnaire language, surveyor performance, survey fraud, etc. Some of the questions that should be asked during a back check are as follows:

  • To test for translation issues, back check questions which can be interpreted differently by different surveyors. *
  • To test whether enumerators are falsifying data to shorten interviews, back check questions that determine repeated sections of the questionnaire. For example, if there is a long series of questions about household members, verify the correct number of household members. If an agricultural survey asks for production information by plot, verify the number of plots.
  • To test for fraud, check simply that an enumerator visited the household and conducted an interview with the correct respondent

A framework for back checks from Innovations for Poverty Action

The following framework for back checks has been developed by Innovation for Poverty Action.

Identifying Respondents and Interview Information
- Check if we have the right person
- Check if they interview took place and when did it take place.
Type 1 Variables
- Straightforward questions where we expect no variation.
- For example - education level, marital status, occupation, has children or not, etc.
Type 2 Variables
- Questions where we expect capable enumerators to get the true answer.
Type 3 Variables
- Questions that we expect to be difficult. We back check these questions to understand if they were correctly interpreted in the field.

The total duration of the back checks should be around 10-15 minutes.

Comparing Back Checks to Actual Survey Data

After completing a back check, you can now compare the data obtained from the back check to your actual survey data. This can be done by using the Stata command bcstats developed by Innovations for Poverty Action. This command compares the back check data and the survey data, and produces a data set of the comparisons between the two data sets. The command also completes enumerator checks and stability checks for variables.

The steps are as follows:

ssc install bcstats
bcstats, surveydata(filename) bcdata(filename) id(varlist) [options]
.

To learn about the options for bcstats and survey back checks, please type help bcstats on Stata after installing the command.

Back to Parent

This article is part of the topic Field Management.

Additional Resources