Back Checks

Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, certain households are re-interviewed for certain questions to verify and determine the legitimacy of the data collected in the actual survey.

Purpose

Back checks are done to monitor the quality of the field work. This gives us valuable information on whether the questionnaire accurately captures the key outcomes of the study or not, and on whether the enumerators are performing their jobs as expected.

Best Practices during back checks

Here are some of the best practices that should be done while performing back checks:

• The number of back checks that can be done depends on the budget of the survey team. The survey team should aim for at least 10% of the total observations.
• Back checks should also be front-heavy i.e. majority of them occurring early in the survey. This helps find whether the questionnaire/enumerators are doing their jobs well and can be remedied through training/replacement.
• The back checks should include surveys done by all the survey teams/surveyors. Households should be selected randomly from these teams.
• The back check sample should be proportional in terms of respondent selection with the actual survey.

How to Select Back Check Questions

Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality i.e. questionnaire language, surveyor performance, CAPI errors etc. Some of the questions that should be asked during a back check are as follows:

To test for questionnaire language, back checks can be done on questions which can be interpreted differently by different surveyors. Asking questions that can be interpreted different during the survey and the back check provides the survey team with the knowledge on whether or not the surveyor is interpreting a question correctly.

Testing surveyor performance can be done using questions which can not have different answers at different times. Simple questions like the age of the respondent, or the number of member in the households are questions who should not differ between the survey and the back check.

To test for CAPI errors, question sections with complex skips can be tested.

A framework for back checks from Innovations for Poverty Action

The following framework for back checks has been developed by Innovation for Poverty Action

Identifying Respondents and Interview Information

- Check if we have the right person

- Check if they interview took place and when did it take place.

Type 1 Variables

-Straightforward questions where we expect no variation.

-For example - education level, marital status, occupation, has children or not, etc.

Type 2 Variables

- Questions where we expect capable enumerators to get the true answer.

Type 3 Variables

- Questions that we expect to be difficult. We back check these questions to understand if they were correctly interpreted in the field.

The total duration of the back checks should be around 10-15 minutes.

Comparing Back Checks to Actual Survey Data

After completing a back check, you can now compare the data obtained from the back check to your actual survey data. This can be done by using the Stata command bcstats developed by Innovations for Poverty Action. This command compares the back check data and the survey data, and produces a data set of the comparisons between the two data sets. The command also completes enumerator checks and stability checks for variables.

The steps are as follows:

ssc install bcstats
bcstats, surveydata(filename) bcdata(filename) id(varlist) [options] .

To learn about the options for bcstats, please type help bcstats on Stata after installing the command.

Action to take after back checks

The three types of questions asked during the back check helps determine whether the problems in the data are due to the surveyor or the questionnaire. The remedial actions after back checks are as follows:

Type 1 data

Since type 1 variables should have little to no variation between the main survey and the back check, discrepancies in the data are most likely due to surveyor errors. A breakdown of the discrepancy percentage and the suggested corrective measures are as follows:

• More than 10% discrepancy - You should warn the surveyor.
• Discrepancy of 20-30% - 2nd back check needs to be conducted to correct the errors.
  • If the errors are surveyor errors, then 3 additional surveys by the surveyors in the same week should be audited. If 20-30% discrepancies are found in those surveys as well, then the surveyor should be put on probation.
• Discrepancy of more than 40%- 2nd back check to determine who made the errors and maybe resurvey the household. If the surveyor made the errors, resurvey the household and audit all the surveys done by the surveyor in the batch.
  • If one more survey has more than 40% discrepancy, fire the surveyor immediately and redo all surveys with 20% or more discrepancy.

Type 2 Data

Since we expect qualified surveyors to get the answers to type 2 questions, there shouldn't be much variation in the data from the survey and the data from the back check. High discrepancy could mean that the surveyors are not particularly well trained to ask the question. Some suggested corrective measures are as follows:

• If the discrepancy is more than 10%, consider retraining the surveyor.
• If a particular surveyor is responsible for more than 30% of the errors in the single survey, follow the steps for Type 1.

Type 3 Data

• If the discrepancy is more than 10%, discuss with your survey team and let your PIs know. They may decide to edit the survey or add additional rounds of surveying.

Back to Parent

This article is part of the topic Monitoring Data Quality

Additional Resources

• World Health Organization's [unstats.un.org/unsd/hhsurveys/pdf/Chapter_10.pdf Quality Assurance in Surveys: standards, guidelines, and procedures]. This chapter provides, in detail, the approach and methodology on quality control during surveys.