Back Checks

Back checks are quality control method used to verify data collected during a survey. After survey data has been collected, certain households are re-interviewed for certain questions to verify and determine the legitimacy of the data collected in the actual survey.

Purpose

Back checks are done to monitor the quality of the field work. This gives us valuable information on whether the questionnaire accurately captures the key outcomes of the study or not, and on whether the enumerators are performing their jobs as expected. A complementary approach to in-person back checks is to do Random Audio Audits.

Best Practices during back checks

Here are some of the best practices that should be done while performing back checks:

• The number of back checks that can be done depends on the budget of the survey team. The survey team should aim for at least 10% of the total observations.
• Back checks should also be front-heavy i.e. majority of them occurring early in the survey. This helps find whether the questionnaire/enumerators are doing their jobs well and can be remedied through training/replacement.
• The back checks should include surveys done by all the survey teams/surveyors. Households should be selected randomly from these teams.
• The back check sample should be proportional in terms of respondent selection with the actual survey.
• The back checks should be done in person by an independent third party. In some contexts (where mobile phone penetration is very high), doing back checks by phone is possible.

How to Select Back Check Questions

Back check questions should be selected with the performance of both the questionnaire and the surveyor in mind. Using different types of questions during the back check helps in finding the cause of poor data quality i.e. questionnaire language, surveyor performance, CAPI errors etc. Some of the questions that should be asked during a back check are as follows:

To test for questionnaire language, back checks can be done on questions which can be interpreted differently by different surveyors. Asking questions that can be interpreted different during the survey and the back check provides the survey team with the knowledge on whether or not the surveyor is interpreting a question correctly.

Testing surveyor performance can be done using questions which cannot have different answers at different times. Simple questions like the age of the respondent, or the number of member in the households are questions who should not differ between the survey and the back check.

To test for CAPI errors, question sections with complex skips can be tested.

A framework for back checks from Innovations for Poverty Action

The following framework for back checks has been developed by Innovation for Poverty Action

Identifying Respondents and Interview Information

- Check if we have the right person

- Check if they interview took place and when did it take place.

Type 1 Variables

- Straightforward questions where we expect no variation.

- For example - education level, marital status, occupation, has children or not, etc.

Type 2 Variables

- Questions where we expect capable enumerators to get the true answer.

Type 3 Variables

- Questions that we expect to be difficult. We back check these questions to understand if they were correctly interpreted in the field.

The total duration of the back checks should be around 10-15 minutes.

Comparing Back Checks to Actual Survey Data

After completing a back check, you can now compare the data obtained from the back check to your actual survey data. This can be done by using the Stata command bcstats developed by Innovations for Poverty Action. This command compares the back check data and the survey data, and produces a data set of the comparisons between the two data sets. The command also completes enumerator checks and stability checks for variables.

The steps are as follows:

ssc install bcstats
bcstats, surveydata(filename) bcdata(filename) id(varlist) [options] .

To learn about the options for bcstats and survey back checks, please type help bcstats on Stata after installing the command.

Back to Parent

This article is part of the topic Monitoring Data Quality

Additional Resources

• World Health Organization's Quality Assurance in Surveys: standards, guidelines, and procedures. This chapter provides, in detail, the approach and methodology on quality control during surveys.