Data Quality Assurance Plan

Quality data is data that is not systematically biased and does not misstate representativeness or coverage. A data quality assurance plan considers everything in data collection that could go wrong ahead of time and makes a plan to preempt these issues. The plan should be shared with all impact evaluation stakeholders – including the impact evaluation team and the survey firm – before data collection starts. This page covers the reasons behind low quality data and outlines key processes to include in the data quality assurance plan.

Read First

• Data can be low quality because of respondent fatigue, enumerator error, or poor quality control.
• Good questionnaire design and programming, back-checks and high frequency checks help to ensure high quality data.

Why Can Data be Low Quality?

The quality of primary data is at risk of low quality because of three main reasons:

• Respondents are human, so they have imperfect recall and motivation and/or can become tired or annoyed.
• Enumerators are human, so they can make mistakes, quickly fill answers to unasked questions when they’re sure they know the answer, or even just conduct fake interviews. Granted, they have really hard jobs! They are often travelling to new places and meeting people who are more or less friendly; dealing with weather, pollution, congestion and other conditions; tracking down hard-to-find respondents; and sometimes trying to decipher unclear instructions.
• Research assistants are human, so they often fail to implement quality-control efforts in a timely manner, they stay away from conflict and don’t confront underperforming staff and they operate with chronic shortages of time and prior experience.

There are three key lines of defense help to overcome these challenges to data quality: survey design, field management, and high frequency checks.

3 Elements of Data Quality Assurance Plan

Survey Design

Within the survey design:

• use constraints and relevance.
• preload data when possible to reduce error.
• when relevant, use SurveyCTO’s case management to pre-assign surveys and/or treatments to individuals.

Make sure the questions are clearly worded in all languages used and do not leave room for nuanced interpretations. Pilot the survey intensely to fine tune these questions and logic. Then, train enumerators well in order to ensure that the questionnaire is clear, intuitive, and easy to conduct as intended.

Field Management

During data collection, maintain close contact with enumerators and managers and give them a “buy-in” to quality; check in constantly and make them feel valued. Consider that unless egregious or fraudulent, the errors are not the field team’s but rather yours. As part of the data quality assurance plan, take the following steps during field management:

• Spot checks/accompaniments:
  • By you: spend the first two weeks shadowing your enumerators and observing how each of them administers the survey. Give them useful feedback after – not during – the interviews
  • By team leaders: team leaders are experienced enumerators who have diverse tasks, the most important of which is observing interviews and supporting their team members in perfecting their interviewing abilities. Give team leaders a form to write down their observations for each enumerator and ask them to submit these forms.
• Tracking sheets: give tracking sheets to enumerators on which they should track every day the information of completed surveys, units not found, etc. Enumerators should enter this information every day as a core task and present it frequently to team leaders for accountability.
• Back-checks:
  • Compare answers with original survey. Every team and surveyor must be back-checked as soon as possible and regularly
• Cover 10% of the same, with 20% being administered in the first two weeks of field work. *Cover a random sub-sample, include missing respondents to verify that your team is not biasing the sample by not tracking hard to find respondents, check observations flagged in other quality tests, and run backchecks on enumerators suspected of cheating.
• See more information on Back Checks.

High Frequency Checks

High frequency checks allow the research team to check quality, catch key mistakes early, and provide visible performance metrics for enumerators. For more information on how to implement high frequency checks, see High Frequency Checks.

Back to Parent

This article is part of the topic Field Management

Additional Resources

• DIME Analytics (World Bank), Data Quality Checks
• DIME Analytics (World Bank), Assuring Data Quality
• DIME's Planning for, Preparing & Monitoring Household Surveys
• UN World Health Surveys Guidance on survey quality assurance
• SurveyCTO’s Survey Design for Quality Data