Difference between revisions of "Preparing for Field Data Collection"

Jump to: navigation, search
 
(50 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''Preparing for field data collection''' involves preparing clear [[Survey Protocols|protocols]] and guidelines for each component of collecting data using [[Field Surveys|field surveys]]. The main components of field data collection are  [[Timeline of Survey Pilot|timelines]], [[IRB Approval|
'''Preparing for field data collection''' involves preparing clear [[Survey Protocols|protocols]] and guidelines for each component of [[Primary Data Collection|primary data collection]] using [[Field Surveys|field surveys]]. This page looks at the '''best practices''' and '''protocols''' that the research team must follow before starting with the process of [[Primary Data Collection|primary data collection]].  
IRB approvals]], [[Questionnaire Design|instrument design]], [[Questionnaire Programming|coding surveys]], and [[Enumerator Training|enumerator training]]. Each of these components must be carried out based on '''best practices''' in field data collection. Therefore, the [[Impact Evaluation Team|research team]] must establish clear channels for regular communication among all participants- [[Impact_Evaluation_Team#Principal_Investigator|principal investigators (PIs)]], [[Impact_Evaluation_Team#Field_Coordinator|field coordinators (FCs)]], [[Survey Firm|survey firm]], '''government agencies''',  and '''enumerators'''.
 
== Read First ==
== Read First ==
* The research team can either conduct the survey directly, or indirectly through a [[Survey Firm|survey firm]].
* [[Primary Data Collection|Primary data collection]] details the workflow from start to finish involved in gathering data through [[Field Surveys|surveys]], interviews, or experiments.
* The [[Impact Evaluation Team|research team]] can either conduct the survey directly, or indirectly through a [[Survey Firm|survey firm]].
* Spending more time on the preparation leads to better quality data.
* Spending more time on the preparation leads to better quality data.
* Plan for field data collection at least 6 months in advance of the [[Field Surveys#Survey Launch|survey launch]].  
* Plan for field data collection at least 6 months in advance of the [[Field Surveys#Survey Launch|survey launch]].  
Line 9: Line 8:
* With the availability of specialized [[Survey Firm|survey firms]], and standardized [[Field Management|field management practices]], it is important for researchers to follow '''best practices''' to collect field data.
* With the availability of specialized [[Survey Firm|survey firms]], and standardized [[Field Management|field management practices]], it is important for researchers to follow '''best practices''' to collect field data.


== Survey Timeline ==
== Preparing the Survey Timeline ==
Preparing the timeline for a field survey involves allocating sufficient time for each stage of the survey process - [[Questionnaire Design|instrument design]], [[Survey Pilot|pilot]], [[IRB Approval|obtaining approval]], [[Procuring a Survey Firm|procurement]], [[Survey Protocols|drafting protocols]], and [[Enumerator Training|training]].  
Preparing the timeline for a field survey involves allocating sufficient time for each stage of the survey process - [[Questionnaire Design|instrument design]], [[Survey Pilot|pilot]], [[IRB Approval|obtaining approval]], [[Procuring a Survey Firm|procurement]], [[Survey Protocols|drafting protocols]], and [[Enumerator Training|training]]. The [[Impact Evaluation Team|research team]] should keep the following points in mind when creating a timeline for '''fieldwork''' (field data collection):
 
The [[Impact Evaluation Team|research team]] should keep the following points in mind when creating a timeline for '''fieldwork'''(field data collection):
* '''Timing of data collection.''' Make sure that data collection happens during the relevant time of the year. For example, data on seasonal farm yields in a region should be collected after the harvesting season.  
* '''Timing of data collection.''' Make sure that data collection happens during the relevant time of the year. For example, data on seasonal farm yields in a region should be collected after the harvesting season.  
* '''Plan backwards.''' It often helps to plan backwards from the end date of the survey, and decide the start date based on this.  However, include some '''buffer''' (extra time) in the timeline for delays that might come up in the data collection process. For example, if a lot of the respondents are not available initially, the interviewers may have to visit them on another day.
* '''Plan backwards.''' It often helps to plan backwards from the end date of the survey, and decide the start date based on this.  However, include some extra '''buffer''' time in the timeline for delays that might come up in the data collection process. For example, if a lot of the respondents are not available initially, the interviewers may have to visit them on another day.
* '''Keep aside more time for panel surveys.''' Always allow more time for surveys that involve '''panel data''', that is, when the same data is collected several times from the same person or area. This is because tracking participants from earlier survey rounds can take some time. An example of such a case is study that seeks to assess the impact of a micro-finance program on farm yields every 6 months for 3 years in a given area.
* '''Keep aside more time for panel surveys.''' Always allow more time for surveys that involve '''panel data''', that is, when the same data is collected several times from the same person or area. This is because tracking participants from earlier survey rounds can take some time. An example of such a case is study that seeks to assess the impact of a micro-finance program on farm yields every 6 months for 3 years in a given area.


== IRB Approvals ==
'''Note:''' The table below provides a sample timeline for each of the steps involved in [[Primary Data Collection|primary data collection]].
An '''institutional review board (IRB)''' is a an organisation that reviews, approves, disapproves, or recommends changes in surveys that involve human '''subjects''' (target population). [[IRB Approval|IRB approvals]] are important to [[Protecting Human Research Subjects|protect the rights of human subjects]].
{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
!style="width:350px; text-align:center;"| Task
!style="width:350px; text-align:center;"| Estimated Time
|-
|style="text-align:center;"| [[Questionnaire Design|Develop questionnaire]]
|style="text-align:center;"| 2 months
|-
|style="text-align:center;"| [[Survey Pilot|Pilot questionnaire]]
|style="text-align:center;"| 1 month
|-
|style="text-align:center;"| [[Sampling & Power Calculations|Pilot recruitment strategy]]
|style="text-align:center;"| 3-4 months
|-
|style="text-align:center;"| [[Survey Budget|Create budget]] and [[Field Management|plan fieldwork]]
|style="text-align:center;"| 2 weeks
|-
|style="text-align:center;"| [[Procuring a Survey Firm|Contract survey firm]]
|style="text-align:center;"| 3 months
|-
|style="text-align:center;"| [[Data Quality Assurance Plan|Data quality assurance plan]]
|style="text-align:center;"| 2 weeks
|-
|style="text-align:center;"| [[IRB Approval|Obtain ethics approvals]]
|style="text-align:center;"| 1 week-3 months
|-
|style="text-align:center;"| [[Enumerator Training|Enumerator training]]
|style="text-align:center;"| 7-14 days
|}


The [[Impact Evaluation Team|research team]] must seek approval at each of the following stages:
== Preparing the Survey Instrument ==
There are several steps involved in preparing and finalizing a '''survey instrument''' (or questionnaire):
* The [[Impact Evaluation Team|impact evaluation team]] (or research team) should [[Timeline of Survey Pilot|allocate sufficient time]] for each of the steps.
* [[Questionnaire Design|Design]] the questionnaire based on the context. If a similar data collection process was conducted in the same region, the research team should check if they can adapt the questionnaire used in that study.
* [[Survey Pilot|Pilot]] the '''instrument''' to receive feedback on aspects like the wording of questions.
* Use this feedback to [[Checklist:_Content-focused_Pilot|improve the content]] of the questionnaire.
 
== Applying for IRB Approvals ==
An '''institutional review board (IRB)''' is a an organisation that reviews, approves, disapproves, or recommends changes in surveys that involve human '''subjects''' (target population). [[IRB Approval|IRB approvals]] are important to [[Protecting Human Research Subjects|protect the rights of human subjects]]. The [[Impact Evaluation Team|research team]] must seek approval at each of the following stages:
* '''Initial approval.''' Seek approval at the beginning of the study, before any research activity involving human participants begins.  
* '''Initial approval.''' Seek approval at the beginning of the study, before any research activity involving human participants begins.  
* '''Amendment approval.''' Seek approval before changing any element of the study including [[Questionnaire Design|design]], [[Survey Protocols|protocols]], or even [[Informed Consent#Elements of Informed Consent|informed consent norms]].  
* '''Amendment approval.''' Seek approval before changing any element of the study including [[Questionnaire Design|design]], [[Survey Protocols|protocols]], or even [[Informed Consent#Elements of Informed Consent|informed consent norms]].  
* '''Continuing approval.''' Seek continuing approval every year, even if no element of the survey has changed. This is because risks to participants may evolve through time. Therefore, the research team must report on the progress of the study which received the '''initial''' or '''amendment approval'''.
* '''Continuing approval.''' Seek continuing approval every year, even if no element of the survey has changed. This is because risks to participants may evolve through time. Therefore, the research team must report on the progress of the study which received the '''initial''' or '''amendment approval'''.


== Procurement ==
== Protocols ==
Discussing and agreeing on a set of [[Survey Protocols|survey protocols]] is just as important as finalizing the survey '''instrument'''. These guidelines describe the responsibilities of each member of the survey process - [[Impact Evaluation Team|impact evaluation team]], [[Survey Firm|survey firm]], '''interviewers'''. Examples of some of the things that '''protocols''' deal with are criteria for respondent selection, criteria for [[Sampling|sampling]], and [[Monitoring Data Quality|data quality checks]].  All protocols should be [[Piloting Survey Protocols|piloted]], and then clearly written out for use during [[Enumerator Training|enumerator training]].
=== Procurement ===
[[Procuring a Survey Firm|Procuring a survey firm]] involves drafting a [[Survey Firm TOR|terms of reference (TOR)]], and preparing a [[Survey Budget|budget]] for each component of the survey process. The [[Impact Evaluation Team|research team]] must carefully prepare these procurement-related documents.
[[Procuring a Survey Firm|Procuring a survey firm]] involves drafting a [[Survey Firm TOR|terms of reference (TOR)]], and preparing a [[Survey Budget|budget]] for each component of the survey process. The [[Impact Evaluation Team|research team]] must carefully prepare these procurement-related documents.
* '''Survey firm TOR.''' The [[Survey Firm TOR|survey firm terms of reference (TOR)]] defines the structure of the project and breaks down the responsibilities of all [[Survey Pilot Participants|participants]] in the data collection process. The '''TOR''' lists the scope of work and '''deliverables''' (expected outcomes), and allows the research team to monitor the performance of the [[Survey Firm|survey firm]] once the data collection starts.
* '''Survey firm TOR.''' The [[Survey Firm TOR|survey firm terms of reference (TOR)]] defines the structure of the project and breaks down the responsibilities of all [[Survey Pilot Participants|participants]] in the data collection process. The '''TOR''' lists the scope of work and '''deliverables''' (expected outcomes), and allows the research team to monitor the performance of the [[Survey Firm|survey firm]] once the data collection starts.
* '''Survey budget/proposal.''' The survey firm that is hired to conduct the '''field survey''' should submit a [[Survey Budget|budget]] that allocates funds to cover salaries, equipment costs, incentives for respondents, and other administrative costs. This should also include country-specific costs, such as taxes.
* '''Survey budget/proposal.''' The survey firm that is hired to conduct the '''field survey''' should submit a [[Survey Budget|budget]] that allocates funds to cover salaries, equipment costs, incentives for respondents, and other administrative costs. This should also include country-specific costs, such as taxes.
== Survey Instrument ==
=== Hiring ===
There are several steps involved in preparing and finalizing a '''survey instrument''' (or questionnaire). The [[Impact Evaluation Team|impact evaluation team]] (or research team) should [[Timeline of a Survey Pilot|allocate sufficient time]] for each of the steps.
The research team should hire experience enumerators and supervisors who have prior experience in field data collection. The number of '''field teams''' (people directly involved in collecting field data) that are hired in a survey depend on factors like average duration of the interview, [[Sample Size and Power Calculations|sample size]], and number of rounds of the survey. For example, if a survey requires a '''follow-up''' survey after the '''baseline''' (or the first round), the research team should hire more field teams.
* '''Draft and pilot.''' [[Questionnaire Design|Design]] the questionnaire based on the context. If a similar data collection process was conducted in the same region, the research team should check if they can adapt the questionnaire used in that study. [[Survey Pilot|Pilot]] the '''instrument''' to receive feedback on aspects like the wording of questions. Use this feedback to [[Piloting Survey Content|improve the content]] of the questionnaire.
* '''Translate.''' Hire professional '''translators''' to [[Questionnaire Translation|translate]] the questionnaire. These translators must be fluent in all local languages of the study area, as well as the language of the research team. It is very important to ensure '''version control''', that is, keep track of different versions during the translation process.  
* '''Program.''' After the translations are finalized, [[Questionnaire Programming|program]] the questionnaire. Check the code and perform test rounds multiple times.
* '''Finalize.''' Conduct a [[Survey Pilot#Stages of a Survey Pilot|data-focused pilot]] to ensure that all questions appear in the correct order. Note down how long it takes on average to finish answering the survey. Finalize the '''instrument''' for [[Field Surveys#Survey Launch|launching]].
 
== Protocols ==
Discussing and agreeing on a set of [[Survey Protocols|survey protocols]] is just as important as finalizing the survey '''instrument'''. These '''protocols''' describe the responsibilities of each member of the survey process- [[Impact Evaluation Team|impact evaluation team]], [[Survey Firm|survey firm]], '''interviewers'''.


Examples include: respondent selection, criteria for dropping and replacing sampling units, guidance on respondent tracking (especially for follow-up surveys), and any other issues related to survey implementation.  
=== Team setup and roles ===
As the the number of field teams and/or the size of each field team increases, it becomes more challenging to monitor the '''field implementation''' (data collection process). Based on '''best practices''', each field team should have 4-6 enumerators and 1 supervisor. There should also be a '''scrutinizer''' in the case of a [[Pen-and-Paper Personal Interviews (PAPI)|pen-and-paper interview (PAPI)]], or a '''data entry clerk''' in case of a [[Computer-Assisted Field Entry (CAFE)|computer-assisted field entry (CAFE)]] survey.


All protocols should be drafted, piloted, and then clearly written out and included in the [[Enumerator Training|Enumerator Manual]].
The roles of various participants of the data collection process as are follows:
=== Develop survey budget ===
* '''Enumerator.''' Conduct household interviews
* '''Supervisor.''' Manage teams of enumerators, check surveys for '''completeness''' (no missing data), keep a record of completed interviews, and ensure smooth communication between different teams. Can also perform [[Back Checks|back checks]].
* '''Scrutinizer.''' Read through the questionnaire for a '''pen-and-paper interview (PAPI)''' in detail to identify errors or inconsistencies. After approving the filled-out questionnaires, share them with the '''data entry team.'''
* '''Back checker.''' - Administers [[Back Checks|back check surveys]].
* '''Research analyst.''' In case of ([[Computer-Assisted Personal Interviews (CAPI)|CAPI]]), they develop a '''template''' (format) for collecting data electronically. Export and review incoming data every day, and ensure that the data matches '''field logs''', that is, the records of respondents who have already been interviewed .
* '''Data entry coordinator.''' Develop forms for electronically entering data, and coordinate the work of the entire data entry team. Export and review the incoming data everyday.
* '''Data entry clerk.''' Enter data during a '''computer-assisted field entry (CAFE) survey into the electronic form developed by the '''data entry coordinator'''.
* '''Field manager.''' Plan and oversee the entire process of field data collection and manage all '''field teams'''. Draft the logistics and budget for the field work, and act as primary '''liaison''' (point-of-contact) with the impact evaluation team.
* '''Field coordinator.''' The [[Impact Evaluation Team#Field Coordinators (FCs)|field coordinators (FCs)]] should also work closely with the [[Survey Firm|survey firm]] and government agencies (where relevant) to decide important aspects such as the team setup and hiring criteria. The research team must include these in the [[Survey Firm TOR|survey firm terms of reference (TOR)]].


Field work should be planned by the [[Field Coordinator]] together with the [[Survey Firm Procurement|Survey Firm]] or implementing partner. Important aspects of the field work, such as the overall timeline and team composition, should be specified in the [[Terms of Reference]] for the survey.
=== Communication ===
Clear communication protocols are important to ensure that everyone involved in the data collection can communicate and discuss any concerns they may face. It is therefore important to draft protocols to address the following aspects:
* '''First point of contact.''' Make sure enumerators and supervisors know who is their first point of contact on the team.
* '''Contact details of respondents.''' Make one or two people responsible for providing these details to enumerators everyday.
* '''Transport credit for enumerators.''' Ensure that enumerators have the funds available for traveling in order to conduct in-person interviews.
* '''Sharing collected data.''' Specify the frequency for sharing data- daily, alternate days, or weekly. Also specify exactly what data the enumerators need to share.
* '''Method of sharing data.''' Specify how enumerators share the data they collect.
* '''Monitoring enumerators.''' Specify how a supervisor can monitor enumerator performance, and parameters they will use to judge performance. For example, number of interviews completed in a day.
* '''Enumerator feedback.''' Specify how to convey issues identified during data quality checks to the enumerators.
* '''Escalating issues.''' Specify whom the enumerator should contact if a respondent requests an escalation or requests clarity on a question. For example, it helps to use an '''instant messaging (IM)''' application to communicate with enumerators, or have frequent in-person meetings with them, if feasible.
* '''Backup.''' Always have a plan B (and C) in place in case of problems like transport issues, or if the respondent is not at home at the time of visit.


==== Composition and Number of Field Teams ====  
=== Special situations ===
The composition and the number of field teams in a survey depend on a lot of variables. Time constraints are typically the major determinant of the number of field teams that are required. Other key determinants include:
Sometimes it is possible that the respondent is either not available at the time of the enumerator's visit, or someone other than the respondent is at home. It is important to provide clear protocols for these situations. In cases where some respondents are not available, '''tracking sheets''' become very important. These are forms that the enumerator can fill. A tracking sheet can be used to track:
*'''Respondent details.''' Names and addresses of all respondents in the sample.
*'''Status of survey.''' Whether the survey could be completed or not.
*'''Number of visits.''' If the respondent was not at home on the first visit.


* Expected interview duration (and closely related, expected number of interviews per enumerator per day)
In cases where someone other than the respondent is at home, the enumerator can do the following:
* Number of interviews per cluster (to determine number of interviewers per cluster)
* '''Take an appointment.''' The enumerator can ask the person who was at the specified address to indicate when the respondents might be available. This can help the supervisor judge if another visit is feasible or not.
* Cluster locations (how far do enumerators need to travel between clusters? is it possible to cover multiple clusters in 1 day?)
* '''Replace respondent.''' It is a good idea to have a pool from which new respondents can be picked to take the survey.
* Transportation plans (how much of the day will enumerators spend in transit vs. completing interviews?)


Note that it is recommended to plan for a team to spend more than 1 day in a cluster, to allow for re-visits and follow-ups. 1 day per cluster requires very thorough advance scheduling of interviews, and may require a transport-heavy follow-up period at the end of the survey.
=== Data quality and security ===
[[Monitoring Data Quality|Monitoring data quality]] is one of the most important parts of data collection. Poor quality data can at best reduce the effectiveness of a policy intervention, and at worst require a repeat of the entire data collection process. Therefore the research team must prepare clear guidelines for the following:
*'''Type of data checks.''' Conduct regular [[Back Checks|back checks]] and [[Monitoring Data Quality#High Frequency Checks|high frequency checks]].
*'''Frequency of data checks.''' Specify how often the supervisor should conduct data checks.
*'''Feedback method.''' Specify method for communicating feedback to the enumerators after a data check. Decide on this before any data collection starts.


Quality control becomes harder as the size of each field team increases. Recommendation is for each field team to be composed of 4-6 enumerators and 1 supervisor (along with a scrutinizer if [[Pen-and-Paper Personal Interviews (PAPI)|PAPI]] and data entry clerk if [[Computer-Assisted Field Entry (CAFE)|CAFE]].
It is equally important to specify clear protocols for [[Data Security|data security]] to ensure no data is lost and no [[Personally Identifiable Information (PII)|personally identifiable information (PII)]] is made public. This includes [[Encryption|encrypting]] the survey form and providing guidelines to enumerators about how to share data. Create a confidentiality agreement that each enumerator signs to ensure that personal details of respondents do not get used for any other purposes.
==== Field Team Roles & Responsibilities ====


The roles of various team members during data collection as are follows:
== Plan Enumerator Training ==
The purpose of [[Enumerator Training|enumerator training]] is to ensure that all participants of the data collection process should be familiar with the [[Survey Protocols]] and the [[Questionnaire Design|content]] of the '''instrument'''. Keep the following things in mind.
* '''Duration and structure.''' The duration and structure of the training will depend on the complexity and length of the survey.
* '''Plan in advance.''' Plan the training well in advance, and make sure the [[Enumerator Training#Enumerator Manual|enumerator manual]] is complete and up-to-date.
* '''Enumerator selection.''' Always train more enumerators than will be required for the field work. At the end of the training select the best enumerators for the data collection.


*Enumerator - Conduct household interviews
== Related Pages ==
*Supervisor - Manage teams of enumerators, introduce survey teams, check all surveys for completeness, keep log of interviews completed, could do back check surveys
*Scrutinizer ([[Pen-and-Paper Personal Interviews (PAPI)|PAPI]]) - read through questionnaire in detail to catch errors or inconsistencies that need to be resolved. After approval, send on to data entry team.
*Back-checker - Administer back-check surveys
*Research Analyst ([[Computer-Assisted Personal Interviews (CAPI)|CAPI]]) - develop electronic data collection template, export and review data on daily basis, ensure data matches field logbooks.
*Data entry coordinator ([[Pen-and-Paper Personal Interviews (PAPI)|PAPI]]) - develop data entry forms, coordinate data entry team, export and review data on daily basis.
*Data entry clerk ([[Pen-and-Paper Personal Interviews (PAPI)|PAPI]]) - enter data from paper surveys into the electronic template
*Field Manager - plan and oversee field work, manage all field teams, handle logistics and budget, primary liaison with field coordinator and research team.
 
== Enumerator Training ==
 
== Back to Parent ==
[[Special:WhatLinksHere/Preparing_for_Field_Data_Collection|Click here for pages that link to this topic]].
[[Special:WhatLinksHere/Preparing_for_Field_Data_Collection|Click here for pages that link to this topic]].


== Additional Resources ==
== Additional Resources ==
* Brief from Oxfam: [http://policy-practice.oxfam.org.uk/publications/planning-survey-research-578973 Planning Survey Research]
* DIME Analytics (World Bank), [https://osf.io/u5evr Engaging with Data Collectors]
*DIME's [http://web.worldbank.org/archive/website01542/WEB/IMAGES/SURVEY.PDF Planning for, Preparing & Monitoring Household Surveys]
* DIME (World Bank), [https://osf.io/357uv Design and Pilot a Survey]
*DIME Analytics’ guidelines on [https://github.com/worldbank/DIME-Resources/blob/master/survey-preparing.pdf preparing for data collection]
* DIME (World Bank), [https://osf.io/63c8t Working with Survey Firms]
* Guidelines and tools for [http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTHEALTHNUTRITIONANDPOPULATION/EXTHSD/EXTIMPEVALTK/0,,contentMDK:23262154~pagePK:64168427~piPK:64168435~theSitePK:8811876,00.html Preparing for Data Collection] from the World Bank's Results Based Financing Impact Evaluation Toolkit
* Oxfam, [http://policy-practice.oxfam.org.uk/publications/planning-survey-research-578973 Planning survey research]
 
* Sandra V. Rozo (World Bank), [https://blogs.worldbank.org/impactevaluations/tips-collecting-surveys-hard-reach-populations?CID=WBW_AL_BlogNotification_EN_EXT Tips for surveying hard-to-reach populations]
 
[[Category: Primary Data Collection]]
[[Category: Primary Data Collection]]

Latest revision as of 13:48, 17 August 2023

Preparing for field data collection involves preparing clear protocols and guidelines for each component of primary data collection using field surveys. This page looks at the best practices and protocols that the research team must follow before starting with the process of primary data collection.

Read First

  • Primary data collection details the workflow from start to finish involved in gathering data through surveys, interviews, or experiments.
  • The research team can either conduct the survey directly, or indirectly through a survey firm.
  • Spending more time on the preparation leads to better quality data.
  • Plan for field data collection at least 6 months in advance of the survey launch.
  • Field data can be collected using Open Data Kit (ODK)-based tools like CAPI and CAFE.
  • With the availability of specialized survey firms, and standardized field management practices, it is important for researchers to follow best practices to collect field data.

Preparing the Survey Timeline

Preparing the timeline for a field survey involves allocating sufficient time for each stage of the survey process - instrument design, pilot, obtaining approval, procurement, drafting protocols, and training. The research team should keep the following points in mind when creating a timeline for fieldwork (field data collection):

  • Timing of data collection. Make sure that data collection happens during the relevant time of the year. For example, data on seasonal farm yields in a region should be collected after the harvesting season.
  • Plan backwards. It often helps to plan backwards from the end date of the survey, and decide the start date based on this. However, include some extra buffer time in the timeline for delays that might come up in the data collection process. For example, if a lot of the respondents are not available initially, the interviewers may have to visit them on another day.
  • Keep aside more time for panel surveys. Always allow more time for surveys that involve panel data, that is, when the same data is collected several times from the same person or area. This is because tracking participants from earlier survey rounds can take some time. An example of such a case is study that seeks to assess the impact of a micro-finance program on farm yields every 6 months for 3 years in a given area.

Note: The table below provides a sample timeline for each of the steps involved in primary data collection.

Task Estimated Time
Develop questionnaire 2 months
Pilot questionnaire 1 month
Pilot recruitment strategy 3-4 months
Create budget and plan fieldwork 2 weeks
Contract survey firm 3 months
Data quality assurance plan 2 weeks
Obtain ethics approvals 1 week-3 months
Enumerator training 7-14 days

Preparing the Survey Instrument

There are several steps involved in preparing and finalizing a survey instrument (or questionnaire):

  • The impact evaluation team (or research team) should allocate sufficient time for each of the steps.
  • Design the questionnaire based on the context. If a similar data collection process was conducted in the same region, the research team should check if they can adapt the questionnaire used in that study.
  • Pilot the instrument to receive feedback on aspects like the wording of questions.
  • Use this feedback to improve the content of the questionnaire.

Applying for IRB Approvals

An institutional review board (IRB) is a an organisation that reviews, approves, disapproves, or recommends changes in surveys that involve human subjects (target population). IRB approvals are important to protect the rights of human subjects. The research team must seek approval at each of the following stages:

  • Initial approval. Seek approval at the beginning of the study, before any research activity involving human participants begins.
  • Amendment approval. Seek approval before changing any element of the study including design, protocols, or even informed consent norms.
  • Continuing approval. Seek continuing approval every year, even if no element of the survey has changed. This is because risks to participants may evolve through time. Therefore, the research team must report on the progress of the study which received the initial or amendment approval.

Protocols

Discussing and agreeing on a set of survey protocols is just as important as finalizing the survey instrument. These guidelines describe the responsibilities of each member of the survey process - impact evaluation team, survey firm, interviewers. Examples of some of the things that protocols deal with are criteria for respondent selection, criteria for sampling, and data quality checks. All protocols should be piloted, and then clearly written out for use during enumerator training.

Procurement

Procuring a survey firm involves drafting a terms of reference (TOR), and preparing a budget for each component of the survey process. The research team must carefully prepare these procurement-related documents.

  • Survey firm TOR. The survey firm terms of reference (TOR) defines the structure of the project and breaks down the responsibilities of all participants in the data collection process. The TOR lists the scope of work and deliverables (expected outcomes), and allows the research team to monitor the performance of the survey firm once the data collection starts.
  • Survey budget/proposal. The survey firm that is hired to conduct the field survey should submit a budget that allocates funds to cover salaries, equipment costs, incentives for respondents, and other administrative costs. This should also include country-specific costs, such as taxes.

Hiring

The research team should hire experience enumerators and supervisors who have prior experience in field data collection. The number of field teams (people directly involved in collecting field data) that are hired in a survey depend on factors like average duration of the interview, sample size, and number of rounds of the survey. For example, if a survey requires a follow-up survey after the baseline (or the first round), the research team should hire more field teams.

Team setup and roles

As the the number of field teams and/or the size of each field team increases, it becomes more challenging to monitor the field implementation (data collection process). Based on best practices, each field team should have 4-6 enumerators and 1 supervisor. There should also be a scrutinizer in the case of a pen-and-paper interview (PAPI), or a data entry clerk in case of a computer-assisted field entry (CAFE) survey.

The roles of various participants of the data collection process as are follows:

  • Enumerator. Conduct household interviews
  • Supervisor. Manage teams of enumerators, check surveys for completeness (no missing data), keep a record of completed interviews, and ensure smooth communication between different teams. Can also perform back checks.
  • Scrutinizer. Read through the questionnaire for a pen-and-paper interview (PAPI) in detail to identify errors or inconsistencies. After approving the filled-out questionnaires, share them with the data entry team.
  • Back checker. - Administers back check surveys.
  • Research analyst. In case of (CAPI), they develop a template (format) for collecting data electronically. Export and review incoming data every day, and ensure that the data matches field logs, that is, the records of respondents who have already been interviewed .
  • Data entry coordinator. Develop forms for electronically entering data, and coordinate the work of the entire data entry team. Export and review the incoming data everyday.
  • Data entry clerk. Enter data during a computer-assisted field entry (CAFE) survey into the electronic form developed by the data entry coordinator.
  • Field manager. Plan and oversee the entire process of field data collection and manage all field teams. Draft the logistics and budget for the field work, and act as primary liaison (point-of-contact) with the impact evaluation team.
  • Field coordinator. The field coordinators (FCs) should also work closely with the survey firm and government agencies (where relevant) to decide important aspects such as the team setup and hiring criteria. The research team must include these in the survey firm terms of reference (TOR).

Communication

Clear communication protocols are important to ensure that everyone involved in the data collection can communicate and discuss any concerns they may face. It is therefore important to draft protocols to address the following aspects:

  • First point of contact. Make sure enumerators and supervisors know who is their first point of contact on the team.
  • Contact details of respondents. Make one or two people responsible for providing these details to enumerators everyday.
  • Transport credit for enumerators. Ensure that enumerators have the funds available for traveling in order to conduct in-person interviews.
  • Sharing collected data. Specify the frequency for sharing data- daily, alternate days, or weekly. Also specify exactly what data the enumerators need to share.
  • Method of sharing data. Specify how enumerators share the data they collect.
  • Monitoring enumerators. Specify how a supervisor can monitor enumerator performance, and parameters they will use to judge performance. For example, number of interviews completed in a day.
  • Enumerator feedback. Specify how to convey issues identified during data quality checks to the enumerators.
  • Escalating issues. Specify whom the enumerator should contact if a respondent requests an escalation or requests clarity on a question. For example, it helps to use an instant messaging (IM) application to communicate with enumerators, or have frequent in-person meetings with them, if feasible.
  • Backup. Always have a plan B (and C) in place in case of problems like transport issues, or if the respondent is not at home at the time of visit.

Special situations

Sometimes it is possible that the respondent is either not available at the time of the enumerator's visit, or someone other than the respondent is at home. It is important to provide clear protocols for these situations. In cases where some respondents are not available, tracking sheets become very important. These are forms that the enumerator can fill. A tracking sheet can be used to track:

  • Respondent details. Names and addresses of all respondents in the sample.
  • Status of survey. Whether the survey could be completed or not.
  • Number of visits. If the respondent was not at home on the first visit.

In cases where someone other than the respondent is at home, the enumerator can do the following:

  • Take an appointment. The enumerator can ask the person who was at the specified address to indicate when the respondents might be available. This can help the supervisor judge if another visit is feasible or not.
  • Replace respondent. It is a good idea to have a pool from which new respondents can be picked to take the survey.

Data quality and security

Monitoring data quality is one of the most important parts of data collection. Poor quality data can at best reduce the effectiveness of a policy intervention, and at worst require a repeat of the entire data collection process. Therefore the research team must prepare clear guidelines for the following:

  • Type of data checks. Conduct regular back checks and high frequency checks.
  • Frequency of data checks. Specify how often the supervisor should conduct data checks.
  • Feedback method. Specify method for communicating feedback to the enumerators after a data check. Decide on this before any data collection starts.

It is equally important to specify clear protocols for data security to ensure no data is lost and no personally identifiable information (PII) is made public. This includes encrypting the survey form and providing guidelines to enumerators about how to share data. Create a confidentiality agreement that each enumerator signs to ensure that personal details of respondents do not get used for any other purposes.

Plan Enumerator Training

The purpose of enumerator training is to ensure that all participants of the data collection process should be familiar with the Survey Protocols and the content of the instrument. Keep the following things in mind.

  • Duration and structure. The duration and structure of the training will depend on the complexity and length of the survey.
  • Plan in advance. Plan the training well in advance, and make sure the enumerator manual is complete and up-to-date.
  • Enumerator selection. Always train more enumerators than will be required for the field work. At the end of the training select the best enumerators for the data collection.

Related Pages

Click here for pages that link to this topic.

Additional Resources