Dimewiki

Difference between revisions of "Preparing for Remote Data Collection"

Jump to: navigation, search
 
(38 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''Preparing for remote data collection''' involves preparing clear protocols and guidelines for each component of collecting data '''remotely''' (that is, from a location that is different from that of the respondent). The main components of remote data collection include '''timelines''', '''approvals''', '''instrument design''', '''coding surveys''', and '''enumerator training'''. Each of these components involves additional steps compared to [[Preparing for Field Data Collection|preparing for field data collection]]. Therefore it is important to establish clear channels for regular communication among everyone involved in the data collection- the research team, government agencies, survey firm, and enumerators.
'''Preparing for remote data collection''' involves preparing clear protocols and guidelines for each component of collecting data '''remotely''' (that is, from a location that is different from that of the respondent). The main components of remote data collection include '''timelines''', '''approvals''', '''instrument design''', '''coding surveys''', and '''enumerator training'''. Each of these components involves additional steps compared to [[Preparing for Field Data Collection|preparing for field data collection]]. Therefore it is important to establish clear channels for regular communication among everyone involved in the data collection- the research team, government agencies, survey firm, and enumerators.
== Read First ==  
== Read First ==  
* '''Remote surveys''' are useful for collecting data in cases where [[Field Surveys|field surveys]] are either to expensive, or not possible due to lack of in-person access to respondents.
* [[Remote Surveys|Remote surveys]] are useful for collecting data in cases where [[Field Surveys|field surveys]] are either too expensive, or not possible due to lack of in-person access to respondents.
* Among the different kinds of remote surveys, researchers consider [[Remote Surveys#Phone Surveys|phone surveys]] more efficient because they retain some degree of human interaction.  
* There are 3 broad kinds of remote surveys - [[Remote Surveys#Phone Surveys|phone surveys]], [[Remote Surveys#Web Surveys|web surveys]], and [[Remote Surveys#Recorded Surveys|recorded surveys]].
* We will look at the steps involved in preparing for phone surveys.
* Researchers consider '''phone surveys''' more efficient because they retain some degree of human interaction.  
* DIME Analytics has also compiled a [https://paper.dropbox.com/doc/A-Guide-to-WFH-Data-Collection-AwiZ_7GxbzaF8aXCRri8qCAg-DVhtGiokwboliGBqmjYfS#:h2=Update-data-collection-protoco guide on transitioning to phone surveys] when field data collection is not possible due to pandemics or natural disasters.
* We will therefore look at the steps involved in preparing for phone surveys.
* Most guidelines for this process can also be extended to cover other kinds of remote surveys.
* [https://www.worldbank.org/en/research/dime/data-and-analytics DIME Analytics] has also compiled a [https://paper.dropbox.com/doc/A-Guide-to-WFH-Data-Collection-AwiZ_7GxbzaF8aXCRri8qCAg-DVhtGiokwboliGBqmjYfS#:h2=Update-data-collection-protoco guide on transitioning to phone surveys] when field data collection is not possible due to pandemics or natural disasters.


== Updates to Survey Timeline ==  
== Survey Timeline ==  
The first step of the transition should be to update the survey timeline. When you transition to a phone survey, each step leading up to, and including, the data collection will take additional time. You must make sure you account for this time, and allocate a reasonable window for each step.
Preparing the timeline for a survey involves allocating sufficient time for each stage of the survey process - [[Questionnaire Design|design]], [[Survey Pilot|pilot]], [[Questionnaire Programming|programming]], and [[Enumerator Training|enumerator training]]. All of these steps require additional time, since there are additional protocols that need to be followed at each stage. The research team must account for this time, and allocate a reasonable window for each step. For instance, '''pre-piloting''' a phone survey might require 3 weeks, instead of 1-2 weeks for a field survey.
== Additional IRB Approvals ==
If the project has any existing IRB approval(s), you will need to submit an amendment to each IRB, and receive approval for each of the following changes:
*Requirement of oral consent
*Adding audio audits to the survey
*Changes in the survey instrument
*Addition of any team members handling PII data
*Changes to incentives provided to respondents
While most IRBs are cognizant of the current crisis and are fast-tracking amendments, you should allocate enough time for the IRB amendment to come through before you can (re)start data collection.


If your project does not have an IRB approval yet, we recommend that you apply to an IRB immediately.
== IRB Approvals ==
In addition to the components of a field study, the research team must also seek [[IRB Approval|IRB approval]] for additional components of a phone survey.
*'''Oral consent.''' The study must make provisions for obtaining oral [[Informed Consent|consent]] from the respondent at the beginning of the phone interview.
*'''Audio audits.''' [[Random Audio Audits|Audio audits]] help to monitor and enrich the quality of data collected through a phone survey. The research team must clearly lay out the details of the process, and inform respondents that the interview is being recorded to improve data quality.
*'''Instrument design.''' This involves seeking approvals for the content of the questionnaire, as well as aspects like the average duration of the survey.
*'''Norms for handling PII.''' Switching to phone surveys will require hiring additional people and strengthening the norms to handle [[Personally Identifiable Information (PII)|personally identifiable information (PII)]].
*'''Incentives to respondents.''' The research team must set up a different system for giving incentives to respondents in a phone survey.
'''Note:''' The research must make changes to these components and seek IRB approval for all of these changes even while '''transitioning''' from an in-person survey to a phone survey.


== Updates to Procurement-related Documents ==  
== Procurement ==  
In cases where the survey firm has been procured by the World Bank, your team will have to make certain changes to procurement-related documents. This includes updates to the:
[[Procuring a Survey Firm|Procuring a survey firm]] involves drafting a [[Survey Firm TOR|terms of reference (TOR)]], and preparing a [[Survey Budget|budget]] for each component of the survey process. These procurement-related documents should also reflect the fact that the data collection will be conducted over the phone, and not in-person.
#Survey firm TOR <br/>Make sure you update the survey terms of reference (TOR) and the survey firm’s technical proposal to include all relevant changes in protocols viz. mode of data collection, change in survey duration, change in timeline, and so on.
* '''Survey firm TOR.''' The '''terms of reference (TOR)''' to hire a [[Survey Firm|survey firm]] must clearly specify the guidelines for conducting phone surveys. These guidelines should cover issues like the mode of data collection, average survey duration, survey timeline, and so on.
#Survey budget / proposal <br/>The survey firm conducting the data collection should submit an updated budget accounting for the changes in costs as a result of the transition - including equipment, incentives, and survey costs.  
* '''Survey budget/proposal.''' The survey firm that is hired to conduct the phone survey should submit a budget that allocates funds to cover equipment costs, incentives, and other administrative costs . An example of costs that may arise in the case of phone surveys is the cost of setting up a call center for enumerators. Note that this cost will not exist if enumerators are working from home, for instance, during a pandemic.
In case of a transition from an in-person survey to a phone survey, the survey firm may need more financial resources. The research team will then have to change the contract to reflect this, and ask for fresh approval from the organization funding the study. In case the World Bank is funding the study, the research team will have to obtain approval from '''Corporate Procurement'''.


In case  additional funds need to be disbursed by the survey firm, the contract will have to be modified. This typically requires approval from the organization funding the survey firm, or from Corporate Procurement in the case of World Bank funding.
== Survey Instrument ==
For a phone survey, keep the following in mind while preparing the '''instrument''' (questionnaire):
* '''Keep the survey short.''' General guidelines and best practices suggest that the average duration of phone interviews should not be longer than 20 minutes. However, reducing the length of a survey takes considerable amount of time. Make sure to account for this in the timeline update. You can try the following to reduce survey length:
*# '''Drop modules which cannot be asked over phone.''' For example, GPS coordinates
*# '''Identify the most essential outcomes of interest.''' Keep questions related to these outcomes
*# '''Ask about aggregates instead of individual items.''' For example, there might be 25 different categories in survey that aims to assess consumption patterns . These can be combined to just a few categories for the purpose of a phone survey like durable items, non-durable, and food items. However, in such cases it is important to define what items fall under which category, and share this information with the respondent.
*# '''Break the survey into 2 or 3 shorter surveys.''' It is acceptable to do so if other methods of reducing survey length do not work. However, the research team should remember to account for '''differential attrition''' which arises when some respondents complete only a few rounds of the survey.
* '''Add identifiers at the start of the survey.''' This helps to ensure that the enumerator is interviewing the intended respondent.
* '''Collect as many telephone numbers as possible.''' This is particularly helpful in case there is a need for a follow-up survey.
* '''Keep the questions as concise and clear as possible.''' Add hints and definitions throughout.
* '''Code and repeatedly test the updated survey.'''


== Update the Survey Instrument ==
== Protocols ==
You may have to radically overhaul the survey instrument, in order to meet the following requirements:
Collecting data using surveys involves clear protocols and considerations that all [[Survey Pilot Participants|participants]] in the survey process must keep in mind. In addition to the [[Survey Protocols|protocols for field surveys]], the guidelines for phone surveys will need to address the following concerns:  
*Keep the survey short.
=== Team setup and hiring ===
It’s absolutely essential to keep the survey super short and the maximum length of a survey can not be over 40 minutes.
One of the most important steps in the data collection process is hiring, and laying out the team structure. The following are the protocols for these components are:
The average survey length should not be greater than 20 minutes.
* '''Hiring.''' The research team should hire enumerators and supervisors who have prior experience with data collection.  
   
* '''Team setup.''' Ensure that the set up of the survey team is such that there are no more than 5 enumerators reporting to 1 supervisor. The research team may consider hiring more supervisors if needed.
Note that reducing the length of a survey takes considerable amount of time. Make sure to account for this in the timeline update. You can try the following to reduce survey length:
   
Drop modules which can not be asked over phone (for example GPS coordinates) Identify the most essential outcomes of interest and keep questions related to these outcomes
*Ask about aggregates instead of individual items.
For example, an in-person consumption module which goes over 25 individual items should be cut down in a phone survey to as few categories as possible (such as durable items, food, etc). When aggregating it is essential to define what items fall under which category and this should be made clear to the respondent during the survey.
*Break the survey into 2 or 3 shorter surveys.
You can do this if key outcomes of interest can not be measured in one survey.  However, you must remember to account for differential attrition across these instruments as all respondents may not complete the entire set of surveys.
#Update the consent to a verbal consent.<br/>A short and easy to understand verbal consent should be included in the survey.  This should also inform the participants about the audio audits.
* Add identifiers at the start of the survey.
This helps to ensure the correct respondent is interviewed.
* Collect as many telephone numbers as possible.
This is particularly helpful in case you are planning a follow-up survey.
* Make the questions as concise and clear as possible.
Add hints and definitions throughout.
* Code and repeatedly test the updated survey for electronic data collection
* Program in audio audits as a data quality check
This adds an additional layer of check on data quality, which is necessary for such a drastic transition in the method of data collection.


== Update Data Collection Protocols ==
=== Procuring equipment. ===  
The majority of the protocols set up for in-person data collection will need to be revised. Make sure allocate enough time for these updates to be put in place. These changes are as essential as the survey instrument updates. You will have to either change the following protocols, or confirm that they still work in the new context:
The research team will have to procure additional equipment and make it available to the enumerators. This will include providing telephone or mobile devices, SIM cards (and money to recharge SIM cards), tablets or computers to enter data, and internet access for each part of the process.
=== Communication ===
Clear communication protocols are important to ensure that the phone survey goes on smoothly. These protocols should cover the following aspects :
* '''First point of contact.''' Make sure enumerators and supervisors know who is their '''first point of contact''' on the team.
* '''Contact details of respondents.''' Make one or two people responsible for providing these details to enumerators everyday.
* '''Phone credit for enumerators.''' Ensure that enumerators have the funds to call respondents in the sample.
* '''Sharing collected data.''' Specify the frequency for sharing data - daily, alternate days, or weekly. Also specify exactly what data the enumerators need to share.
* '''Method of sharing data.''' Specify how enumerators share the data they collect.
* '''Monitoring enumerators.''' Specify the people who will monitor enumerator performance, and '''parameters''' to judge performance. For example, number of phone calls completed per day.
* '''Enumerator feedback.''' Specify how to convey issues identified during data quality checks to the enumerators.
* '''Escalating issues.''' Specify whom the enumerator should contact if a respondent requests an escalation or requests clarity on a question. For example, it helps to communicate with enumerators using '''instant messaging (IM)''' platforms.
* '''Backup.''' Always have a plan B (and C) in place in case of problems like connectivity issues.


=== Set up of the team ===
Ensure that the set up of the survey team is such that there are not too many enumerators reporting to 1 supervisor (no more than 5). You should consider adding more supervisors if needed (remember, you’re already saving on budget with no travel to the field!)
=== Hiring ===
If you require to hire enumerators/supervisors, make sure to hire locally and find candidates who have previous enumeration experience.
=== Survey manuals ===
The survey manuals will have to go through a detailed update to ensure it accounts for all changes put in place. See here an example of survey protocols from Tavneet Suri’s Kenya UBI project.
=== Procuring equipment. ===
Additional equipment will ideally have to be procured and made available to the enumerators. This includes, telephone or mobile devices, SIM Cards, money to recharge SIM cards, tablets/computers to enter data on, and internet access for all parts of the process - from training to sharing data.
=== Communication within the team ===
Decide with your team about communication flows within the team. This should include details about :
* who is the first point of contact for each person in the team’ (for all enumerators, supervisors, and above)
* how enumerators will be provided with the sample they have to call on a daily basis
* who will ensure enumerators have enough phone credit to call respondents
* what data will the enumerators submit on a daily basis (or a regular previously decided upon frequency)
* how will enumerators share the data collected (submit forms at the end of each day)
* who will monitor enumerator performance (e.g. number of phone calls completed per day)
* how will issues identified in the data quality checks be conveyed to the enumerators
* whom the enumerator should reach out to in case of any questions or if a respondent requests an escalation. It often works well to create a WhatsApp group with all enumerators to stay connected
* Always have a plan B (and C) in place in case the decided upon plan fails for any reason (like connectivity issues)
* Tracking sheets
Tracking sheets become a more crucial component during phone surveys. These should be created as a digital survey form the enumerator fills in as we can not share printed sheets on a daily basis. A tracking sheet should be provided to each enumerator to be able to track
Respondent details
Updated contact numbers
Status of survey
Call back time in case respondent was unavailable
Number of attempts
Time of each attempt
       
=== Special situations ===
=== Special situations ===
Someone other than the respondent answers.
Sometimes it is possible that the respondent is either not available, or someone other than the respondent answers. It is important to provide clear protocols for these situations. In cases where some respondents are not available, '''tracking sheets''' become very important. These are digital survey forms that the enumerator can fill. A tracking sheet can be used to track:
Create a protocol which directs the enumerator on this front. The enumerator can try to get other contact numbers the respondent can be reached on, take an appointment at a time the respondent will be available,  if the enumerator can survey another member if the respondent is still not available, or to replace the respondent.
*'''Respondent details.''' Names and contact numbers of all respondents in the sample.
*Appointments.  
*'''Status of survey.''' Whether the respondent answered or not.
Very often respondents are not available to speak when an enumerator calls. A protocol can be put in place to allow respondents to provide times they are available to talk at and to ensure that the enumerator calls the respondent at the time of the appointment.  
*'''Call back time.''' In case the respondent was unavailable earlier.
* Replacement of respondent.
*'''Number of attempts.''' If the respondent did not call back.
Put in place a protocol for how many attempts an enumerator should make to reach a respondent before the respondent is replaced. It is also essential to think about how far apart these attempts should be from one another to be able to catch the respondent at a time they are free to speak. Example of a good protocol is 9 attempts with maximum of 3 attempts per day and each attempt spread at least 3 hours apart.
*'''Time of each attempt.''' To calculate average to contact all respondents in the sample.
 
In cases where someone other than the respondent answers the phone, the enumerator can do the following:
*'''Take an appointment.''' The enumerator can ask the person answering the call to specify when the respondents will be available to talk. The enumerator can then calls the same number again at the specified time.
*'''Replace respondent.''' Specify the number of attempts an enumerator should make to reach a respondent before replacing the respondent. Also, specify how long enumerators should wait before making another attempt. A good example of such a protocol specifies  making 9 attempts in total, with maximum of 3 attempts per day and at least 3 hours between each attempt.
 
== Data Quality and Security ==
[[Monitoring Data Quality|Monitoring data quality]] is one of the most important parts of data collection. Poor quality data can at best reduce the effectiveness of a policy intervention, and at worst require a repeat of the entire data collection process. Therefore the research team must prepare clear guidelines for the following:
*'''Type of data checks.''' Conduct regular [[Back Checks|back checks]] and [[Monitoring Data Quality#High Frequency Checks|high frequency checks]].
*'''Frequency of data checks.''' Specify how often the supervisor should conduct data checks.
*'''Feedback method.''' Specify method for communicating feedback to the enumerators after a data check. Decide on this before any data collection starts.
 
It is equally important to specify clear protocols for [[Data Security|data security]] to ensure no data is lost and no sensitive information is made public. This includes encrypting the survey form and providing guidelines to enumerators about how to share data. Create a confidentiality agreement that each enumerator signs (digitally) to ensure that contact numbers of respondents do not get used for any other purposes.
 
'''Note:''' These guidelines also apply in case there is a need to '''transition''' from an in-person survey to a remote survey. Make sure the survey manuals incorporate all these protocols, and share them with all members of the team.
 
== Enumerator Training ==
Given the extremely detailed protocols that need to be in place, it is also important to [[Enumerator Training|train enumerators]] rigorously to make them familiar with all protocols, questionnaire content, and ways to keep respondents engaged.
 
In cases where it is not possible to train enumerators in person, such as in the case of a pandemic, this process becomes even more challenging. In such a case, the training can be conducted online. With regards to the logistics of this process, keep in mind the following:
* '''Size of groups.''' If there are many enumerators break them down into smaller manageable groups (typically 5) to make it easier to conduct the training.
* '''Platform.''' Use a secure web-calling platform. Depending on bandwidth, use the video feature where possible to provide some sense of personal connection. If an online live training is not possible, consider alternatives such as recorded videos and phone calls.
* '''Connectivity issues.''' Record the training sessions and share them with enumerators in case they miss out on parts of the training due to connectivity issues. The recorded versions can also help enumerators practice all steps until they are confident.
* '''Training schedule.''' Allocate more time in the training schedule for these additional steps and the possibility of facing connection issues.
 
Similarly, for the content of the training, follow these guidelines:
* '''Training manual.''' Provide the team with a detailed [[Enumerator_Training#Enumerator_Manual|training manual]].
* '''Script.''' Provide enumerators with a detailed script for the entire survey instrument(s) so that they know how to ask the questions.
* '''Mock interviews.''' Create recordings of test interviews to use them for the training. '''Note''' that it is important to conduct more mock interviews during virtual training than you would in an in-person training
* '''Asking questions.''' Include content on how to ask questions during the interview.
* '''Doubts of respondents.''' Train the enumerators to clarify any doubts that respondents might have. These doubts can be regarding the project, the design, the purpose of the interview, or about a particular question.
* '''Engaging respondents.''' Train enumerators on how to keep respondents engaged during the interview. Some ways to do this include making the respondents feel comfortable, compensating them for their time, clarifying their doubts (if any), offering to get them in touch with the manager if they have any concerns, among others.


=== Data quality checks ===
== Related Pages ==
The data quality checks that will be run, the frequency of running the same, and how the results will be communicated to the teams should be decided on before any data collection starts.
[[Special:WhatLinksHere/Preparing_for_Remote_Data_Collection|Click here for pages that link to this topic]].
=== Ensuring data security ===
A data security protocol should be put in place to try and ensure no data is lost. This includes encrypting the survey form and providing guidelines to enumerators about how to share data. Create a confidentiality agreement that each enumerator signs (digitally) in an effort to ensure the respondent contact numbers do not get used for any other purposes.
== (Re)Train the enumerators ==
The survey team will have to be extensively trained on the protocols, the survey instrument, and ways to keep the team engaged. This is one of the most challenging tasks as there is no way to get the entire survey team together and conduct an in-person training. The training has to be done virtually, and you will need to be innovative with multiple alternative plans for training the enumerators.
Here are some guidelines to keep in mind when designing the training.
=== Logistics ===
* Have smaller groups of enumerators for the training.
If there are many enumerators break them down into smaller manageable groups (typically 5) to make it easier to conduct the training.
       
* Use a calling platform like Skype, Zoom, Webex. 
Depending on bandwidth, try to conduct the training using video to provide some sense of personal connection. If a live virtual training is not possible, consider alternatives such as recorded videos and phone calls.  


You can record the training and share with enumerators for future use or in case they missed parts of the training due to connectivity issues.  Also, be sure to allocate more time in the training schedule for these additional steps, or in case things go wrong.
== Additional Resources ==  
=== Content ===
* Akuffo et. al. (World Bank Group), [https://documents.worldbank.org/en/publication/documents-reports/documentdetail/643951622007959941/remote-technical-assistance-for-surveys-technical-guidance-note Remote Technical Assistance for Surveys : Technical Guidance Note]
* Provide the team with a detailed training manual (example here). You should have a detailed script for the entire survey instrument(s) so the enumerators know how to ask the questions.
* Chen et. al. (World Bank Group), [https://documents.worldbank.org/en/publication/documents-reports/documentdetail/457071612329086331/planning-and-implementing-household-surveys-under-covid-19 Planning and Implementing Household Surveys Under COVID-19]
* Create recordings of test interviews. You can then use these for the training.
* DIME Analytics (World Bank), [https://paper.dropbox.com/doc/A-Guide-to-WFH-Data-Collection--AxVTbsy~pSkZUlReHqqOVNbsAg-DVhtGiokwboliGBqmjYfS Guide on work-from-home (WFH) data collection]
* Conduct more mock interviews during virtual training than you would in an in-person training
* ideas42, [https://www.ideas42.org/blog/fieldwork-during-the-pandemic-four-behavioral-insights-for-conducting-remote-interviews/ Fieldwork During the Pandemic? Four Behavioral Insights for Conducting Remote Interviews]
* Include content on how to probe respondents on each question
* J-PAL, [https://www.povertyactionlab.org/blog/3-20-20/best-practices-conducting-phone-surveys Best practices for conducting phone surveys]
* Train the enumerators to answer potential questions the respondents might ask. These can be questions about the project, the design, the purpose of the interview, on each question, or anything else.
* J-PAL, [https://www.povertyactionlab.org/sites/default/files/files/2020/03/spouse-and-gender-relations-phone-survey-protocol.pdf Protocols for phone surveys]
* Include suggestions on how to keep respondents engaged during the interview. Some examples include
* J-PAL South Asia, [https://drive.google.com/file/d/1QYfamF0UG9-t6sceLxJyTNt0KKnixrp-/view Example of a training manual for phone surveys]
Building on established rapport between enumerator and respondent
* J-PAL South Asia, [https://www.povertyactionlab.org/sites/default/files/research-resources/2020/03/transitioning-to-CATI-Checklists.pdf Checklists and resources for transitioning to CATI]
Compensating respondents via mobile money or airtime if feasible
* Özler and Cuevas (World Bank), [https://blogs.worldbank.org/impactevaluations/reducing-attrition-phone-surveys?CID=WBW_AL_BlogNotification_EN_EXT Blog post on reducing attrition in phone surveys]
Including the expected survey duration in the consent form.
* Sandra V. Rozo (World Bank), [https://blogs.worldbank.org/impactevaluations/tips-collecting-surveys-hard-reach-populations?CID=WBW_AL_BlogNotification_EN_EXT Tips for surveying hard-to-reach populations]
Attempting to answer each question of the respondent as comprehensively as possible
Being inquisitive and kind with the respondent
Offering to escalate to the manager in case of any issue

Latest revision as of 19:34, 11 November 2021

Preparing for remote data collection involves preparing clear protocols and guidelines for each component of collecting data remotely (that is, from a location that is different from that of the respondent). The main components of remote data collection include timelines, approvals, instrument design, coding surveys, and enumerator training. Each of these components involves additional steps compared to preparing for field data collection. Therefore it is important to establish clear channels for regular communication among everyone involved in the data collection- the research team, government agencies, survey firm, and enumerators.

Read First

  • Remote surveys are useful for collecting data in cases where field surveys are either too expensive, or not possible due to lack of in-person access to respondents.
  • There are 3 broad kinds of remote surveys - phone surveys, web surveys, and recorded surveys.
  • Researchers consider phone surveys more efficient because they retain some degree of human interaction.
  • We will therefore look at the steps involved in preparing for phone surveys.
  • Most guidelines for this process can also be extended to cover other kinds of remote surveys.
  • DIME Analytics has also compiled a guide on transitioning to phone surveys when field data collection is not possible due to pandemics or natural disasters.

Survey Timeline

Preparing the timeline for a survey involves allocating sufficient time for each stage of the survey process - design, pilot, programming, and enumerator training. All of these steps require additional time, since there are additional protocols that need to be followed at each stage. The research team must account for this time, and allocate a reasonable window for each step. For instance, pre-piloting a phone survey might require 3 weeks, instead of 1-2 weeks for a field survey.

IRB Approvals

In addition to the components of a field study, the research team must also seek IRB approval for additional components of a phone survey.

  • Oral consent. The study must make provisions for obtaining oral consent from the respondent at the beginning of the phone interview.
  • Audio audits. Audio audits help to monitor and enrich the quality of data collected through a phone survey. The research team must clearly lay out the details of the process, and inform respondents that the interview is being recorded to improve data quality.
  • Instrument design. This involves seeking approvals for the content of the questionnaire, as well as aspects like the average duration of the survey.
  • Norms for handling PII. Switching to phone surveys will require hiring additional people and strengthening the norms to handle personally identifiable information (PII).
  • Incentives to respondents. The research team must set up a different system for giving incentives to respondents in a phone survey.

Note: The research must make changes to these components and seek IRB approval for all of these changes even while transitioning from an in-person survey to a phone survey.

Procurement

Procuring a survey firm involves drafting a terms of reference (TOR), and preparing a budget for each component of the survey process. These procurement-related documents should also reflect the fact that the data collection will be conducted over the phone, and not in-person.

  • Survey firm TOR. The terms of reference (TOR) to hire a survey firm must clearly specify the guidelines for conducting phone surveys. These guidelines should cover issues like the mode of data collection, average survey duration, survey timeline, and so on.
  • Survey budget/proposal. The survey firm that is hired to conduct the phone survey should submit a budget that allocates funds to cover equipment costs, incentives, and other administrative costs . An example of costs that may arise in the case of phone surveys is the cost of setting up a call center for enumerators. Note that this cost will not exist if enumerators are working from home, for instance, during a pandemic.

In case of a transition from an in-person survey to a phone survey, the survey firm may need more financial resources. The research team will then have to change the contract to reflect this, and ask for fresh approval from the organization funding the study. In case the World Bank is funding the study, the research team will have to obtain approval from Corporate Procurement.

Survey Instrument

For a phone survey, keep the following in mind while preparing the instrument (questionnaire):

  • Keep the survey short. General guidelines and best practices suggest that the average duration of phone interviews should not be longer than 20 minutes. However, reducing the length of a survey takes considerable amount of time. Make sure to account for this in the timeline update. You can try the following to reduce survey length:
    1. Drop modules which cannot be asked over phone. For example, GPS coordinates
    2. Identify the most essential outcomes of interest. Keep questions related to these outcomes
    3. Ask about aggregates instead of individual items. For example, there might be 25 different categories in survey that aims to assess consumption patterns . These can be combined to just a few categories for the purpose of a phone survey like durable items, non-durable, and food items. However, in such cases it is important to define what items fall under which category, and share this information with the respondent.
    4. Break the survey into 2 or 3 shorter surveys. It is acceptable to do so if other methods of reducing survey length do not work. However, the research team should remember to account for differential attrition which arises when some respondents complete only a few rounds of the survey.
  • Add identifiers at the start of the survey. This helps to ensure that the enumerator is interviewing the intended respondent.
  • Collect as many telephone numbers as possible. This is particularly helpful in case there is a need for a follow-up survey.
  • Keep the questions as concise and clear as possible. Add hints and definitions throughout.
  • Code and repeatedly test the updated survey.

Protocols

Collecting data using surveys involves clear protocols and considerations that all participants in the survey process must keep in mind. In addition to the protocols for field surveys, the guidelines for phone surveys will need to address the following concerns:

Team setup and hiring

One of the most important steps in the data collection process is hiring, and laying out the team structure. The following are the protocols for these components are:

  • Hiring. The research team should hire enumerators and supervisors who have prior experience with data collection.
  • Team setup. Ensure that the set up of the survey team is such that there are no more than 5 enumerators reporting to 1 supervisor. The research team may consider hiring more supervisors if needed.

Procuring equipment.

The research team will have to procure additional equipment and make it available to the enumerators. This will include providing telephone or mobile devices, SIM cards (and money to recharge SIM cards), tablets or computers to enter data, and internet access for each part of the process.

Communication

Clear communication protocols are important to ensure that the phone survey goes on smoothly. These protocols should cover the following aspects :

  • First point of contact. Make sure enumerators and supervisors know who is their first point of contact on the team.
  • Contact details of respondents. Make one or two people responsible for providing these details to enumerators everyday.
  • Phone credit for enumerators. Ensure that enumerators have the funds to call respondents in the sample.
  • Sharing collected data. Specify the frequency for sharing data - daily, alternate days, or weekly. Also specify exactly what data the enumerators need to share.
  • Method of sharing data. Specify how enumerators share the data they collect.
  • Monitoring enumerators. Specify the people who will monitor enumerator performance, and parameters to judge performance. For example, number of phone calls completed per day.
  • Enumerator feedback. Specify how to convey issues identified during data quality checks to the enumerators.
  • Escalating issues. Specify whom the enumerator should contact if a respondent requests an escalation or requests clarity on a question. For example, it helps to communicate with enumerators using instant messaging (IM) platforms.
  • Backup. Always have a plan B (and C) in place in case of problems like connectivity issues.

Special situations

Sometimes it is possible that the respondent is either not available, or someone other than the respondent answers. It is important to provide clear protocols for these situations. In cases where some respondents are not available, tracking sheets become very important. These are digital survey forms that the enumerator can fill. A tracking sheet can be used to track:

  • Respondent details. Names and contact numbers of all respondents in the sample.
  • Status of survey. Whether the respondent answered or not.
  • Call back time. In case the respondent was unavailable earlier.
  • Number of attempts. If the respondent did not call back.
  • Time of each attempt. To calculate average to contact all respondents in the sample.

In cases where someone other than the respondent answers the phone, the enumerator can do the following:

  • Take an appointment. The enumerator can ask the person answering the call to specify when the respondents will be available to talk. The enumerator can then calls the same number again at the specified time.
  • Replace respondent. Specify the number of attempts an enumerator should make to reach a respondent before replacing the respondent. Also, specify how long enumerators should wait before making another attempt. A good example of such a protocol specifies making 9 attempts in total, with maximum of 3 attempts per day and at least 3 hours between each attempt.

Data Quality and Security

Monitoring data quality is one of the most important parts of data collection. Poor quality data can at best reduce the effectiveness of a policy intervention, and at worst require a repeat of the entire data collection process. Therefore the research team must prepare clear guidelines for the following:

  • Type of data checks. Conduct regular back checks and high frequency checks.
  • Frequency of data checks. Specify how often the supervisor should conduct data checks.
  • Feedback method. Specify method for communicating feedback to the enumerators after a data check. Decide on this before any data collection starts.

It is equally important to specify clear protocols for data security to ensure no data is lost and no sensitive information is made public. This includes encrypting the survey form and providing guidelines to enumerators about how to share data. Create a confidentiality agreement that each enumerator signs (digitally) to ensure that contact numbers of respondents do not get used for any other purposes.

Note: These guidelines also apply in case there is a need to transition from an in-person survey to a remote survey. Make sure the survey manuals incorporate all these protocols, and share them with all members of the team.

Enumerator Training

Given the extremely detailed protocols that need to be in place, it is also important to train enumerators rigorously to make them familiar with all protocols, questionnaire content, and ways to keep respondents engaged.

In cases where it is not possible to train enumerators in person, such as in the case of a pandemic, this process becomes even more challenging. In such a case, the training can be conducted online. With regards to the logistics of this process, keep in mind the following:

  • Size of groups. If there are many enumerators break them down into smaller manageable groups (typically 5) to make it easier to conduct the training.
  • Platform. Use a secure web-calling platform. Depending on bandwidth, use the video feature where possible to provide some sense of personal connection. If an online live training is not possible, consider alternatives such as recorded videos and phone calls.
  • Connectivity issues. Record the training sessions and share them with enumerators in case they miss out on parts of the training due to connectivity issues. The recorded versions can also help enumerators practice all steps until they are confident.
  • Training schedule. Allocate more time in the training schedule for these additional steps and the possibility of facing connection issues.

Similarly, for the content of the training, follow these guidelines:

  • Training manual. Provide the team with a detailed training manual.
  • Script. Provide enumerators with a detailed script for the entire survey instrument(s) so that they know how to ask the questions.
  • Mock interviews. Create recordings of test interviews to use them for the training. Note that it is important to conduct more mock interviews during virtual training than you would in an in-person training
  • Asking questions. Include content on how to ask questions during the interview.
  • Doubts of respondents. Train the enumerators to clarify any doubts that respondents might have. These doubts can be regarding the project, the design, the purpose of the interview, or about a particular question.
  • Engaging respondents. Train enumerators on how to keep respondents engaged during the interview. Some ways to do this include making the respondents feel comfortable, compensating them for their time, clarifying their doubts (if any), offering to get them in touch with the manager if they have any concerns, among others.

Related Pages

Click here for pages that link to this topic.

Additional Resources

Retrieved from "https://dimewiki.worldbank.org/index.php?title=Preparing_for_Remote_Data_Collection&oldid=8393"