Remote Surveys

Depending on whether data is collected in-person or remotely, there are two methods of primary data collection - field surveys and remote surveys. Researchers often remote surveys to collect data in areas that are not accessible, or are affected by conflict. In such cases, researchers can still get access to data that can help them solve common development challenges such as lack of access to water, electricity or healthcare.

Read First

• Primary data collection is the process of gathering data through surveys, interviews, or experiments.
• Dime Analytics Guidelines on preparing for data collection
• Remote surveys are also a cost-effective method of performing a follow-up after an initial in-person interview.
• Remote surveys can be of different types, depending on the medium over which they are conducted: phone surveys, web surveys, and recorded surveys.

Phone Surveys

A phone survey or a computer-assisted telephone interview (CATI) is one where the enumerator calls the respondent and asks them the questions over telephone. The enumerator enters these responses onto a programmed digital survey, which is then shared electronically with the research team. Phone surveys have 2 major advantages. Firstly, they allow data to be collected remotely, thus saving resources since collecting data through face-to-face interviews can often be expensive. Secondly, they are a great option for collecting data in emergencies or in conflict areas while still retaining the personal touch that researchers prefer.

However, when a research team decides to use a phone survey for collecting data, they must keep in mind concerns about:

1. Feasibility. Whether or not a phone survey is feasible.
2. Preparation. Things to keep in mind before starting data collection.
3. Communication and Data Quality. During data collection.

Feasibility

The research team must keep in mind that in a lot of cases, data collection through phone surveys will not be feasible. In that case, they will have to rely on field data collection, or pause data collection in case in-person data collection is not possible. In order to decide whether conducting a phone survey is feasible, ask the following questions:

• Is it practical to ask these questions over a telephone?

This is an important consideration because in some cases asking questions over the phone can drastically alter the nature of data that is collected, or the manner in which respondents interpret questions in the survey.

Examples of questions which cannot be asked over the phone are: questions on sensitive topics (such as gender-based violence), and questions with test components (such as a Math test for a school-based survey).

• How long is the survey?

General guidelines and best practices suggest that phone interviews should be short and longer surveys result in lower response rates. If your original questionnaire (for a field survey) took longer than 20 minutes to answer (on average), you might need to reduce the length of your questionnaire or the number of questions when transitioning to a phone survey.

However sometimes it is not possible to reduce the duration of the survey without affecting the outcomes of interest. In such cases, you might consider the feasibility of conducting the survey over multiple phone calls.

• Does a majority of the target population have access to telephones?

This is important because if the target population does not have access to telephones , the survey will introduce a selection bias into the sample. Further, it will lead to lower response rates and reduce the cost-effectiveness of the survey.

• Are contact details of the target population easily available?

In the case of a follow-up survey (that is, if a prior round of surveys has already been conducted), it is likely that the research team has already collected contact information about the respondents. But in the case of a baseline survey (that is, when it is the first round of survey), if the research team does not have access to the contact numbers, they can consider random digit sampling (RDS). This is a method for selecting people for involvement in telephone statistical surveys by generating or selecting telephone numbers at random. However it is only possible when the research team has access to a long list of phone numbers from the population.

If the answers to the above questions are yes, then the research team can move to the next step, that of preparing for remote data collection. In the next section, we discuss the logistics of data collection for a phone survey.

Preparation

The steps involved in preparing for phone surveys are different from the steps in preparing for a field survey. At each stage of this process, everyone involved in the data collection (government agencies, survey firm, enumerators) must communicate clearly. The research team must carefully work through certain additional steps before conducting a phone survey. These steps deal with the following:

1. Timeline for a phone survey.
   The research team agrees on a comprehensive timeline of a field survey pilot at the beginning of the process of preparing a survey. The research team must therefore keep in mind that for a phone survey, each step leading up to, and including, the data collection will take additional time. Therefore it is important to account for this, and allocate a bigger window for each step.
2. IRB approvals for a phone survey.
   Every survey must obtain an IRB approval. For a phone survey, the research team must obtain additional approvals for aspects like oral consent, audio audits, and incentives for respondents. (link to Preparing for Remote Data Collection#IRB Approvals)
3. Procurement-related documents.
   Procuring a survey firm involves drafting terms of reference (TOR), and preparing a budget for each component of the survey process. These procurement-related documents should also reflect the fact that the data collection will be conducted over the phone, and not in-person. The research team must spend time on estimating the costs of procuring adequate equipment, such as setting up a call-center for interviewers.
4. Questionnaire for a phone survey.
   The process of designing a field survey involves various considerations. However, there are additional considerations while designing a phone survey, such as managing the length of the survey, including verbal consent, and including audio audits.
5. Remote data collection protocols.
   Similarly, the protocols for phone surveys are different from protocols for field surveys. Some of these differences are in terms of the team setup, communication protocols and data-quality checks.
6. Enumerator training.
   Enumerator training for phone interviews also involves certain additional steps compared to Enumerator training for in-person interviews. This process becomes even more challenging in the case of a work-from-home phone survey, for instance, when a pandemic like COVID-19 prevents the research team from training enumerators in-person. In such cases, the research team must establish certain logistical and content-based changes to the training method. This includes sharing recorded training sessions, preparing enumerators for follow-up questions from respondents, and discussing steps to keep respondents engaged.

Note: These steps will be useful in case the the research team has to transition from an in-person survey to a phone survey, for instance, during a pandemic or natural disaster.

Communication during data collection

Since there are so many aspects involved in the process of data collection, it is important to make sure that all members of the research team communicate frequently and effectively. Communication becomes even more important when data collection is being carried out remotely. Some guidelines to keep in mind are:

• Set up strong monitoring and communication protocols before starting data collection. Test them during training and the pilot, to identify any gaps that might arise.
• Set up a clear chain of command, and make sure each enumerator knows who their first point of contact (POC) is on the research team.
• Schedule regular calls to check-in with each enumerator, and to resolve any concerns they may have.
• Conduct team meetings at least once a week.
• Consider using a team-collaboration and communication tool to maintain a 24X7 channel for communication with supervisors in the survey team. You may add enumerators too if the survey team is small.
• Monitor data carefully, and on a daily basis. Provide regular feedback to enumerators on the quality of data being collected.

Data quality assurance

When dealing with phone surveys, the research team must run data quality checks more extensively. Since phone surveys have an inherent degree of self selection, it is essential to keep track of outcomes of interest. This is because self selection can result in samples which are not entirely representative. Therefore, in addition to the guidelines for monitoring field data quality, there are some steps that the research team must follow to monitor the quality of phone data in the context of phone surveys:

• Track progress and completion rate of surveys more closely.
• Since the survey is short, include consistency checks on all questions if possible.
• Include more stringent reviews on enumerator performance, and share this feedback with enumerators regularly.
• Insert audio audits at predefined or random points of the survey. Then ask the supervisors to review the audio audits on a daily basis and cross-check the responses with the submitted forms. One way to do this is to create an audio audits form for the supervisor to fill in while reviewing audio audits.
• If conference calls are feasible, the supervisor can shadow the enumerator a few times to ensure that the enumerator is following all guidelines.
• Ask the supervisor to call a few respondents at random to confirm that the enumerator actually called them
• Back checks can be run on certain questions as well.
• Carefully track data quality throughout the process of data collection. This is a departure from in-person data collection where the emphasis on data quality checks is higher during the initial days of data collection.

Back to Parent

This page redirects from primary data collection.

Additional Resources

• Andrew et. al. (World Bank Group), General guide on conducting phone surveys
• Özler and Cuevas (World Bank), Blog post on reducing attrition in phone surveys
• Goldstein and Kondylis on IEs in the time of Covid-19
• Markus Goldstein (World Bank), Blog post on using phones for repeat surveys
• Herath et al. (J-PAL Africa), Article on increasing response rates
• J-PAL, Best practices for conducting phone surveys
• J-PAL, Protocols for phone surveys
• J-PAL South Asia, Checklists and resources for transitioning to CATI
• IPA, Guide on conducting phone surveys during a pandemic
• Tavneet Suri, Webinar on adaptations for phone surveys