Survey Pilot Participants

Typically, conducting a survey pilot requires several stages of planning and discussions. Apart from the impact evaluation team (or research team), survey pilot participants include interviewers, respondents, and even local government agencies. For instance, during the actual field data collection, such as in a computer-assisted personal interview (CAPI), it is the interviewer who reads out the questions to the respondent. But in the piloting stages leading up to the actual survey, people like the field coordinators (FCs) and the principal investigators (PIs) oversee the overall preparation and finalize the survey protocols.

Read First

• Survey pilot refers to the process of testing different components of a survey, including content and protocols.
• The pilot is conducted before procuring a survey firm.
• The research team must decide on the timeline and structure of a pilot.
• A complete pilot has three stages - pre-pilot, content-focused pilot, and data-focused pilot.
• Each of the pilot participants has a specific role during each of the three stages.

Characteristics

		Stage 1 - Pre-Pilot	Stage 2 - Content-focused Pilot	Stage 3 - Data-focused Pilot
Respondents		Respondents for the pilot should be as similar as possible to respondents for the actual survey in terms of age, gender, education, and socioeconomic (income) status. Neighboring villages, schools, or firms are good options for selecting respondents for the pilot. Never use any of these respondents in the final sample.	Test the protocols for sampling in this stage. Use these protocols to select respondents for the pilot. This helps to find out if the sampling frame (list from which respondents are selected) needs to be revised. It also helps to reduce selection bias.	Re-test your sampling protocols. Include revisions based on the previous round of piloting.
Enumerators (or interviewers)	Language	Fully fluent (can read, write and/or speak) in the local language(s) and the language of research team.	Fully fluent in the local language(s) and the language of research team.	Fully fluent in the local language(s) and the language of research team.
	Required experience	Qualitative experience as an enumerator. For instance, experience in identifying issues in wording and structure of questions. Sectoral knowledge. For instance, knowledge about agricultural patterns in a survey that aims to collect data on farm output.	Quantitative experience as an enumerator. For instance, experience in entering data electronically. Sectoral knowledge.	Quantitative experience as an enumerator. Experience working with phones or tablets.
	Background	The pool of interviewers should be diverse in terms of age, race, religion, gender etc.	Interviewers should be mindful of (and learn about) cultural considerations. For instance, if women can interview men, or if men can interview women in the study area.	The background of the team of interviewers should be similar to that of the final survey team, which is decided based on previous stages of piloting.
	Size of the team	Very small - not more than 2. A member of the research team should always accompany the enumerators.	Small - between 2 and 4. Ideally, a member of the research team should monitor each enumerator.	Large - between 4 and 8. More enumerators help simplify the process of monitoring data quality and debugging (identifying errors) in the coded instrument.
Survey firm		Not part of this stage. This stage is typically conducted before survey firm is on board.	Not part of this stage. This stage is typically conducted before survey firm is on board.	Part of this stage. In fact, the survey firm should lead the pilot in this round.
Members of the research team		The field coordinators (FCs). Ideally they should participate in each stage. The principal investigators (PIs) should participate directly. They should also conduct daily discussions to debrief (exchange notes with) the enumerators.	The field coordinators (FCs). The impact evaluation coordinator (or research manager) should participate directly.	The field coordinators (FCs). The programmer should participate directly (to debug the instrument in real time) Members of the research team who will be field supervisors in the actual survey should act as interviewers in this stage. This gives them experience with the instrument, which helps them conduct enumerator training.
Others		Local research assistants (RAs). Includes short-term consultants (STCs) hired through the World Bank or through a local principal investigator (PI). Local government staff. University students.	Local research assistants (RAs). Local government staff. University students.	Local research assistants (RAs). Local government staff. University students.

Guidelines

Finally, the pilot participants should keep certain best practices in mind while planning and conducting a survey pilot. These can improve the outcomes of the pilot, for instance, by reducing the down-time (gap) between surveys.

• Throughout the process of planning the pilot, discuss with other members of the research team, and take notes of what needs to be part of the pilot.
• Take careful notes of the clarifications that come up during the review sessions. These will be an important part of the enumerator manual which supervisors use to train enumerator.
• Hire a local mobilizer to coordinate with respondents. Mobilizers explain the purpose behind conducting the survey to the respondents, and facilitate the process of obtaining consent. This is particularly helpful in urban areas, or in cases where respondents are busy.

Related Pages

Click here for pages that link to this topic.

Additional Resources

• DIME Analytics’ guidelines on piloting surveys
• DIME Analytics’ guidelines on survey design and pilot