Difference between revisions of "Structuring a Survey Pilot"

Jump to: navigation, search
(Created page with "* * Plan sufficient training time for interviewers! ** Length depends completely on complexity of instrument and survey protocols. 1-day minimum ** Interviewers must be famil...")
 
Line 1: Line 1:
* * Plan sufficient training time for interviewers!
= Guidelines for Structuring Survey Pilot =
** Length depends completely on complexity of instrument and survey protocols. 1-day minimum
== Training Enumerators for the Survey Pilot ==
** Interviewers must be familiar with the instrument and the objectives of the pilot by the end  
* Length of training depends completely on complexity of instrument and survey protocols. 1-day minimum.
** Interviewers will have useful insights and feedback on the survey instrument at the training itself. Plan time to incorporate their feedback/ make revisions before starting the actual pilot
* Interviewers must be familiar with the instrument and the objectives of the pilot by the end  
** Build in a minimum of 1 day between training and the start of the piloting
* Interviewers will have useful insights and feedback on the survey instrument at the training itself. Plan time to incorporate their feedback/ make revisions before starting the actual pilot
** For data-focused pilots, essential to have interviewers do mock interviews with each other to familiarize themselves with programming, and catch any bugs missed in office tests.  
* Build in a minimum of 1 day between training and the start of the piloting
* For data-focused pilots, essential to have interviewers do mock interviews with each other to familiarize themselves with programming, and catch any bugs missed in office tests.  
 
==IRB Approvals==
* Check what approvals are needed. Sometimes local IRB approval is required even for a pilot.  
* Check what approvals are needed. Sometimes local IRB approval is required even for a pilot.  
** A letter of support from the relevant Ministry or implementing partner always helps.  
* A letter of support from the relevant Ministry or implementing partner always helps.  
* Plan time for group feedback and discussion sessions at the end of each day.  
 
==Feedback Sessions & Revisions==
* Plan time for group feedback and discussion sessions at the end of each day.
* Finish fieldwork early so there is time to debrief. You will get better feedback if the team is not tired and hungry!
* Plan sufficient time to make revisions each evening, and pilot again the next day
* Plan sufficient time to make revisions each evening, and pilot again the next day
** If logistically feasible, best to pilot every other day (otherwise all-nighters are common)
** If logistically feasible, best to pilot every other day (otherwise all-nighters are common)
Line 15: Line 21:
** Best to work with an assistant who speaks the local language to make the edits in that version of the survey while the FC makes them in the English version.  
** Best to work with an assistant who speaks the local language to make the edits in that version of the survey while the FC makes them in the English version.  
*** Ideally this is someone other than an enumerator (they should go rest so they're fresh for the pilot on the following day, and end of the day edits can go well into the night if piloting needs to happen the next morning)
*** Ideally this is someone other than an enumerator (they should go rest so they're fresh for the pilot on the following day, and end of the day edits can go well into the night if piloting needs to happen the next morning)
** Where logistically possible, piloting every other day is a better plan
 
==Length of the pilot==
* Pilot until there are no more substantial changes to be made. Field plan will depend on:  
* Pilot until there are no more substantial changes to be made. Field plan will depend on:  
** Extent of changes to be made  
** Extent of changes to be made  
** Availability of printing facilities (if paper pilot) in the area where you are piloting.  
** Availability of printing facilities (if paper pilot) in the area where you are piloting.  
** Where the pilot location is (if far from home-base, and teams are staying in the area, breaking for a full day may not be practical / cost-effective)
** Where the pilot location is (if far from home-base, and teams are staying in the area, breaking for a full day may not be practical / cost-effective)
==Special considerations for follow-up surveys==
* If you will be ‘pre-loading’ data during the survey (e.g. from a baseline survey), you will need to simulate this during the pilot.  
* If you will be ‘pre-loading’ data during the survey (e.g. from a baseline survey), you will need to simulate this during the pilot.  
** If you have reduced the sample size from previous rounds, you can use non-sampled households for whom baseline data exists.  
** If you have reduced the sample size from previous rounds, you can use non-sampled households for whom baseline data exists.  
** Otherwise you may need to do ‘pre-interviews’ one day ahead to collect basic indicators for pre-loading. This is logistically challenging, but very worthwhile!
** Otherwise you may need to do ‘pre-interviews’ one day ahead to collect basic indicators for pre-loading. This is logistically challenging, but very worthwhile!
* Ideally the survey firm will participate in the data-focused pilot. This has to be clearly specified in the Terms of Reference (for more details see Guidelines on Survey Firm Procurement). Important points:
 
** Timeline includes a few days’ pause between the end of the piloting and enumerator training, to incorporate any all final revisions
= Back to Parent Article =
** Distinguish between the data-focused pilot and the field practice during enumerator trainings.
This article is part of the chapter [[Survey pilot]]
** Make sure the timeline for the piloting is clear from first discussions.
 
= Further Reading & External Resources =

Revision as of 18:36, 16 January 2017

Guidelines for Structuring Survey Pilot

Training Enumerators for the Survey Pilot

  • Length of training depends completely on complexity of instrument and survey protocols. 1-day minimum.
  • Interviewers must be familiar with the instrument and the objectives of the pilot by the end
  • Interviewers will have useful insights and feedback on the survey instrument at the training itself. Plan time to incorporate their feedback/ make revisions before starting the actual pilot
  • Build in a minimum of 1 day between training and the start of the piloting
  • For data-focused pilots, essential to have interviewers do mock interviews with each other to familiarize themselves with programming, and catch any bugs missed in office tests.

IRB Approvals

  • Check what approvals are needed. Sometimes local IRB approval is required even for a pilot.
  • A letter of support from the relevant Ministry or implementing partner always helps.

Feedback Sessions & Revisions

  • Plan time for group feedback and discussion sessions at the end of each day.
  • Finish fieldwork early so there is time to debrief. You will get better feedback if the team is not tired and hungry!
  • Plan sufficient time to make revisions each evening, and pilot again the next day
    • If logistically feasible, best to pilot every other day (otherwise all-nighters are common)
    • For pen-and-paper pilots, make sure you will have access to a printer to make and share revisions in real time.
      • Depending on context, using research budget to purchase a printer the FC can travel with may be necessary/ cost-effective.
    • Be aware of the need for careful version control: If the survey is in a language the FC doesn't understand, it can be tricky to keep track of daily changes in both local language and English version.
    • Best to work with an assistant who speaks the local language to make the edits in that version of the survey while the FC makes them in the English version.
      • Ideally this is someone other than an enumerator (they should go rest so they're fresh for the pilot on the following day, and end of the day edits can go well into the night if piloting needs to happen the next morning)

Length of the pilot

  • Pilot until there are no more substantial changes to be made. Field plan will depend on:
    • Extent of changes to be made
    • Availability of printing facilities (if paper pilot) in the area where you are piloting.
    • Where the pilot location is (if far from home-base, and teams are staying in the area, breaking for a full day may not be practical / cost-effective)

Special considerations for follow-up surveys

  • If you will be ‘pre-loading’ data during the survey (e.g. from a baseline survey), you will need to simulate this during the pilot.
    • If you have reduced the sample size from previous rounds, you can use non-sampled households for whom baseline data exists.
    • Otherwise you may need to do ‘pre-interviews’ one day ahead to collect basic indicators for pre-loading. This is logistically challenging, but very worthwhile!

Back to Parent Article

This article is part of the chapter Survey pilot

Further Reading & External Resources