Checklist: Content-focused Pilot
Get printable version here. For more detailed instructions on how to implement the different tasks in this checklist, see questionnaire design. Note that this checklist is best displayed in Chrome, Firefox, Safari or any other modern browser.
| |||
|---|---|---|---|
| Project name: _______________________________________ | |||
| Country: ___________________________________________ | |||
| District: ____________________________________________ | |||
| Year, Month and/or Day: _____________________________ | |||
| 1. Pre-pilot | |||
| Initials | #No | Checklist Item | |
| [ __ ] | 1.1 | Are the conceptual / structural issues identified in the early questionnaire design process sufficiently explored? | |
| [ __ ] | 1.2 | How might the composition of the focus (gender, age, religion, caste/socioeconomic status) affect responses? | |
| [ __ ] | 1.3 | Try to get at how potential respondents think about the key indicators you are trying to measure | |
| [ __ ] | 1.4 | Do you have a translator providing simultaneous translation? If not, does the amount of note-taking correspond to the amount of discussion? | |
| 2. Survey Design | |||
| Initials | #No | Checklist Item | |
| [ __ ] | 2.1 | Do the questions make sense to the respondent? | |
| [ __ ] | 2.1.1 | Watch how the respondent reacts to each question – any confusion? How is the reaction time? | |
| [ __ ] | 2.1.2 | Are there questions that require explanation by enumerator, or clarification from respondent? | |
| [ __ ] | 2.1.3 | Follow-up with the enumerator (and possibly the respondent) on questions that seemed problematic: is the issue translation? Phrasing? Conceptual? Cultural? | |
| [ __ ] | 2.2 | Are answer options comprehensive? | |
| [ __ ] | 2.2.1 | Ensure that all ‘other’ responses are specified and recorded | |
| [ __ ] | 2.3 | Is the enumerator following the scripted translations? | |
| [ __ ] | 2.3.1 | If not, ask the enumerator to note any translation issues to discuss with the team. | |
| [ __ ] | 2.3.2 | If you do not speak the language, you can still note if interviewer’s questions were noticeably longer/shorter than the written question. | |
| 3. Interview flow and timing | |||
| Initials | #No | Checklist Item | |
| [ __ ] | 3.1 | How is the flow of the interview? | |
| [ __ ] | 3.1.1 | Any pauses? (likely areas where interviewers need more instructions) | |
| [ __ ] | 3.1.2 | Are there times when the respondent looks bored? Uncomfortable? Losing interest? | |
| [ __ ] | 3.2 | Could the order of modules be improved? The order of questions within modules? | |
| [ __ ] | 3.3 | How long does the interview take? | |
| [ __ ] | 3.3.1 | Check length of each module by noting start and stop time. | |
| [ __ ] | 3.3.2 | Expect that pilot interviews will take much longer than actual interviews (likely twice as long) – interviewers are expected to do extra probing, take qualitative notes, and record open-ended responses, and the survey instrument may not yet flow well | |
| The checklist are edited through Git Hub. This checklist corresponds to the file with the name chk_questcont.js. To read a simple step by step guide on how to edit the checklist, see this documentation: https://github.com/worldbank/DIMEwiki/tree/master/Topics/Checklists. | |||
Related Pages
See here for other important check lists.
Additional Resources
- DIME Analytics’ guidelines on survey design and pilot