Dimewiki

Difference between revisions of "Enumerator Training"

Jump to: navigation, search
Line 60: Line 60:


== Complementary Approach  ==
== Complementary Approach  ==
=== Content ===
Providing the team with an overview of the background, objectives and relevance of the project is a key way to ensure the team becomes engaged and motivated to conduct high quality data collection. Conveying the importance of their role in the research will allow enumerators to take ownership of the project, a key requirement in ensuring enumerators remain committed throughout the duration of the data collection phase. This will be particularly important for projects lasting several weeks or requiring a significant amount of travel, both of which can result in enumerator fatigue.
====Core modules====
The time dedicated to training on the questionnaire content will depend on the complexity of the questions themselves, the use of scales or experiments etc. For example, scoring scales or questions about perceptions, attitude or motivation, will require more in-depth instruction which will include a detailed review of what is being measured and how this should be captured, as well as various practical exercises which will allow the enumerators to test their understanding, and the trainers to tailor subsequent sessions to the areas in which there is a discrepancy or inconsistency in comprehension.
It will be important not to assume that enumerators will understand questions in the same way, as even relatively straightforward questions or items can be interpreted differently, particularly if working with people with different levels and types of experience. It is better to dedicate time to going through all questions, however briefly, in the training than to deal with inconsistencies after the data collection has begun. Preparing and Enumerator manual and additional materials to support the training, on which enumerators can take notes to review after the training, and which they can refer to throughout the fielding of the survey, will be crucial.
====Interviewing====
Spending some time on interviewing techniques is vital. While the content of this part of the training will depend on the nature of the questionnaire, general tips on how to conduct a good interview will also be helpful. Some general topics for discussion include:
*'''Eliciting good data''': some respondents are likely to have confidentiality concerns, so it is of utmost importance that the enumerators make it clear throughout the interview, both by stating so openly and in the way they approach sensitive topics, that there is no risk for them in participating as all answers will be anonymized and kept confidential. Further, respondents should be made to understand that the aim of the interview is to learn from them, capture their personal thoughts, opinions and beliefs, and not the official or sanctioned versions.
*'''Time management''': how long interviews last will vary depending on the questionnaire and the respondent. While it is normal for the duration to vary, time management will be essential if enumerators are to complete their target number of interviews each day. Not only does an overly long interview have the potential to cut short another interview, it also risks not being completed. Enumerators should have an idea of approximately how long they should spend on each section of the questionnaire, and should be especially careful with the more complex sections as these risk taking longer if the respondent is not clear on what he or she is being asked. A good understanding of the questions and concepts underlying each one will allow enumerators to run through the interview at a good pace.
*''Respondent fatigue'' can impact data quality and affect the results of the survey. It can result from respondents becoming bored, tired, or uninterested with the interview and begin to respond at a substandard level. This can be caused by several things, including an overly long interview, poor interviewing skills, or unease with the questions being asked. Enumerators can prevent this from happening by ensuring the respondents are fully aware of the value of their participation, by keeping the interview within a reasonable time and by interacting with the respondent in an engaging and interested manner.
====Using tablets====
Tablets and electronic data collection platforms such as Survey CTO are relatively straightforward to use, but time should be set aside to go through their correct use, as well as all data uploading protocols. While electronic methods provide a safe and easy way to collect and upload data quickly and efficiently, incorrect use of the tablet can lead to irretrievable data losses so it’s crucial that all enumerators are clear on the protocols to follow, such as saving, reviewing and uploading data. It is advisable to have a set of step-by-step instructions on how to do this rather than relying on the assumption that all enumerators will be comfortable using electronic devices. Before starting with Survey CTO on the tablets, it is important to ensure all the participants know how to turn on the device, unlock the screen, modify the screen brightness, activate and disable an app, etc. A good use of the electronic device and its settings  allow an optimal use of the Survey CTO software.
In case paper forms are used in the survey (e.g. for self-administered questionnaires), protocols should also be clearly set out to ensure all forms are labelled correctly and organized appropriately.
Is is strongly recommended to go through the entire paper questionnaire before moving to the electronic questionnaire on the device. Enumerators should have a good understanding of the structure of the questionnaire and the modules before using the tablets. Even if skip parterns are programmed on Survey CTO, enumerators should be able to identify inconsistencies on the electronic version of the survey.
Dime Analytics has developed a [https://osf.io/n7ctd/ SurveyCTO user guide for data collectors]. This manual can be use as training material and goes through all the important steps using SurveyCTO and the device for the data collection: life of the survey form, tablets preparation, process to send the forms…
=== Structure ===
==== Agenda for enumerator training ====
Use the Enumerator Manual as the guide for structuring the training agenda. A typical agenda includes:
# Introductions & Overview of Study (0.5 days)
# Review of Survey Protocols (0.5 - 2 days)
# Review of Questionnaire (paper version) (2.5-4 days)
## This is done module-by-module. Each question is read aloud and discussed. At the end of each module, a mock interview.
# Review of research standards and confidentiality (0.5 days)
# Introduction to tablets to be used for data collection (0.5 - 1 days)
# Review of Questionnaire (electronic version) (2-3 days)
# Mock interviews in-classroom (1-2 days)
# Field testing (1-2 days)
# Final team selection and logistics (1 day)
==== Training tips ====
The training should be highly interactive. Pop quizzes at the beginning of each day are a good way to keep people's attention, and to catch any comprehension gaps in real time.
Include both classroom training and field testing. Allow for anonymous questions.
Classroom training should include both large and small group sessions. Ensure that every participant practices all survey components (introductions, consent, survey modules) in front of others and receives feedback.
It will seems tedious to read through and practice every single question, but it is a key part of the training and very important.
==== Duration of Training ====
Allocate plenty of time for training! [[Survey Firm Procurement|Survey Firms]] often push for short training, but rushing the training will only cause problems and delays later on.
For a typical complex, multi-module household surveys, 10 days is a good estimate for minimum training time. Exact duration will depend on the complexity of the survey instrument, level of education of the enumerators, and number of people being trained.
==== Enumerator Selection ====
Do train more enumerators than will ultimately be needed. Let the full group know in the beginning their selection into the final team will depend on their performance. Test them early and often.
Use objective measures of performance, such as:
* Active participation in training, punctual arrival every day
* Understanding of survey modules and underlying concepts
* Scores on quizzes
* Feedback from the Field Testing (including interviewing skills)
Explain the performance measures, so it is clear that the final selection is transparent.
Test early and often.
==== Modifying the Questionnaire & Enumerator Manual ====
Throughout training, mistakes may be found in questions, translations, programming, or the manual. Take notes each day on necessary changes, and updated materials daily if possible. Do not underestimate the value of feedback from enumerators- if they have concerns or issues of understanding, it is likely respondents will too.
Print an updated Manual at the end of training, and provide it to all selected enumerators in a plastic (waterproof) folder.


== Assessing Enumerators ==
== Assessing Enumerators ==

Revision as of 21:44, 23 June 2020

Enumerator training is an extremely important part of the primary data collection, and should be planned in advance. It is a joint effort between the field coordinators, the survey firm, and other members of the impact evaluation team (or research team). The research team prepare and approve a field manual. The field manual acts as the basis for the training content, and helps organize the training.

Read First

  • The research team should make sure all members of the field team are familiar with the survey protocols and survey design by the end of the enumerator training.
  • Always train more enumerators than are required for the field data collection.
  • Select the best enumerators at the end of the training, based on rigorous enumerator assessments.
  • The training should ensure that enumerators are aware of the scientific approach, and ensure confidentiality of respondents during the survey.
  • Broadly, the training should cover the following aspects - objectives, planning, content, structure, and enumerator assessment.

Training Objectives

The research team should use the enumerator training to provide the rest of the team members with a clear overview of the context, objectives, and relevance of the impact evaluation. A good, well-organized enumerator training deals with the following aspects:

  • Key roles: The training should also ensure that all members of the research team, survey firm, and the field team understand their roles and duties. This allows everyone to take responsibility of their tasks, and remain committed throughout the process of data collection. For instance, the survey firm executes the tasks involved in data collection, while the field coordinators (FCs) supervise these tasks, and ensure quality of the work done by enumerators and the survey firm. Similarly, the research assistants (RAs) provide support in preparing the data quality assurance plan.

Planning

Before starting with enumerator training, it is important for everyone involved in the data collection to be aware of their roles and responsibilities. Planning is a continuous process that requires constant interaction between the survey firm and the field coordinators (FCs). This stage has the following components:

  • Logistics and recruitment
  • Train support staff
  • Develop field manual
  • Finalize time frame

Logistics and recruitment

The survey firm is responsible for coordinating logistics, which includes finalizing the training venue, and providing materials like printed questionnaires (or survey forms) and training agenda, as well as tablets, pens, and notebooks. The survey firm is also responsible for hiring potential enumerators and skilled supervisors to help with the training. In this process, the survey firm should coordinate with field coordinators (FCs) to understand the context of the impact evaluation, and become familiar with the questionnaire content.

Train support staff

In the context of enumerator training, the following people are considered a part of the support staff - survey facilitators, survey firm managers, and potential supervisors. The field coordinators (FCs) are responsible for training support staff to make sure they are familiar with various aspects of the project, including the context of the study, the questionnaire content, and the potential survey protocols. The support staff can then work with survey firm during the actual enumerator training.

Develop field manual

A field manual is extremely important because it acts as a resource for enumerators during the field survey. Field manuals also contain all field protocols, provide crucial guidelines to the survey firm, and also provide content for the training. A comprehensive field manual (see Figure 1 for template) should list the following:

  • Study objectives: The field manual should briefly explain the purpose of the study, and the possible outcomes that the research team hopes to achieve. This provides enumerators and field teams a good reference during the actual field interview, and helps them understand their roles more clearly.
  • Roles and responsibilities: The field manual should also list the roles and responsibilities of each member in the field team. This allows field staff to take more responsibility for their work, and perform their tasks efficiently.
  • Survey protocols: Survey protocols play an important role in ensuring high data quality in the field. The field manual should list all protocols, along with examples that explain the importance of following these protocols.
  • Key terms: The field manual should clearly define all key terms that are used in the questionnaire, as well as throughout the field manual. Key terms include common acronyms like Open Data Kit (ODK), and technical terms like sampling frame.
  • Description of questions: The field manual should also explain the questions that are part of the questionnaire, along with common rules and methods for asking questions during the field interview.
Figure 1: Template for developing a field manual

Finalize training time frame

The training time frame refers to the duration of the enumerator training, and depends on factors like:

  • Length and complexity of the questionnaire. If the questionnaire is longer, and is more complex (that is, has several modules, and many repeat groups), then the training will also need to be longer to make sure enumerators are comfortable with the questionnaire.
  • Capacity of potential enumerators. If the potential enumerators are more experienced, the duration of the training will be shorter, compared to a situation where the potential enumerators have less experience.
  • Complexity of study design. Again, if the study itself is based on a complex theory of change, or is trying to answer questions that were not a part of any previous studies, the training too will have to be longer to explain the objectives and protocols.

Further, keep the following points in mind when deciding the training time frame:

  • Allow for sufficient rest. Include sufficient time to rest after the sessions.
  • Include time for practice. Include extra days for practice in the classrooms, as well as in the field.
  • Include a day for enumerator selection. Include at least one day for the process of selecting enumerators for the actual survey.
  • Use the field manual. Use the field manual as a guide for finalizing the time frame, since the manual contains all information about the study and its various aspects.

Complementary Approach

Assessing Enumerators

After the enumerator training is complete, the field coordinators (FCs), the survey firm, and the supervisors should coordinate to conduct the enumerator assessment. Always train more enumerators than are needed for the actual survey (or interview). This motivates enumerators to perform better. It also ensures that a group of qualified enumerators are available as backup in case a few enumerators are unable to conduct the interviews on a given day. The following are the main criteria for enumerator selection:

  • Quiz scores
  • Observations during field practice
  • Participation
  • Interpersonal skills
  • Previous experience

Quiz scores

Although quiz scores may seem like a highly academic criteria of enumerator assessment, they offer important feedback which field teams can use to improve the training sessions. It is important to reassure the enumerators that the quiz scores act more as measure of how much progress each enumerator has made during the enumerator training, and less as a measure of performance. Be creative while designing the quiz. The quiz should assess the following:

  • Understanding of materials. Use the quiz to get an idea of how well an enumerator has understood the training materials such as field manuals, protocols, and standard guidelines for conducting interviews.
  • Reading skills in different languages. The quiz should also evaluate basic reading skills in the relevant languages. This also includes assessing familiarity of enumerators with the translated versions of the questionnaire in various languages. For example, if the questions are to be asked in English and Hindi, it is important to ensure that enumerators are able to read out the questions in both languages during the interview.
  • Understanding of questions. The quiz should also assess if enumerators are able to understand the meaning and relevance of certain questions. For example, in a COVID-19 pulse survey, it would be important for the research team to assess how households are preparing to deal with the economic and health-related consequences of COVID-19. In this case, for a question about how households assess the threat of COVID-19, enumerators must be able to understand the question themselves, before asking the respondents. Further, enumerators should be able to explain how respondents can answer the question using a scale from 1 to 5, with 1 meaning "No threat", and 5 meaning "Severe threat".
  • Numeracy skills. The quiz must also assess basic numeracy skills of enumerators, including the ability to enter responses on a tablet.

The research team and the field staff should also keep the following things in mind while designing a quiz:

  • Be well-organized. Create the quiz before the start of enumerator training. Edit the quizzes based on observations during the training sessions.
  • Conduct regular quizzes. Carry out one quiz per day, to test understanding of topics covered on the previous day. Correct the quizzes quickly, ideally on the same day. Share feedback simultaneously, to allow enumerators to correct their mistakes and fill any gaps in their understanding.
  • Be transparent. Inform enumerators before the start of training that they will be required to take regular quizzes. Share the skills that they should focus on in order to fully utilize the training. This also allows enumerators to concentrate better during sessions.
  • Encourage enumerators. Quizzes can often be stressful, so motivate enumerators throughout the training. Inform the enumerators that their scores in these quizzes are only one of the several criteria for evaluating them. Provide constructive feedback to enumerators after sharing results of each quiz.

Observations during field practice

It is important to plan for one day of field practice. For some surveys, the field practice is part of the Survey pilot. All participants should administer at least one interview. Split the enumerators into groups and have one facilitator in each group to monitor and observe interviews. Facilitators should take notes on each enumerators' performances and share their comments at the end of the day. Here is a non-exhaustive list of elements to observe:

  • Complete Equipment (pen, notebook,…)
  • Selecting the right respondent
  • Proper introduction of the survey goal to the respondent and consent
  • Introductory sentences and reading the questions correctly
  • Probing correctly and the right time
  • Familiarity with the survey and confidence
  • Language proficiency
  • Interactions with the respondent (verbally and non-verbally)
  • Objectivity, patience, attention to details
  • Creating a conducive environment (privacy)

Evaluation of participation

Facilitators of the training should observe each participant and take regular notes. Some criteria to evaluate the participation:

  • Punctuality
  • Active participation and initiative
  • Communication skills and local language proficiency
  • Device literacy

Best Practices

Research standards and confidentiality

Scientific approach

The first, and foremost, of the qualities of a good survey team is a commitment to the scientific method. A scientific method is the standard approach for such surveys and in order to produce concrete, defensible and valuable results, this method must be applied. Enumerators must be instructed on what the scientific approach means for them: that they are committed to identifying the true situation on the ground, not one that seems to be real but rather arises from errors in the way we have measured opinions.

The only way that a comparison between organisations is valid is to use the same survey method for all respondents. This means:

  • Introducing the process in the same way in each organization
  • Making people feel equally comfortable that the process is anonymous
  • Making people feel that their responses will be confidential
  • Giving each group roughly the same amount of time to fill in the questionnaire
  • Guiding the discussion session in a similar way
  • Collecting and filing all the questionnaires systematically

If we deviate from this approach, for example by treating one group differently to all the others, we won’t be able to tell if the differences between that group and the others are down to actual differences or just responses to our differential treatments.

To standardise responses to any eventuality that may arise during the interview process, it is advisable to write a ‘Frequently Asked Questions’ (FAQ) document which all enumerators can refer to.

Clearly, there will be times when the situation deviates from a perfect replication of all other interviews and will not correspond to any items on the FAQs. To ensure that this is accounted for, enumerators should be able to discuss aspects of the methodology of the project, so it is advisable to include this in the training.

Confidentiality and anonymity

One of the key selling points of the interview for many respondents will be a commitment to anonymizing all interviews and safeguarding respondents’ confidentiality. The tablet goes a long way in helping enumerators achieve this through the use of de-identified IDs, however enumerators must ensure all interactions with respondents adhere to the strictest degree of confidentiality. This entails:

  • Holding all opinions, claims, and other features that can be associated with individuals (confidential information in trust and confidence)
  • Using such confidential information only for the purposes set out in the training, and for any other purpose, or disclosed to any third party
  • Not to copy or retain any written information or record that could be associated with identifying features of individuals, or identifying features of any sort, outside the survey team’s own collections.
  • At the conclusion of the surveys, or upon demand by the survey team, all confidential information, including questionnaires, written notes, photographs, memoranda or other types of notes taken to be returned to the survey team.
  • Confidential information is not to be disclosed to any employee, consultant or third party unless it has been approved by the survey team.

Interview practice and pilot interviews/ field testing

Before going out in the field it is imperative that all enumerators practice interviewing at least twice, both as a way to familiarize themselves with the questionnaire and to receive feedback on their interviewing (and scoring if applicable) by the trainers and Field Coordinator. It is normal for the first few interviews conducted by each enumerator will be of a lesser quality, so it is important that these can be discarded as ‘practice’ and not used as part of the dataset.

Practice interviews can take the following forms:

  • Team exercises: practicing sections of the questionnaire in pairs or groups
  • Mock interviews: conducting group mock interviews with a trainer and holding discussion sessions afterwards
  • Pilot interviews: interviews with civil servants, not to be included in the dataset. To be conducted in pairs or groups in order to encourage feedback on interviewing and scoring

Enumerator Manual

The Enumerator Manual should form the basis for the enumerator training. It is designed as a resource for field teams to consult as questions or issues arise during data collection. The manual should include the following:

  1. Brief overview of study objectives
  2. All Survey Protocols
  3. Roles and responsibilities of field staff
  4. Definitions of key terms
  5. Instructions for using (and troubleshooting) tablets
  6. Questionnaire conventions
  7. Module-by-module description of Questionnaire Content
  8. 'Frequently Asked Questions' (FAQs)

The Enumerator Manual is typically developed by the Field Coordinator, with input from the research team and the survey firm (or other data collection partner).

Back to Parent

This article is part of the topic Primary Data Collection

Additional Resources

Retrieved from "https://dimewiki.worldbank.org/index.php?title=Enumerator_Training&oldid=7371"