Survey Firm TOR

A survey firm term of reference (TOR) defines the structure of the project and breaks down the responsibilities of all parties involved, including that of the impact evaluation team. The TOR is an important document for survey firm procurement and ongoing monitoring of survey firm performance. This page suggests the scope of work for the survey firm and impact evaluation team, discusses deliverables, and provides an example excerpt from a survey firm TOR.

Read First

• Prepare the TOR early in the firm procurement process.
• Within the TOR, clearly delineate tasks between the survey firm and the impact evaluation team; define deliverables carefully; and specify all expectations and consequences if the expectations are not met.
• Refer to the TOR throughout field work. Any changes to survey protocols or sampling that deviate from the TOR must be confirmed in writing.

Overview

A detailed TOR is essential for smoothly running fieldwork. Prepare the TOR before publishing the Request for Expression of Interest (rEOI) for survey firms; publish it after shortlisting firms based on the expression of interest (EOI). After hiring the survey firm, monitor and provide feedback early and often, using the TOR as an anchor for expectations. Any changes to survey protocols or sampling that deviate from the TOR must be confirmed in writing. If there are cost implications, a contract modification may be required.

Defining the Scope of Work

Survey Firm Tasks

1. Obtain necessary permits or clearance for the survey
2. Translate and field test all questionnaires
3. Create detailed field procedure plan
4. Recruit, train, and contract experienced field staff
5. Manage and oversee household survey data collection

Impact Evaluation Team Tasks

1. Design questionnaire
2. Manage survey pilot
3. Program questionnaire
4. Sample
5. Monitor data quality
6. Clean data
7. Conduct data analysis

The field coordinator should work closely with survey firm every step of the process.

Defining the Deliverables

Review of deliverables is the formal and enforceable way to provide feedback to the survey firm. It is important to assess quality and provide feedback early and often.

Deliverables should be spaced throughout the contract rather than only at the end of the survey. Each major stage of the process should include a deliverable. Deliverables should be specified as a detailed list in the TOR, with intermediate outputs. Finally, deliverables should have clearly indicated quality standards. Consequences should be clearly outlined in the case that the firm does not meet standards.

TOR Example Excerpt

Activity 5 : Household survey data collection

Train all enumerators, supervisors, and data manager on the administration of questionnaires provided by the research team. The training should also serve as a screening process for skilled interviewers. Consequently, the survey company must recruit more interviewers for the training than will be ultimately hired for the project. Five enumerators should be included in the training as a reserve. The following components must be included in the training:

1. Theoretical: training should include a review the theory of the questionnaire and each question in order for trainees to fully understand the objective of each question. Standard quantitative interviewing techniques and field protocols should also be covered.
2. Classroom practice: training should include individual and group exercises for trainees to become familiar with the practice of asking questions and filling questionnaires. This part of the training may include in-class demonstrations, where the questionnaire is projected and one interviewer completes the questionnaire in front of the classroom. The training may also use vignettes, where the company designs case scenarios based on typical households (perhaps those found during the supervisor training or piloting) and have interviewers complete the questionnaire based on the vignette. Finally, the trainees should conduct pilot interviews on the same subject, and have the interviewers complete a questionnaire for the interview to test consistency across interviewers.
3. Field practice: after the theoretical and classroom practice, the interviewers should go to the field to administer the full questionnaire to a small number of households (outside the study sample). The pre-test should not focus on major adjustments to the questionnaire, but rather simulate the administration of the questionnaire under normal circumstances. All field team members must demonstrate they clearly understand their roles and are correctly following the survey protocols.
4. Evaluation: following the training, interviewers, supervisors and data managers should be evaluated based on their understanding of the questionnaire and their ability to correctly record data.

The training period will conclude only when the field teams have demonstrated mastery of the designated tasks. Decisions as to which field staff will take part in the data collection must be made on the basis of this evaluation.

Back to Parent

This article is part of the topic Primary Data Collection

Additional Resources

• DIME Analytics' Survey Firm Technical Review Scoring Matrix
• DIME Analytics' Working with Survey Firms