Difference between revisions of "Field Surveys"

Jump to: navigation, search
Line 1: Line 1:
<onlyinclude>
<onlyinclude>
'''Field surveys''' (or survey instruments/questionnaires) are one of the most commonly used methods used by researchers for the process of [[Primary Data Collection|primary data collection]]. When [[Secondary Data Sources|secondary sources of data]] are either insufficient, field surveys allow researchers to monitor and evaluate the impact of field experiments. For example, consider a study that aims to evaluate the impact of micro-loans on farm output in a small village. It is possible that data on farm output for the last 10 years is not available, or is insufficient. In this case, researchers can conduct a survey among local farmers to [[Primary Data Collection|collect data]] on farmer incomes and farm outputs.
'''Field surveys''' (or survey instruments/questionnaires) are one of the most commonly used methods used by researchers for the process of [[Primary Data Collection|primary data collection]]. In cases where [[Secondary Data Sources|secondary sources of data]] do not provide sufficient information, field surveys allow researchers to better monitor and evaluate the impact of field experiments. For example, consider a study that aims to evaluate the impact of micro-loans on farm output in a small village. It is possible that data on farm output for the last 10 years is not available, or is insufficient. In this case, researchers can conduct a survey among local farmers to [[Primary Data Collection|collect data]] on farmer incomes and farm outputs.
</onlyinclude>
</onlyinclude>
== Read First ==
== Read First ==
*'''Primary data''' has become the dominant source for empirical inquiry in development economics.  
*'''Primary data''' is vital for conducting empirical inquiry in the field of development economics.  
*With increasing access to specialized ODK-based software like CAPI, [[Survey Firm|survey firms]], and standardized [[Field Management|field management practices]], there are plenty of useful guidelines on streamlining the process of conducting surveys.
*With increasing access to specialized ODK-based software like [[Computer-Assisted Personal Interviews (CAPI)|computer-assisted personal interviews (CAPI)]], [[Survey Firm|survey firms]], and standardized [[Field Management|field management practices]], it is important for researchers to follow certain best practices to gather data.
*Surveys involve multiple stages from start to finish, like drafting, piloting, programming, with a clear [[Timeline of Survey Pilot|timeline]] for each step.
Surveys can be conducted either directly by the research team or indirectly by [[Procuring a Survey Firm|procuring a survey firm]].
*Surveys can either be conducted directly by the research team or by [[Procuring a Survey Firm|procuring a survey firm]].


== Stages of a Survey ==
== Preparing a Survey ==
The process of preparing surveys involves multiple stages, like drafting, piloting, programming, and translating, and a clear pre-decided [[Timeline of Survey Pilot|timeline]].


=== Draft and pre-pilot ===
=== Pre-pilot and draft ===
The '''draft and pre-pilot stage''' of implementing a survey involves defining rules and guidelines in the form of [[Survey Protocols|survey protocols]]. Clear protocols ensure that fieldwork is carried out consistently across teams and regions, and are important for [[Reproducible Research |reproducible research]].
The '''pre-pilot and draft stage''' of implementing a survey starts by defining rules and guidelines in the form of [[Survey Protocols|survey protocols]]. Clear protocols ensure that fieldwork is carried out consistently across teams and regions, and are important for [[Reproducible Research |reproducible research]].


Then a '''pre-pilot''' is conducted, which is the first component of [[Survey Pilot|piloting a survey]]. This involves answering '''qualitative''' questions about the following:
Then a '''pre-pilot''' is conducted, which is the first component of [[Survey Pilot|piloting a survey]]. This involves answering '''qualitative''' questions about the following:
Line 18: Line 18:
*Number of revisits.
*Number of revisits.
*Dropping and replacement criteria.
*Dropping and replacement criteria.
Based on the pre-pilot, the research team [[Questionnaire Design|designs]] and drafts a questionnaire. In this process, it helps to use existing studies and questionnaires as a resource, to avoid starting from scratch.
Based on the pre-pilot, the research team [[Questionnaire Design|designs]] a questionnaire to generate a first '''draft'''. For this purpose, avoid starting from scratch, and try using existing studies and questionnaires as a point of reference.


=== Content-focused pilot ===
=== Content-focused pilot ===
The next step is to conduct a [[Piloting Survey Content|content-focused pilot]]. This stage involves answering questions about the '''structure''' and '''content''' of the questionnaire. Global best practices recommend conducting the '''content-focused pilot''' on paper, to make it easier to revise and refine the survey instrument.
The next step is to conduct a [[Piloting Survey Content|content-focused pilot]]. This stage involves answering questions about the '''structure''' and '''content''' of the questionnaire. Global best practices recommend conducting the content-focused pilot on paper, to make it easier to revise and refine the survey instrument.


In this stage, it is equally important to [[Piloting Survey Protocols|test survey protocols]] like scheduling, survey infrastructure, and sampling methods.
It is equally important to simultaneously [[Piloting Survey Protocols|pilot survey protocols]] like scheduling, testing survey infrastructure, and sampling methods.
DIME Analytics has created the following checklists for this process:
DIME Analytics has created the following checklists for this purpose:
* [[Preparing_for_the_survey_checklist|Checklist: Preparing for a Survey Pilot]]
* [[Preparing_for_the_survey_checklist|Checklist: Preparing for a Survey Pilot]]
* [[Checklist: Refine the Questionnaire (Content)|Checklist: Refining questionnaire content]]
* [[Checklist: Refine the Questionnaire (Content)|Checklist: Refining questionnaire content]]
Line 31: Line 31:
Once the questionnaire content and design have been finalized, the next step is to [[Questionnaire Programming|program the survey instrument]]. Researchers should not program the instrument before finalizing the design of the questionnaire, otherwise they will waste crucial time and resources in going back and forth.  
Once the questionnaire content and design have been finalized, the next step is to [[Questionnaire Programming|program the survey instrument]]. Researchers should not program the instrument before finalizing the design of the questionnaire, otherwise they will waste crucial time and resources in going back and forth.  


While there are various tools to do this, CAPI is the most widely used. Researchers must set aside 2-3 weeks for '''programming''', and another 2-3 weeks for '''debugging and testing'''.
While there are various tools to do this, [[Computer-Assisted Personal Interview (CAPI)|computer-assisted personal interviews (CAPI) are the most widely used. Researchers must set aside 2-3 weeks for '''programming''', and another 2-3 weeks for '''testing and debugging'''.


=== Data-focused pilot ===
=== Data-focused pilot ===
Before moving forward, the research team must [[Procuring a Survey Firm|finalize a survey firm]]. Then the next step is to conduct a '''data-focused pilot'''. This step tests the following:
Before conducting a '''data-focused pilot''', the research team must sign a contract with a survey firm. The data-focused pilot tests the following:
*Survey design and interview flow (re-check design and revisions made earlier)
*Survey design and interview flow (re-check design and revisions made earlier)
*Survey programming (check if questions display correctly, and built-in data checks are working)  
*Survey programming (check if questions display correctly, and built-in data checks are working)  

Revision as of 19:19, 25 March 2020

Field surveys (or survey instruments/questionnaires) are one of the most commonly used methods used by researchers for the process of primary data collection. In cases where secondary sources of data do not provide sufficient information, field surveys allow researchers to better monitor and evaluate the impact of field experiments. For example, consider a study that aims to evaluate the impact of micro-loans on farm output in a small village. It is possible that data on farm output for the last 10 years is not available, or is insufficient. In this case, researchers can conduct a survey among local farmers to collect data on farmer incomes and farm outputs.

Read First

Surveys can be conducted either directly by the research team or indirectly by procuring a survey firm.

Preparing a Survey

The process of preparing surveys involves multiple stages, like drafting, piloting, programming, and translating, and a clear pre-decided timeline.

Pre-pilot and draft

The pre-pilot and draft stage of implementing a survey starts by defining rules and guidelines in the form of survey protocols. Clear protocols ensure that fieldwork is carried out consistently across teams and regions, and are important for reproducible research.

Then a pre-pilot is conducted, which is the first component of piloting a survey. This involves answering qualitative questions about the following:

  • Selection of respondents.
  • Tracking mechanism.
  • Number of revisits.
  • Dropping and replacement criteria.

Based on the pre-pilot, the research team designs a questionnaire to generate a first draft. For this purpose, avoid starting from scratch, and try using existing studies and questionnaires as a point of reference.

Content-focused pilot

The next step is to conduct a content-focused pilot. This stage involves answering questions about the structure and content of the questionnaire. Global best practices recommend conducting the content-focused pilot on paper, to make it easier to revise and refine the survey instrument.

It is equally important to simultaneously pilot survey protocols like scheduling, testing survey infrastructure, and sampling methods. DIME Analytics has created the following checklists for this purpose:

Program instrument

Once the questionnaire content and design have been finalized, the next step is to program the survey instrument. Researchers should not program the instrument before finalizing the design of the questionnaire, otherwise they will waste crucial time and resources in going back and forth.

While there are various tools to do this, [[Computer-Assisted Personal Interview (CAPI)|computer-assisted personal interviews (CAPI) are the most widely used. Researchers must set aside 2-3 weeks for programming, and another 2-3 weeks for testing and debugging.

Data-focused pilot

Before conducting a data-focused pilot, the research team must sign a contract with a survey firm. The data-focused pilot tests the following:

  • Survey design and interview flow (re-check design and revisions made earlier)
  • Survey programming (check if questions display correctly, and built-in data checks are working)
  • Data (whether all variables appear or not, missing data, variance in data)
  • High frequency checks

Also refer to the DIME Analytics Checklist for refining data.

Translate

The next step in the process is to translate the questionnaire. Translation will be considered good or complete only when enumerators and respondents have the same understanding for each question. Often, with repeated translations, the research team must ensure adequate version control norms.

Train enumerators

These steps must be completed before training enumerators. This helps to minimize enumerator effects, which arise due to differences in the way a question is asked to each respondent because different enumerators may share a different translation of the same question.

Challenges

The process of implementing field surveys comes with its own set of challenges. These include:

  • Measurement challenges. Sometimes some questions are sensitive, or aim to measure things that seem hard to quantify. For instance, while employee-satisfaction is very important for various studies to assess labor-market conditions, it is also very hard to measure objectively.
  • Data-quality assurance- It is very important to have a plan for ensuring data quality.
  1. Translation errors - If translation is poor or incomplete, the data that is collected might be incorrect, or insufficient.
  2. Gaps in enumerator training - This can also lead to improper data collection, and can therefore hamper the results of an evaluation.

Back to Parent

This page redirects from primary data collection.

Additional Resources