Difference between revisions of "Measuring Difficult Topics"
| (5 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
This article talks about [[Questionnaire Design|designing your questionnaire]] to measure outcomes that are hard for respondents to remember or estimate. | |||
== Read First == | == Read First == | ||
Best practice is to rely on objective indicators as much as possible. For example, rather than asking a respondent the size of her agricultural plot, it is better to measure the plot area directly using GPS devices. However, objective measures are often more expensive, and may not always be possible. | Best practice is to rely on objective indicators as much as possible. For example, rather than asking a respondent the size of her agricultural plot, it is better to measure the plot area directly using GPS devices. However, objective measures are often more expensive, and may not always be possible. You can find below recommendations and strategies for how to best capture difficult-to-know topics in a [[Questionnaire Design|questionnaire]]. | ||
== Guidelines == | == Guidelines == | ||
=== Internal Consistency Checks === | === Internal Consistency Checks === | ||
How much did you spend in the last week on X? How much did you spend in the last 4 weeks on X? | Internal consistency assesses the correlation between multiple items in a test that are intended to measure the same construct. For example. [[Enumerator Training|enumerators]] can ask ''How much did you spend in the last week on X? How much did you spend in the last 4 weeks on X?'' | ||
=== Multiple Measurements === | === Multiple Measurements === | ||
How many minutes does it take to walk to the grocery store? How many miles away is the grocery store? | [[Enumerator Training|Enumerators]] can ask related questions to measure the object of interest in several ways to ensure that there is internal consistency in the respondent's answers. | ||
For example, consider the questions ''How many minutes does it take to walk to the grocery store? How many miles away is the grocery store?'' The '''enumerator''' is asking for two different measurements, the second question acting as a check on the first: suppose the respondent says it takes them 15-20 minutes to walk to the store. If they answer the follow-up question by saying the store is about one mile away, then taking multiple measurements will have shown the internal consistency of their answers. | |||
=== Contextual References === | === Contextual References === | ||
In some contexts, it may be hard for respondents to tell you their age, or the year of important life events (e.g. marriage, sexual debut). Providing enumerators with a calendar of significant national events, such as elections (or other change of leadership), political independence, national holidays, notable | In some contexts, it may be hard for respondents to tell you their age, or the year of important life events (e.g. marriage, sexual debut). Providing [[Enumerator Training|enumerators]] with a calendar of significant national events, such as elections (or other change of leadership), political independence, national holidays, notable meteorological, or natural history events (floods, droughts, earthquakes, etc.), can be useful in prompting people to estimate. | ||
== Back to Parent == | == Back to Parent == | ||
This article is part of the topic [[Questionnaire Design]] | This article is part of the topic [[Questionnaire Design]] | ||
== Additional Resources == | == Additional Resources == | ||
*DIME Analytics’ guidelines on [https://github.com/worldbank/DIME-Resources/blob/master/survey-instruments.pdf survey design and pilot] | |||
[[Category: Questionnaire Design]] | [[Category: Questionnaire Design]] | ||
Latest revision as of 16:15, 7 August 2023
This article talks about designing your questionnaire to measure outcomes that are hard for respondents to remember or estimate.
Read First
Best practice is to rely on objective indicators as much as possible. For example, rather than asking a respondent the size of her agricultural plot, it is better to measure the plot area directly using GPS devices. However, objective measures are often more expensive, and may not always be possible. You can find below recommendations and strategies for how to best capture difficult-to-know topics in a questionnaire.
Guidelines
Internal Consistency Checks
Internal consistency assesses the correlation between multiple items in a test that are intended to measure the same construct. For example. enumerators can ask How much did you spend in the last week on X? How much did you spend in the last 4 weeks on X?
Multiple Measurements
Enumerators can ask related questions to measure the object of interest in several ways to ensure that there is internal consistency in the respondent's answers.
For example, consider the questions How many minutes does it take to walk to the grocery store? How many miles away is the grocery store? The enumerator is asking for two different measurements, the second question acting as a check on the first: suppose the respondent says it takes them 15-20 minutes to walk to the store. If they answer the follow-up question by saying the store is about one mile away, then taking multiple measurements will have shown the internal consistency of their answers.
Contextual References
In some contexts, it may be hard for respondents to tell you their age, or the year of important life events (e.g. marriage, sexual debut). Providing enumerators with a calendar of significant national events, such as elections (or other change of leadership), political independence, national holidays, notable meteorological, or natural history events (floods, droughts, earthquakes, etc.), can be useful in prompting people to estimate.
Back to Parent
This article is part of the topic Questionnaire Design
Additional Resources
- DIME Analytics’ guidelines on survey design and pilot