Measuring Difficult Topics
This article talks about designing your questionnaire to measure outcomes that are hard for respondents to remember or estimate.
Read First
Best practice is to rely on objective indicators as much as possible. For example, rather than asking a respondent the size of her agricultural plot, it is better to measure the plot area directly using GPS devices. However, objective measures are often more expensive, and may not always be possible. You can find below recommendations and strategies for how to best capture difficult-to-know topics in a questionnaire.
Guidelines
Internal Consistency Checks
Internal consistency assesses the correlation between multiple items in a test that are intended to measure the same construct. For example. enumerators can ask How much did you spend in the last week on X? How much did you spend in the last 4 weeks on X?
Multiple Measurements
Enumerators can ask related questions to measure the object of interest in several ways to ensure that there is internal consistency in the respondent's answers.
For example, consider the questions How many minutes does it take to walk to the grocery store? How many miles away is the grocery store? The enumerator is asking for two different measurements, the second question acting as a check on the first: suppose the respondent says it takes them 15-20 minutes to walk to the store. If they answer the follow-up question by saying the store is about one mile away, then taking multiple measurements will have shown the internal consistency of their answers.
Contextual References
In some contexts, it may be hard for respondents to tell you their age, or the year of important life events (e.g. marriage, sexual debut). Providing enumerators with a calendar of significant national events, such as elections (or other change of leadership), political independence, national holidays, notable meteorological, or natural history events (floods, droughts, earthquakes, etc.), can be useful in prompting people to estimate.
Back to Parent
This article is part of the topic Questionnaire Design
Additional Resources
- DIME Analytics’ guidelines on survey design and pilot