Difference between revisions of "Recall Bias"

Jump to: navigation, search
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Error caused by inaccurate or incomplete recollection of events. A particular concern for retrospective survey questions.  
<onlyinclude>Recall bias is bias caused by inaccurate or incomplete recollection of events by the respondent. It is a particular concern for retrospective survey questions. </onlyinclude>


== Read First ==
== Read First ==
* Lower salience and longer recall periods increase forgetfulness [http://blogs.worldbank.org/impactevaluations/decomposing-response-error-improve-consumption-survey-design]
* Research shows that lower salience and longer recall periods increase forgetfulness [http://blogs.worldbank.org/impactevaluations/decomposing-response-error-improve-consumption-survey-design] [http://www.sciencedirect.com/science/article/pii/S0304387811000939]
* How long is "too long" for a recall period for a survey question? It depends on the type of event: infrequent events (e.g. purchases of major assets) will be memorable for longer periods of time than routine events (e.g. use of public transportation).  


== Guidelines ==
== Guidelines ==


===How to avoid?===
=== How long is "too long" for recall? ===
It depends on the type of event respondents are being asked to recall. Research shows strong evidence of recall bias in food consumption, but little evidence for agricultural production.
As a rule of thumb, infrequent events (e.g. purchases of major assets) will be memorable for longer periods of time than routine events (e.g. use of public transportation).
 
===How to avoid recall bias?===
Useful strategies:
Useful strategies:
# Where possible, use methods to reduce recall periods for key indicators. For example, more frequent follow-up surveys by phone, or personal diaries
# Reduce recall periods as much as possible. For example, add follow-up surveys by phone, or personal diaries.
# When [[Survey Pilot|Piloting your Survey]], carefully test different recall periods; if possible try shorter and longer periods and check for differences in variance
# Conduct focus groups to understand salience of the indicator in question, and gauge a reasonable recall period.
# When [[Survey Pilot|Piloting your Survey]], carefully test recall periods; if possible try shorter and longer periods and check for differences in variance


===Subsection 2===
===Subsection 3===


== Back to Parent ==
== Back to Parent ==
Line 20: Line 22:


== Additional Resources ==
== Additional Resources ==
* Development Impact Blogpost on [http://blogs.worldbank.org/impactevaluations/decomposing-response-error-improve-consumption-survey-design Response Error in Consumption Surveys] and the related [http://documents.worldbank.org/curated/en/122481467999693721/pdf/WPS7646.pdf paper]
* DIME Analytics (World Bank), [https://osf.io/rqb5m/ Survey Instrument Design and Pilot]
* Blog from Financial Access on [http://www.financialaccess.org/blog/2015/7/30/reliability-of-self-reported-data-recall-bias The Reliability of Self-reported Data]
* IFPRI, [https://www.ifpri.org/blog/do-you-remember-measuring-anchoring-bias-recall-data Measuring anchoring bias in recall data]
* Jishnu Das, Jeffrey Hammer, Carolina Sánchez-Paramo, [http://www.sciencedirect.com/science/article/pii/S0304387811000708 The impact of recall periods on reported morbidity and health seeking behavior], In Journal of Development Economics, Volume 98, Issue 1, 2012, Pages 76-88, ISSN 0304-3878
* Jed Friedman (World Bank), [http://blogs.worldbank.org/impactevaluations/decomposing-response-error-improve-consumption-survey-design Response Error in Consumption Surveys], and the [https://documents1.worldbank.org/curated/en/122481467999693721/pdf/WPS7646.pdf related paper]
** Abstract: Between 2000 and 2002, we followed 1621 individuals in Delhi, India using a combination of weekly and monthly-recall health questionnaires. In 2008, we augmented these data with another 8 weeks of surveys during which households were experimentally allocated to surveys with different recall periods in the second half of the survey. We show that the length of the recall period had a large impact on reported morbidity, doctor visits; time spent sick; whether at least one day of work/school was lost due to sickness and; the reported use of self-medication. The effects are more pronounced among the poor than the rich. In one example, differential recall effects across income groups reverse the sign of the gradient between doctor visits and per-capita expenditures such that the poor use health care providers more than the rich in the weekly recall surveys but less in monthly recall surveys. We hypothesize that illnesses – especially among the poor – are no longer perceived as “extraordinary events” but have become part of “normal” life. We discuss the implications of these results for health survey methodology, and the economic interpretation of sickness in poor populations.
* Financial Access Initiative, [http://www.financialaccess.org/blog/2015/7/30/reliability-of-self-reported-data-recall-bias The Reliability of Self-reported Data]
 
* Jishnu Das, Jeffrey Hammer, Carolina Sánchez-Paramo, [http://www.sciencedirect.com/science/article/pii/S0304387811000708 The impact of recall periods on reported morbidity and health seeking behavior]
* Kathleen Beegle, Calogero Carletto, Kristen Himelein, [http://www.sciencedirect.com/science/article/pii/S0304387811000939 Reliability of recall in agricultural data]
* Philip Wollburg, Marco Tiberti and Alberto Zezza, [https://www.sciencedirect.com/science/article/pii/S0306919220302098?fbclid=IwAR1at8ueH2h4j3mHXlGvcIGEX4wgoxTgN6IdmxGejJJsz3DJkmra2bn6jas Recall length and measurement error in agricultural surveys]
[[Category: Questionnaire Design]]
[[Category: Questionnaire Design]]

Latest revision as of 18:56, 19 October 2021

Recall bias is bias caused by inaccurate or incomplete recollection of events by the respondent. It is a particular concern for retrospective survey questions.

Read First

  • Research shows that lower salience and longer recall periods increase forgetfulness [1] [2]

Guidelines

How long is "too long" for recall?

It depends on the type of event respondents are being asked to recall. Research shows strong evidence of recall bias in food consumption, but little evidence for agricultural production. As a rule of thumb, infrequent events (e.g. purchases of major assets) will be memorable for longer periods of time than routine events (e.g. use of public transportation).

How to avoid recall bias?

Useful strategies:

  1. Reduce recall periods as much as possible. For example, add follow-up surveys by phone, or personal diaries.
  2. Conduct focus groups to understand salience of the indicator in question, and gauge a reasonable recall period.
  3. When Piloting your Survey, carefully test recall periods; if possible try shorter and longer periods and check for differences in variance


Back to Parent

This article is part of the topic Questionnaire Design


Additional Resources