# Difference between revisions of "List Experiments"

A technique to get around Social Desirability Bias, typically used when trying to measure Sensitive Topics

## What is a list experiment?

List experiments aggregate responses to sensitive questions with responses to non-sensitive questions, e.g. how many of the following statements do you agree with? This provides the respondent with an additional level of privacy, as the researcher can never perfectly infer an individual’s answer to the sensitive item (unless either 0 or N+1 items are true).

Procedure: Randomly divide the sample into two groups

• Direct response: report how many of N items are true for themselves, where items are neutral and non-sensitive
• Veiled response: report how many of N+1 items are true, with N items being identical to control group’s items, and the N+1st item being a sensitive item

With a large enough sample, estimate the population mean for the N+1st item (sensitive item), by differencing out the mean of the sum of N items estimated from the control. In other words, subtracting the average response in control group from the average response in the treatment group yields the proportion of people who say 'yes' to the sensitive statements.

## Guidelines

### Examples of list experiments

From Coffman, K. B., Coffman, L. C., & Ericson, K. M. M. (2016). The size of the lgbt population and the magnitude of antigay sentiment are substantially underestimated. Management Science.

How many of the following statements are true for you?

Group A (direct response) gets:

1. I remember where I was the day of the Challenger space shuttle disaster
2. I spent a lot of time playing video games as a kid
3. I would vote to legalize marijuana if there was a ballot question in my state
4. I have voted for a political candidate who was pro-life

Group B (veiled response) gets: The same list as Group A, plus

5. I consider myself to be heterosexual

From Gilens, M., Sniderman, P. M., and Kuklinski, J. H. (1998). Affirmative action and the politics of realignment. British Journal of Political Science 28, 1, 159– 183.

"Now I am going to read you three things that sometimes make people angry or upset. After I read all three, just tell me HOW MANY of them upset you. I don’t want to know which ones, just HOW MANY."

Group A gets the list:

1. The federal government increasing the tax on gasoline;
2. Professional athletes earning large salaries;
3. Requiring seat belts be used when driving;
4. Large corporations polluting the environment

Group B receives Group A's list, plus:

Group C receives Group A's list, plus:

5. Awarding college scholarships on the basis of race

### Issues with List Experiments

• Require people to count/add, possibly introducing noise to the data (especially if the list is long)
• Unless the “innocent” questions are completely unrelated and have a known distribution, there is a chance that the treatment in your RCT might have an effect on its distribution.
• Moreover, designing your common questions that way makes your sensitive ones stand out even more.
• Reduces power
• However, as this is individually randomized, may not have to allocate half of your sample to direct response (instead could do 5-10%)