Randomization in SurveyCTO

During surveys, you might often need to randomize various aspects of the questionnaire. While SurveyCTO has a random number generator, is is usually not recommended that you use it. This article will argue for doing the randomization in Stata, R or similar software, before the start of the survey, and preload the results of the randomization as dummies or categorical variables.

Why randomization is better to do before the Survye

During surveys, we often need to randomize various aspects of the questionnaire. For example – sometimes we need to randomize which household members to interview, and sometimes - which set of questions to ask. While most CAPI software have random number generators, it is not the preferred option. Using, for example, Stata to randomize and then preloading the generated data file into the survey software is in almost all cases the better option among the two. The main advantages of using Stata over CAPI software during randomization are as follows:

• Randomization in Stata is transparent and reproducible which is important for publishing research.
• Randomization results in Stata can be dependent, so that we are guaranteed that no disproportional large share of the results falls into any group. Randomization is always independent in SurveyCTO which means that no groups could be assigned observations if the number of observation per groups is low.
• Randomization in Stata provides the option of ensuring that the result of the randomization is balanced over other variables, i.e. stratas. This means that we can guarantee that, for example, not all female respondents end up in a certain group.
• Randomization in Stata is done before the survey takes place. This provides an opportunity to double check the result of a randomization and fix bugs and typos in the randomization code before it is used in the field, as it then would be too late to fix.

Practical guide to how to randomize in Stata for a survey

This is a basic examplefor how to do this. See Randomization in Stata more details for how to implement more a

• Use with a dataset which has a unique ID [respondent ID, household number, etc.]. • While writing a do-file, pay close attention to the following things: o Set version. This ensures that the randomization algorithm is the same, as it sometimes changes between Stata versions. o Set seed. This makes sure that the same random number is generated for the first observation, for the second observation, and so on, for every time the code is run. o Properly sorting the data. The data should be sorted such that observations are in the same order every time the code is run. The most optimal situation is sorting using an ID variable which uniquely and fully identifies each observation. • Convert the random numbers into categorical variables or dummy variables. This helps you check if the data is balanced. • The end goal is to have a CSV format file containing the ID variable used for randomization and the categorical variables created from the random numbers generated. This dataset will be preloaded into SurveyCTO so that after an enumerator enters the respondent ID at the start of a survey questionnaire the result of the randomization will be loaded for the form and can be used for various sections of the survey.

A researcher randomizing through Stata first creates a do file to document their method of randomizing. This allows the researcher to replicate the results in the future. After that, they randomize in Stata and make sure that the randomization takes into account various demographic characteristics like age, sex, income, etc, so that the survey is well balanced. They then create the data _le and check for bugs and irregularities in the randomization. Finally, they load the dataset into the survey software.

Randomizing using the CAPI software does not provide the researcher with the opportunity to replicate results in the future, test for bugs or ensure balance in gender and age distribution because the randomization takes place during the field survey itself.

The next section provides information on effectively using Stata for randomization.

If you need to randomize certain things in your questionnaire, randomization using SurveyCTO is not recommended. The best way to randomize things in a questionnaire is by using Stata. The reason that Stata is preferable is because the randomization in Stata is transparent and easily reproducible which is not only necessary for impact evaluation projects but also for publishing research. Randomization in Stata is also done before a survey so there is time to check for errors/bugs and also to ensure that there is balance in the dataset. When Stata is not available, Excel can also be used but randomization done in Excel is not as reproducible as randomization in Stata.

Back to Parent

This article is part of the topic Randomized Control Trials