ID Variable Properties

An ID variable that identifies an observation should have the properties listed below. Note that this relates to the ID variable that identifies observations across data sets in out project folder. Some commands in Stata, for example reclink requires a masterid() and an userid() and these ID variables created temporarily for that command does not have to have all of these properties.

Read First

The first and the second properties are the properties that everyone should always test for before working with a new data set. The third, fourth and fifth property is more relevant when creating an ID variable or when assigning new values to newly encountered observations.

First property: Uniquely Identifying

The first and the second properties are the most commonly referred to property of an ID variable. An ID variable is uniquely identifying when no two observation share a value in the ID variable. Next paragraph shows that this is easy to test for a single data set, however, the first property does not only apply to a single data set, it applies to the full project. To test the first property for a full project one must first make sure that all observations are added to the master data set, and then test for the first property on the master data set as described in the next paragraph.

There are several ways to test for this in Stata. For example duplicates report idvar where idvar is the ID variable. It is also possible to test the first property using this command isid idvar. While duplicates report provides a more informative output, isid is a quick and easy way to test for both the first and the second property.

Second property: Fully Identifying

An ID variable is fully identifying when all observations have a value in the ID variable. This property is, similarly to the first property, very easy to test on a single data set, and depends on how well the master data set has been kept up to data in order to test for a full project. If all observations in all data set has been added to the master data set, then they should all been given an value in the ID variable, but each time you modify the master data set you should test for this property to be sure.

There are several ways to test for this in Stata but the command isid idvar where idvar is the ID variable is often used as it tests for both the first and the second property. Note that missing values should not be used as an ID value even though a missing value technically could be used to identify a single observation. Missing values implies that the information is missing so the command isid in Stata treats a missing value as if the ID variable is not fully identifying the data set.

Third property: Constant Across a Project

The third property says that no observation should have different IDs in different data set. Data set's collected from different sources might have different IDs when they are first included in project, but one ID variable should be made the dominant one, and it should be clearly marked that the other ID variable is not the ID variable for this project if it is at all kept in the data set.

There is no specific test for this, but this is a rule to follow when created an ID variable. If the best practice of making sure that all observations are always added to the master data set is followed, then that should usually makes sure that no observation has two values in the ID variable, and it also easy to keep just one ID variable after the observations are added to the master data set.

Fourth property: Constant Throughout the Duration of a Project

The fourth property is similar to the third property but it says that the same observation should have the same value in the ID variable throughout the project. The ID that an observation was assigned at baseline (or whenever it were assigned) should be kept the same throughout the rest of the project. One exception to this rule is obviously when we find a mistake in the ID variable. This hopefully happens rarely as it is very labor demanding to go over all do-files in a project in order to make sure that no values have to be updated for the code to work as intended.

Another example is if the format of the ID variable needs to be extended in case a project runs out of IDs. In case this happens it could be a good idea to create a new ID variable where the new value is based on the old value. For example, the new variable have two more digits or similarly. Then the old ID variable can be kept so that old code does not have to be updated. Although it is best practice to update all references to the old ID variable with the new one, but this can be unfeasible due to taking too much time.

Fifth property: Anonymous IDs

The fifth property is less a requirement and more a good practice. Sometimes we have access to IDs that satisfy all the properties above, but we should be very careful before using them. Examples of such cases could be individual national IDs, public company IDs, a hospital's patient ID etc. Since records over those IDs are available to people outside our team, there is no way for us to guarantee that we can protect the privacy of the data we collect. In all of these cases we need to create our own ID that has no association with the ID variable created by someone else and is unique to our project and thereby be an anonymous ID that only identifies the observation to us. In the master data set we can include the other ID to enable us to merge data quickly, but then the information in the master data set becomes even more sensitive then usual.

There is an exception to this rule that can simplify the data work but should only be used with care. If a project has a high-level unit of observation for which the project team is absolutely certain that it will not collect sensitive data, and there is an official code for it, then we could perhaps use this code. It could for example be done for districts or region so that we can easier include publicly available data from those district or region. However, if there is any probability that we would include any data not publicly available, for example district budgets etc., then we need to make our own code. Also, if we have a unit of observation for which we have a single instance in which we have few observations of another level, for example a school with few students or a village with a few households, then we have to create an anonymous IDs for all instances at that level. Not just that one school or village, but all schools or villages.

It is never incorrect to create an anonymous ID, so if there is any uncertainty whether a public ID can be used, then always go for the anonymous option.

Back to Parent

This article is part of the topic Data Management

Additional Resources

• list here other articles related to this topic, with a brief description and link