Research Ethics

Impact evaluation teams (or research teams) often work with, or have access to, datasets that contain sensitive or personal identifiable information (PII) on individuals. They can either have direct access (for instance, through a field survey), or indirect access to this data (for instance, in the form of call data records (CDR)). In both cases, it is important to ensure that research teams act ethically. Broadly, ethical research means that research teams must obtain necessary approvals, protect human subjects, obtain informed consent, and ensure confidentiality. Research ethics ensure that that steps taken to reach the outcomes of a study are just as important as the outcomes themselves, and help improve the validity of results of a study.

Read First

• Research ethics are the first pillar of the DIME Research Standards compiled by DIME Analytics.
• The DIME Research Ethics Guidelines discuss key guidelines for researchers affiliated with DIME.
• A human subject is a living individual about whom the research team obtains personal identifiable information (PII), either directly or indirectly.
• The idea of research ethics has the following aspects - ethics approvals, protecting human subjects, informed consent, and confidentiality.
• Each of these components is important to ensure that research teams stick to certain ethical standards while conducting research.
• Research ethics, along with other pillars of research - transparency, reproducibility, data security, and data publication - ensures greater validity for the results of a research study.

Protecting Human Subjects

Members of the research team must ensure that they protect human research subjects and their rights, including the right to privacy. In this context, all living individuals whose sensitive or personally identifiable information (PII) is contained in the datasets being used in the study are considered human subjects. Therefore, ethical research requires that all members of the research team who handle personally identifiable information (PII) must have up-to-date human research subjects certification. Two organizations that offer courses to certify research team members are:

• Protecting Human Research Participants (PHRP)
• Collaborative Institutional Training Initiative (CITI)

Note: It does not matter whether the research team has direct, or indirect access to the data. If the dataset being used in the study contains personally identifiable information (PII) about individuals, then research team must protect the right to privacy of every such individual.

Ethics Approvals

Institutional review boards (IRBs) are organizations that review and monitor research studies to protect the rights of human subjects. The research team must obtain IRB approvals for studies that use personally identifiable information. This is a powerful tool to promote ethical research because IRBs can deny research teams the right to use data if they do not follow proper guidelines while handling the data.

Note: Keep the following points in mind with respect to ethics approvals:

• It does not matter whether the research team collected the data directly, or indirectly. For example, consider a study that uses mobile phone location data to assess effectiveness of stay-at-home orders during COVID-19. In this case, the research team does not directly collect this data, but gets access to it through the telecom service provider. However, since this dataset contains sensitive information like names, phone numbers, and location of individuals, the research team must obtain an IRB approval.

• In addition to IRB approvals, the research team should also obtain approvals from local institutions in the location of the study. This will ensure that the study complies with local regulations, and does not violate any laws in that area, particular with respect to the right to privacy.

Informed Consent

Before involving any individual in a research study, the research team must obtain informed consent from each individual. This means that the research team must clearly mention all possible risks and benefits from participating in a study, either for the survey pilot or as a respondent in the actual survey.

Note: Keep the following points in mind regarding informed consent:

• The human subjects must be able to refuse participation in a study at any point during the study.

• DIME Analytics has created templates for both written and oral consent.

• The mechanism of informed consent applies only to cases where the research team itself is collecting data, such as through a field survey, because in such cases the research team is directly interacting with the participants.

• However, in some cases, there is no opportunity for the research team to obtain informed consent. For example, consider cases where the research team only has indirect access to data, such as through call data records (CDR), or administrative data. This does not mean that the right to privacy of individuals is not important in such a scenario. It simply means that the research team must use other mechanisms, such as ensuring confidentiality of sensitive information, which is discussed in the next section.

Confidentiality

The research team must also ensure complete anonymity and confidentiality. This means that the identity of study participants should remains hidden, and sensitive information of individuals should never be shared with anyone outside the research team. Research team members should be held personally liable for any actions that result in disclosure of sensitive and personal data.

Note: Keep the following points in mind regarding confidentiality and anonymity of study participants:

• All direct identifiers must be removed from working data sets as early in the research process as possible. This process is called de-identification.

• De-identification must be done even in cases where the research team collects data indirectly, for instance, in the case of administrative data.

• At the time of de-identifying data, the research team must store an encrypted version of the original raw data which still contains the personal identifiers.

• Only those members of the research team who are listed by the institutional review board (IRB) should have access to the encrypted versions of confidential information.

• All research assistants (RAs), field coordinators (FCs), and other consultants or interns affiliated with the research team must sign non-disclosure agreements (NDAs). These agreements should clearly state that they will not share any sensitive information with anyone outside the research team. For example, the DIME NDA is an example of such an agreement.

Other Pillars of Research

Transparency

One of the most common concerns in research is the possibility of research teams manipulating the results of a study before publishing it in a journal. This is called publication bias, and can waste precious time and money. In order to make results of a study more transparent, the research team can pre-register their study with a registry. Some of the common registries are:

• The American Economic Association (AEA) hosts a trial registry specifically for randomized control trials.

• The International Initiative for Impact Evaluation (3ie) hosts the Registry for International Development Impact Evaluations (RIDIE) for both, experimental and quasi-experimental research in developing countries.

• The Open Science Framework offers a pre-registration platform, and also houses its own registry.

However, while ensuring transparency, research teams must not forget concerns about privacy of participants. Therefore, the research team must make sure that they do not disclose or publish personally identifiable information (PII) or confidential data, and should encrypt all such data as soon as they receive it from the field teams, or an external organisation.

Reproducibility

Reproducible research is the system of documenting and publishing results of an impact evaluation. At the very least, reproducibility allows other researchers to analyze the same data to get the same results as the original study, which strengthens the conclusions of the original study. It is important to push researchers towards publishing reproducible research because the path to research findings is just as important as the findings themselves.

DIME Analytics has also drafted the DIME Research Reproducibility Standards, which lists guidelines for ensuring reproducibility of research.

Data security

Data security is another important aspect of responsible research. The research team must handle ensure that members of the research team who are not listed by the IRB can not access any confidential data. Data can be confidential for multiple reasons, but the most common reason is that it contains personally identifiable information (PII). Other reasons include that the data was shared under a data usage license that requires the data to be kept confidential.

The most important aspect of data security is encryption. DIME Analytics has drafted the DIME Data Security Policy, which lists guidelines for ensuring data security during research.

Data publication

Data publication is the release of data and data documentation following data collection and analysis. It also involves de-identifying sensitive, or personal information before publication, and ensures that research is both ethical, as well as reproducible. While de-identifying, the research team must store an encrypted version of the original raw data which includes personal identifiers like names, addresses, location, etc.

It is also important to keep in mind the risks associated with data publication, such as concerns about right to privacy of human subjects. DIME Analytics has drafted the DIME Data Publication Standards, which lists guidelines for data publication.

Related Pages

Click here for pages that link to this topic.

Additional Resources

• Berkeley Initiative for Transparency in the Social Sciences (BITSS), Pre-Registration and Pre-Analysis Plans
• Berk Özler (World Bank), Taking ethics seriously: Response #1
• Berk Özler (World Bank), Research with adolescents: issues surrounding consent
• David McKenzie (World Bank), A pre-analysis plan checklist
• DIME Analytics (World Bank), Research Ethics & Data Security
• Martin Ravallion (World Bank), Taking ethical validity seriously
• J-PAL, Ethics