Research Ethics

Impact evaluation teams (or research teams) often work with, or have access to, datasets that contain sensitive or personal identifiable information (PII) on individuals. They can either have direct access (for instance, through a field survey), or indirect access to this data (for instance, in the form of call data records (CDR)). In both cases, it is important to ensure that research teams act ethically. Broadly, ethical research means that research teams must obtain necessary approvals, protect human subjects, obtain informed consent, and ensure confidentiality. Research ethics ensure that that steps taken to reach the outcomes of a study are just as important as the outcomes themselves, and help improve the validity of results of a study.

Read First

• Research ethics are the first pillar of the DIME Research Standards compiled by DIME Analytics.
• The DIME Research Ethics Guidelines discuss key guidelines for researchers affiliated with DIME.
• A human subject is a living individual about whom the research team obtains personal identifiable information (PII), either directly or indirectly.
• The idea of research ethics has the following aspects - ethics approvals, protecting human subjects, informed consent, and confidentiality.
• Each of these components is important to ensure that research teams stick to certain ethical standards while conducting research.
• Research ethics, along with other pillars of research - transparency, reproducibility, data security, and data publication - ensures greater validity for the results of a research study.

Protecting Human Subjects

Members of the research team must ensure that they protect human research subjects and their rights, including the right to privacy. In this context, all living individuals whose sensitive or personally identifiable information (PII) is contained in the datasets being used in the study are considered human subjects. Therefore, ethical research requires that all members of the research team who handle personally identifiable information (PII) must have up-to-date human research subjects certification. Two organizations that offer courses to certify research team members are:

• Protecting Human Research Participants (PHRP)
• Collaborative Institutional Training Initiative (CITI)

Note: It does not matter whether the research team has direct, or indirect access to the data. If the dataset being used in the study contains personally identifiable information (PII) about individuals, then research team must protect the right to privacy of every such individual.

Ethics Approvals

Institutional review boards (IRBs) are organizations that review and monitor research studies to protect the rights of human subjects. The research team must obtain IRB approvals for studies that use personally identifiable information. This is a powerful tool to promote ethical research because IRBs can deny research teams the right to use data if they do not follow proper guidelines while handling the data.

Note: Keep the following points in mind with respect to ethics approvals:

• It does not matter whether the research team collected the data directly, or indirectly. For example, consider a study that uses mobile phone location data to assess effectiveness of stay-at-home orders during COVID-19. In this case, the research team does not directly collect this data, but gets access to it through the telecom service provider. However, since this dataset contains sensitive information like names, phone numbers, and location of individuals, the research team must obtain an IRB approval.

• In addition to IRB approvals, the research team should also obtain approvals from local institutions in the location of the study. This will ensure that the study complies with local regulations, and does not violate any laws in that area, particular with respect to the right to privacy.

Informed Consent

Before involving any individual in a research study, the research team must obtain informed consent from each individual. This means that the research team must clearly mention all possible risks and benefits from participating in a study, either for the survey pilot or as a respondent in the actual survey.

Note: Keep the following points in mind regarding informed consent:

• The human subjects must be able to refuse participation in a study at any point during the study.

• DIME Analytics has created templates for both written and oral consent.

• The mechanism of informed consent applies only to cases where the research team itself is collecting data, such as through a field survey, because in such cases the research team is directly interacting with the participants.

• However, in some cases, there is no opportunity for the research team to obtain informed consent. For example, consider cases where the research team only has indirect access to data, such as through call data records (CDR), or administrative data. This does not mean that the right to privacy of individuals is not important in such a scenario. It simply means that the research team must use other mechanisms, such as ensuring confidentiality of sensitive information, which is discussed in the next section.

Confidentiality

The research team must also ensure complete anonymity and confidentiality. This means that the identity of study participants should remains hidden, and sensitive information of individuals should never be shared with anyone outside the research team. Research team members should be held personally liable for any actions that result in disclosure of sensitive and personal data.

Note: Keep the following points in mind regarding confidentiality and anonymity of study participants:

• All direct identifiers must be removed from working data sets as early in the research process as possible. This process is called de-identification.

• De-identification must be done even in cases where the research team collects data indirectly, for instance, in the case of administrative data.

• All research assistants (RAs), field coordinators (FCs), and other consultants or interns affiliated with the research team must sign non-disclosure agreements (NDAs). These agreements should clearly state that they will not share any sensitive information with anyone outside the research team. For example, the DIME NDA is an example of such an agreement.

• Only those members of the research team who are listed by the institutional review board (IRB) should have access to confidential information.

Other Pillars of Research

Transparency

One of the most common concerns in research is the possibility of research teams manipulating the results of a study before publishing it in a journal. This is called publication bias, and can waste precious time and money. In order to make results of a study more transparent, the research team can pre-register their study with a registry. Some of the common registries are:

• The American Economic Association (AEA) hosts a trial registry specifically for randomized control trials.

• The International Initiative for Impact Evaluation (3ie) hosts the Registry for International Development Impact Evaluations (RIDIE) for both, experimental and quasi-experimental research in developing countries.

• The Open Science Framework offers a pre-registration platform, and also houses its own registry.

Reproducibility

Reproducible research is the system of documenting and publishing results of an impact evaluation. At the very least, reproducibility allows other researchers to analyze the same data to get the same results as the original study, which strengthens the conclusions of the original study. It is important to push researchers towards publishing reproducible research because the path to research findings is just as important as the findings themselves.

Related Pages

Click here for pages that link to this topic.

Additional Resources

• Berkeley Initiative for Transparency in the Social Sciences (BITSS), Pre-Registration and Pre-Analysis Plans
• Berk Özler (World Bank), Taking ethics seriously: Response #1
• Berk Özler (World Bank), Research with adolescents: issues surrounding consent
• David McKenzie (World Bank), A pre-analysis plan checklist
• DIME Analytics (World Bank), Research Ethics & Data Security
• Martin Ravallion (World Bank), Taking ethical validity seriously
• J-PAL, Ethics