Difference between revisions of "Research Ethics"
(96 intermediate revisions by 6 users not shown) | |||
Line 1: | Line 1: | ||
[[Impact Evaluation Team|Impact evaluation teams]] (or '''research teams''') often work with, or have access to, datasets that contain sensitive or [[Personally Identifiable Information (PII)|personal identifiable information (PII)]] on individuals. They can either have [[Primary Data Collection|direct access]] (for instance, through a [[Field Surveys|field survey]]), or [[Secondary Data Sources|indirect access]] to this data (for instance, in the form of '''call data records (CDR)'''). In both cases, it is important to ensure that '''research teams''' act '''ethically'''. Broadly, '''ethical research''' means that research teams must [[IRB Approval|obtain necessary approvals]], [[Protecting Human Research Subjects|protect human subjects]], [[Informed Consent|obtain informed consent]], and [[De-identification|ensure confidentiality]]. This also means that the research team should clearly think about, and be transparent about [[Data Ownership|ownership]] of the data they use. '''Research ethics''' ensure that that steps taken to reach the outcomes of a study are just as important as the outcomes themselves, and help improve the validity of results of a study. Note that Research Ethics and [[Data Security|data security]] go hand-in-hand. | |||
== Read First == | == Read First == | ||
* [https://github.com/worldbank/dime-standards/tree/master/dime-research-standards/pillar-1-research-ethics DIME Research Ethics | * The [https://github.com/worldbank/dime-standards/tree/master/dime-research-standards/pillar-1-research-ethics DIME Research Ethics Guidelines] discuss key guidelines for researchers affiliated with [https://www.worldbank.org/en/research/dime DIME]. | ||
* [[Protecting Human Research Subjects| | * A '''human subject''' is a living individual about whom the [[Impact Evaluation Team|research team]] obtains [[Personally Identifiable Information (PII)|personal identifiable information (PII)]], either directly or indirectly. | ||
* [[ | * The idea of '''research ethics''' has the following aspects - [[IRB Approval|ethics approvals]], [[Protecting Human Research Subjects|protecting human subjects]], [[Informed Consent|informed consent]], and [[De-identification|confidentiality]]. | ||
* Each of these components is important to ensure that '''research teams''' stick to certain ethical standards while conducting research. | |||
* '''Research ethics''', along with other pillars of research - [[Pre-Registration|transparency]], [[Reproducible Research|reproducibility]], [[Data Security|data security]], and [[Publishing Data|data publication]] - ensures greater validity for the results of a research study. | |||
== Protecting Human Subjects == | |||
Members of the [[Impact Evaluation Team|research team]] must ensure that they [[Protecting Human Research Subjects|protect human research subjects]] and their rights, including the '''right to privacy'''. In this context, all living individuals whose sensitive or [[Personally Identifiable Information (PII)|personally identifiable information (PII)]] is contained in the datasets being used in the study are considered '''human subjects'''. Therefore, '''ethical research''' requires that all members of the '''research team''' who handle '''personally identifiable information (PII)''' must have up-to-date '''human research subjects certification'''. Two organizations that offer courses to certify research team members are: | |||
* [https://phrp.nihtraining.com/users/login.php Protecting Human Research Participants (PHRP)] | |||
* [https://about.citiprogram.org/en/series/human-subjects-research-hsr/ Collaborative Institutional Training Initiative (CITI)] | |||
'''Note''': It does not matter whether the research team has [[Primary Data Collection|direct]], or [[Secondary Data Sources|indirect]] access to the data. If the dataset being used in the study contains '''personally identifiable information (PII)''' about individuals, then research team must protect the '''right to privacy''' of every such individual. Right to privacy includes ensuring confidentiality, privacy, and anonymity of study participants. | |||
== Ethics Approvals == | |||
'''Institutional review boards (IRBs)''' are organizations that review and monitor research studies to [[Protecting Human Research Subjects|protect the rights]] of '''human subjects'''. The [[Impact Evaluation Team|research team]] must obtain [[IRB Approval|IRB approvals]] for studies that use [[Personally Identifiable Information (PII)|personally identifiable information]]. This is a powerful tool to promote '''ethical research''' because '''IRBs''' can deny research teams the right to use data if they do not follow proper guidelines while handling the data. | |||
'''Note''': Keep the following points in mind with respect to '''ethics approvals''': | |||
* It does not matter whether the research team collected the data [[Primary Data Collection|directly]], or [[Secondary Data Sources|indirectly]]. For example, consider a study that uses mobile phone location data to assess effectiveness of stay-at-home orders during COVID-19. In this case, the research team does not directly collect this data, but gets access to it through the telecom service provider. However, since this dataset contains sensitive information like names, phone numbers, and location of individuals, the research team must obtain an '''IRB approval'''. | |||
* In addition to '''IRB approvals''', the research team should also obtain approvals from local institutions in the location of the study. This will ensure that the study complies with local regulations, and does not violate any laws in that area, particular with respect to the '''right to privacy'''. | |||
== | == Informed Consent == | ||
Before the [[Impact Evaluation Team|research team]] | Before collecting personal data from any individual involved in a research study, the [[Impact Evaluation Team|research team]] must obtain [[Informed Consent|informed consent]] from each individual. This means that the research team must clearly mention all possible risks and benefits from participating in a study, either for the [[Survey Pilot|survey pilot]] or as a [[Survey Pilot Participants|respondent]] in the actual survey. This is done through an informed consent module conducted at the start of a survey interview. | ||
* | '''Note''': Keep the following points in mind regarding '''informed consent''': | ||
* The '''human subjects''' must be able to refuse participation in a study at any point during the study. | |||
* | * [https://www.worldbank.org/en/research/dime/data-and-analytics DIME Analytics] has created templates for [https://github.com/worldbank/dime-standards/blob/master/dime-research-standards/pillar-1-research-ethics/research-ethics-resources/informed-consent-templates/informed-consent-template-written.md written], [https://github.com/worldbank/dime-standards/blob/master/dime-research-standards/pillar-1-research-ethics/research-ethics-resources/informed-consent-templates/informed-consent-template-verbal.md oral], and [https://github.com/worldbank/dime-standards/blob/master/dime-research-standards/pillar-1-research-ethics/research-ethics-resources/informed-consent-templates/informed-consent-template-sms.md SMS] consent. Templates adapted to include COVID-19 risks are also available on the same [https://github.com/worldbank/dime-standards/tree/master/dime-research-standards/pillar-1-research-ethics/research-ethics-resources/informed-consent-templates repository]. | ||
* ''' | * The mechanism of '''informed consent''' applies only to cases where the research team itself is [[Primary Data Collection|collecting data]], such as through a [[Field Surveys|field survey]], because in such cases the research team is directly interacting with the participants. | ||
* However, in some cases, there is no opportunity for the research team to obtain '''informed consent'''. For example, consider cases where the research team only has [[Secondary Data Sources|indirect]] access to data, such as through '''call data records (CDR)''', or [[Administrative and Monitoring Data|administrative data]]. This does not mean that the '''right to privacy''' of individuals is not important in such a scenario. It simply means that the research team must use other mechanisms, such as ensuring '''confidentiality''' of sensitive information, which is discussed in the next section. | |||
== Confidentiality == | |||
The [[Impact Evaluation Team|research team]] must also ensure complete '''anonymity''' and '''confidentiality'''. This means that the identity of study participants should remain hidden, and sensitive information of individuals should never be shared with anyone outside the research team. Research team members should be held personally liable for any actions that result in disclosure of sensitive and [[Personally Identifiable Information (PII)|personal]] data. | |||
'''Note''': Keep the following points in mind regarding '''confidentiality''' and '''anonymity''' of study participants: | |||
* All direct identifiers must be removed from working data sets as early in the research process as possible. This process is called [[De-identification|de-identification]]. | |||
* '''De-identification''' must be done even in cases where the research team collects data [[Secondary Data Sources|indirectly]], for instance, in the case of [[Administrative and Monitoring Data|administrative data]]. | |||
== | * At the time of '''de-identifying''' data, the research team must store an [[Encryption|encrypted]] version of the original raw data which still contains the '''personal identifiers'''. | ||
* Only those members of the research team who are listed by the [[IRB Approval|institutional review board (IRB)]] should have access to the '''encrypted''' versions of confidential information. | |||
* All research assistants '''(RAs)''', field coordinators '''(FCs)''', and other consultants or interns affiliated with the research team must sign '''non-disclosure agreements (NDAs)'''. These agreements should clearly state that they will not share any sensitive information with anyone outside the research team. For example, the [https://github.com/worldbank/dime-standards/blob/master/dime-research-standards/pillar-4-data-security/data-security-resources/dime-data-nda.md DIME NDA] is an example of such an agreement. | |||
== Other Aspects of Ethical Research == | |||
=== Transparency === | |||
One of the most common concerns in research is the possibility of [[Impact Evaluation Team|research teams]] manipulating the results of a study before publishing it in a journal. This is called '''publication bias''', and can waste precious time and money. In order to make results of a study more '''transparent''', the '''research team''' can [[Pre-Registration | pre-register]] their study with a '''registry'''. Some of the common registries are: | |||
* The [https://www.aeaweb.org/ American Economic Association (AEA)] hosts a [https://www.socialscienceregistry.org/ trial registry] specifically for [[Randomized Control Trials|randomized control trials]]. | |||
* The [https://www.3ieimpact.org/ International Initiative for Impact Evaluation (3ie)] hosts the [https://ridie.3ieimpact.org/ Registry for International Development Impact Evaluations (RIDIE)] for both, [[Experimental Methods|experimental]] and [[Quasi-Experimental Methods|quasi-experimental research]] in developing countries. | |||
* The [https://osf.io/institutions/cos/?gclid=Cj0KCQjw-_j1BRDkARIsAJcfmTFQUYhsJDTOljYCOKEsRzjNJo5QQHO_SSgFCglP5wxcF_l0zImzEaoaAvAuEALw_wcB Open Science Framework] offers a [https://osf.io/prereg/ pre-registration platform], and also houses its own [https://osf.io/registries registry]. | |||
However, while ensuring '''transparency''', research teams must not forget concerns about privacy of participants. Therefore, the research team must make sure that they do not disclose or publish [[Personally Identifiable Information (PII)|personally identifiable information (PII)]] or confidential data, and should [[Encryption|encrypt]] all such data as soon as they receive it from the '''field teams''', or an external organisation. | |||
=== Reproducibility === | |||
[[Reproducible Research|Reproducible research]] is the system of [[Data Documentation|documenting]] and [[Publishing Data|publishing]] results of an '''impact evaluation'''. At the very least, '''reproducibility''' allows other researchers to [[Data Analysis|analyze]] the same data to get the same results as the original study, which strengthens the conclusions of the original study. It is important to push researchers towards publishing '''reproducible research''' because the path to research findings is just as important as the findings themselves. | |||
[https://www.worldbank.org/en/research/dime/data-and-analytics DIME Analytics] has also drafted the [https://github.com/worldbank/dime-standards/tree/master/dime-research-standards/pillar-3-research-reproducibility DIME Research Reproducibility Standards], which lists guidelines for ensuring '''reproducibility''' of research. | |||
=== Data security === | |||
[[Data Security|Data security]] is another important aspect of responsible research. The [[Impact Evaluation Team|research team]] must ensure that members of the '''research team''' who are not listed by the [[IRB Approval|IRB]] can not access any confidential data. Data can be confidential for multiple reasons, but the most common reason is that it contains [[Personally Identifiable Information (PII)|personally identifiable information (PII)]]. Other reasons include that the data was shared under a data usage license that requires the data to be kept confidential. | |||
The most important aspect of '''data security''' is [[Encryption|encryption]]. [https://www.worldbank.org/en/research/dime/data-and-analytics DIME Analytics] has drafted the [https://github.com/worldbank/dime-standards/blob/master/dime-research-standards/pillar-4-data-security/dime-data-security-guidelines.md DIME Data Security Policy], which lists guidelines for ensuring '''data security''' during research. | |||
=== Data publication === | |||
[[Publishing Data|Data publication]] is the release of data and [[Data Documentation|data documentation]] following [[Primary Data Collection | data collection]] and [[Data Analysis | analysis]]. It also involves [[De-identification|de-identifying]] sensitive, or [[Personally Identifiable Information (PII)|personal information]] before publication, and ensures that research is both ethical, as well as [[Reproducible Research|reproducible]]. While '''de-identifying''', the [[Impact Evaluation Team|research team]] must store an [[Encryption|encrypted]] version of the original raw data which includes '''personal identifiers''' like names, addresses, location, etc. | |||
It is also important to keep in mind the risks associated with '''data publication''', such as concerns about '''right to privacy''' of '''human subjects'''. [https://www.worldbank.org/en/research/dime/data-and-analytics DIME Analytics] has drafted the [https://github.com/worldbank/dime-standards/tree/master/dime-research-standards/pillar-5-data-publication DIME Data Publication Standards], which lists guidelines for '''data publication'''. Finally, the research team must also think about [[Data Ownership|data ownership]] through a set of clear rules for use of research data. | |||
== Related Pages == | == Related Pages == | ||
Line 42: | Line 80: | ||
== Additional Resources == | == Additional Resources == | ||
* DIME Analytics (World Bank), [https://osf.io/6fcvk Research Ethics] | |||
* DIME Analytics (World Bank), [https://osf.io/7yhjm Handling Personal Data Safely] | |||
* Berkeley Initiative for Transparency in the Social Sciences (BITSS), [https://www.bitss.org/research-transparency-mooc/pre-registration-and-pre-analysis-plans/ Pre-Registration and Pre-Analysis Plans] | |||
* Berk Özler (World Bank), [http://blogs.worldbank.org/impactevaluations/taking-ethics-seriously-response-1 Taking ethics seriously: Response #1] | * Berk Özler (World Bank), [http://blogs.worldbank.org/impactevaluations/taking-ethics-seriously-response-1 Taking ethics seriously: Response #1] | ||
* Berk Özler (World Bank), [https://blogs.worldbank.org/impactevaluations/research-adolescents-issues-surrounding-consent Research with adolescents: issues surrounding consent] | * Berk Özler (World Bank), [https://blogs.worldbank.org/impactevaluations/research-adolescents-issues-surrounding-consent Research with adolescents: issues surrounding consent] | ||
* | * David Evans (Center for Global Development), [https://www.cgdev.org/blog/practical-suggestions-more-ethical-social-science-rcts Practical Suggestions for More Ethical Social Science RCTs] | ||
* David McKenzie (World Bank), [https://blogs.worldbank.org/impactevaluations/a-pre-analysis-plan-checklist A pre-analysis plan checklist] | |||
* Emma Cohn and Douglas MacKay, [https://dmackay.web.unc.edu/ethics-of-field-experiments-a-bibliography/ Ethics of Field Experiments: A Bibliography] | |||
* Martin Ravallion (World Bank), [https://blogs.worldbank.org/impactevaluations/taking-ethical-validity-seriously Taking ethical validity seriously] | * Martin Ravallion (World Bank), [https://blogs.worldbank.org/impactevaluations/taking-ethical-validity-seriously Taking ethical validity seriously] | ||
* J-PAL, [https://www.povertyactionlab.org/ethics Ethics] | * J-PAL, [https://www.povertyactionlab.org/ethics Ethics] | ||
* J-PAL, [https://www.povertyactionlab.org/resource/ethical-conduct-randomized-evaluations Ethical conduct of randomized evaluations] | |||
* Laterite, [https://www.laterite.com/blog/ethics-approvals-and-research-permits-in-east-africa/ Ethics approvals and research permits in East Africa] | |||
[[Category: Reproducible Research]] |
Latest revision as of 20:22, 16 August 2023
Impact evaluation teams (or research teams) often work with, or have access to, datasets that contain sensitive or personal identifiable information (PII) on individuals. They can either have direct access (for instance, through a field survey), or indirect access to this data (for instance, in the form of call data records (CDR)). In both cases, it is important to ensure that research teams act ethically. Broadly, ethical research means that research teams must obtain necessary approvals, protect human subjects, obtain informed consent, and ensure confidentiality. This also means that the research team should clearly think about, and be transparent about ownership of the data they use. Research ethics ensure that that steps taken to reach the outcomes of a study are just as important as the outcomes themselves, and help improve the validity of results of a study. Note that Research Ethics and data security go hand-in-hand.
Read First
- The DIME Research Ethics Guidelines discuss key guidelines for researchers affiliated with DIME.
- A human subject is a living individual about whom the research team obtains personal identifiable information (PII), either directly or indirectly.
- The idea of research ethics has the following aspects - ethics approvals, protecting human subjects, informed consent, and confidentiality.
- Each of these components is important to ensure that research teams stick to certain ethical standards while conducting research.
- Research ethics, along with other pillars of research - transparency, reproducibility, data security, and data publication - ensures greater validity for the results of a research study.
Protecting Human Subjects
Members of the research team must ensure that they protect human research subjects and their rights, including the right to privacy. In this context, all living individuals whose sensitive or personally identifiable information (PII) is contained in the datasets being used in the study are considered human subjects. Therefore, ethical research requires that all members of the research team who handle personally identifiable information (PII) must have up-to-date human research subjects certification. Two organizations that offer courses to certify research team members are:
- Protecting Human Research Participants (PHRP)
- Collaborative Institutional Training Initiative (CITI)
Note: It does not matter whether the research team has direct, or indirect access to the data. If the dataset being used in the study contains personally identifiable information (PII) about individuals, then research team must protect the right to privacy of every such individual. Right to privacy includes ensuring confidentiality, privacy, and anonymity of study participants.
Ethics Approvals
Institutional review boards (IRBs) are organizations that review and monitor research studies to protect the rights of human subjects. The research team must obtain IRB approvals for studies that use personally identifiable information. This is a powerful tool to promote ethical research because IRBs can deny research teams the right to use data if they do not follow proper guidelines while handling the data.
Note: Keep the following points in mind with respect to ethics approvals:
- It does not matter whether the research team collected the data directly, or indirectly. For example, consider a study that uses mobile phone location data to assess effectiveness of stay-at-home orders during COVID-19. In this case, the research team does not directly collect this data, but gets access to it through the telecom service provider. However, since this dataset contains sensitive information like names, phone numbers, and location of individuals, the research team must obtain an IRB approval.
- In addition to IRB approvals, the research team should also obtain approvals from local institutions in the location of the study. This will ensure that the study complies with local regulations, and does not violate any laws in that area, particular with respect to the right to privacy.
Informed Consent
Before collecting personal data from any individual involved in a research study, the research team must obtain informed consent from each individual. This means that the research team must clearly mention all possible risks and benefits from participating in a study, either for the survey pilot or as a respondent in the actual survey. This is done through an informed consent module conducted at the start of a survey interview.
Note: Keep the following points in mind regarding informed consent:
- The human subjects must be able to refuse participation in a study at any point during the study.
- DIME Analytics has created templates for written, oral, and SMS consent. Templates adapted to include COVID-19 risks are also available on the same repository.
- The mechanism of informed consent applies only to cases where the research team itself is collecting data, such as through a field survey, because in such cases the research team is directly interacting with the participants.
- However, in some cases, there is no opportunity for the research team to obtain informed consent. For example, consider cases where the research team only has indirect access to data, such as through call data records (CDR), or administrative data. This does not mean that the right to privacy of individuals is not important in such a scenario. It simply means that the research team must use other mechanisms, such as ensuring confidentiality of sensitive information, which is discussed in the next section.
Confidentiality
The research team must also ensure complete anonymity and confidentiality. This means that the identity of study participants should remain hidden, and sensitive information of individuals should never be shared with anyone outside the research team. Research team members should be held personally liable for any actions that result in disclosure of sensitive and personal data.
Note: Keep the following points in mind regarding confidentiality and anonymity of study participants:
- All direct identifiers must be removed from working data sets as early in the research process as possible. This process is called de-identification.
- De-identification must be done even in cases where the research team collects data indirectly, for instance, in the case of administrative data.
- At the time of de-identifying data, the research team must store an encrypted version of the original raw data which still contains the personal identifiers.
- Only those members of the research team who are listed by the institutional review board (IRB) should have access to the encrypted versions of confidential information.
- All research assistants (RAs), field coordinators (FCs), and other consultants or interns affiliated with the research team must sign non-disclosure agreements (NDAs). These agreements should clearly state that they will not share any sensitive information with anyone outside the research team. For example, the DIME NDA is an example of such an agreement.
Other Aspects of Ethical Research
Transparency
One of the most common concerns in research is the possibility of research teams manipulating the results of a study before publishing it in a journal. This is called publication bias, and can waste precious time and money. In order to make results of a study more transparent, the research team can pre-register their study with a registry. Some of the common registries are:
- The American Economic Association (AEA) hosts a trial registry specifically for randomized control trials.
- The International Initiative for Impact Evaluation (3ie) hosts the Registry for International Development Impact Evaluations (RIDIE) for both, experimental and quasi-experimental research in developing countries.
- The Open Science Framework offers a pre-registration platform, and also houses its own registry.
However, while ensuring transparency, research teams must not forget concerns about privacy of participants. Therefore, the research team must make sure that they do not disclose or publish personally identifiable information (PII) or confidential data, and should encrypt all such data as soon as they receive it from the field teams, or an external organisation.
Reproducibility
Reproducible research is the system of documenting and publishing results of an impact evaluation. At the very least, reproducibility allows other researchers to analyze the same data to get the same results as the original study, which strengthens the conclusions of the original study. It is important to push researchers towards publishing reproducible research because the path to research findings is just as important as the findings themselves.
DIME Analytics has also drafted the DIME Research Reproducibility Standards, which lists guidelines for ensuring reproducibility of research.
Data security
Data security is another important aspect of responsible research. The research team must ensure that members of the research team who are not listed by the IRB can not access any confidential data. Data can be confidential for multiple reasons, but the most common reason is that it contains personally identifiable information (PII). Other reasons include that the data was shared under a data usage license that requires the data to be kept confidential.
The most important aspect of data security is encryption. DIME Analytics has drafted the DIME Data Security Policy, which lists guidelines for ensuring data security during research.
Data publication
Data publication is the release of data and data documentation following data collection and analysis. It also involves de-identifying sensitive, or personal information before publication, and ensures that research is both ethical, as well as reproducible. While de-identifying, the research team must store an encrypted version of the original raw data which includes personal identifiers like names, addresses, location, etc.
It is also important to keep in mind the risks associated with data publication, such as concerns about right to privacy of human subjects. DIME Analytics has drafted the DIME Data Publication Standards, which lists guidelines for data publication. Finally, the research team must also think about data ownership through a set of clear rules for use of research data.
Related Pages
Click here for pages that link to this topic.
Additional Resources
- DIME Analytics (World Bank), Research Ethics
- DIME Analytics (World Bank), Handling Personal Data Safely
- Berkeley Initiative for Transparency in the Social Sciences (BITSS), Pre-Registration and Pre-Analysis Plans
- Berk Özler (World Bank), Taking ethics seriously: Response #1
- Berk Özler (World Bank), Research with adolescents: issues surrounding consent
- David Evans (Center for Global Development), Practical Suggestions for More Ethical Social Science RCTs
- David McKenzie (World Bank), A pre-analysis plan checklist
- Emma Cohn and Douglas MacKay, Ethics of Field Experiments: A Bibliography
- Martin Ravallion (World Bank), Taking ethical validity seriously
- J-PAL, Ethics
- J-PAL, Ethical conduct of randomized evaluations
- Laterite, Ethics approvals and research permits in East Africa