Research Documentation

When publishing research, it is important to make documentation available so that readers can understand the details of the research design that the work reports. This includes all of the technical details and decisions that could influence how the findings are read or understood. Usually, this will involve producing a document along the lines of a methodological note or appendix. That document will describe how a given study was designed and how the design was carried out. The level of detail is in such a document should be relatively high. This page will describe some common approaches to compiling this kind of material and retaining the needed information in an organized fashion throughout the life of a research project.

Read First

• Research documentation provides the context to understanding the results of a given research output.
• There is no standard form for this documentation, and its location and format will depend on the type of research output produced.
• For academic materials, this documentation often takes the form of a structured methodological appendix.
• For policy outputs or online products, it may be appropriate to include an informative README webpage or document.
• The most important process for preparing this documentation will be retaining and organizing the needed information throughout the life of the project, so that the team will not have to search through communications or data archives for small details at publication time.

What to include in research documentation

Research documentation should include all the information that is needed to understand the underlying design for the research output. This can include descriptions of:

• Populations of interest that informed the study
• Methods of sampling or other sources of data about selecting the units of observation that were actually included in the study
• Power calculations and pre-analysis plans
• Field work, including data collection or experimental manipulation, such as study protocols and monitoring or quality assurance information
• Data collection tools such as survey instruments, search keywords, and instructions or code for API requests or database queries
• Statistical approaches such as definitions of key constructed indicators, corrections or adjustments to data, and precise definitions of estimators and estimation procedures
• Data completeness, including non-observed units or quantities that were planned or "tracking" information

All of the research documentation taken together should broadly allow a reader to understand how information was gathered, what it represents, what kind of information and data files to expect, and how to relate that information to the results of the research. Research documentation is not a complete guide to data, however; it does not need to provide the level of detail or instructions that would enable a reader to approach different research questions using the same data.

Documentation will take different forms depending on the information included. Much of it will be written narrative rather than, for example, formal datasets. Understanding research documentation should not require the user to have any special software or to undertake any analytical tasks themselves. Relevant datasets (such as tracking of units of observation over time) might be included alongside the documentation, but the documentation should summarize in narrative form all the information from that dataset that is likely to affect the interpretation of the research.

Structuring research documentation as a publication appendix

If you are preparing documentation to accompany the publication of an academic output such as a working paper or journal article, the most common form of research documentation is a structured supplemental appendix. Check the journal's publication process for details. Some publishers allow unlimited supplementary materials to be included in a format such as an author-created document. These materials may or may not be included under the peer review of the main manuscript and might only be intended to provide context for readers and reviewers. In this case you should provide complete information in that material. Other publishers expect all supplementary materials to be read and reviewed as part of the publication process. In this case you should provide the minimum additional detail required to understand the research here (since much of the appendix will likely be taken up by supplementary results rather than documentation), and consider other methods for releasing complete documentation, such as self-publication on OSF or Zenodo.

Since there is unlimited space and you may have a large amount of material to include in a documentation appendix, organization is essential. It is appropriate to have several appendices that cover different aspects of the research. For example, Appendix A may include information about the study population and data, such as the total number of units available for observation, the number selected or included for observation, the number successfully included, and descriptive statistics about subgroups, strata, clusters, or other units relevant to the research. It could be accompanied by a tracking dataset with full information about the process. Appendix B might include information about an intended experimental manipulation in one section, and information about implementation, take-up, and fidelity in a second section. It could be accompanied by a dataset with key indicators. Appendix C might include data collection protocols and definitions of constructed variables and comparisons with alternative definitions, and be accompanied by data collection instruments and illustrative figures. Each appendix should included relevant references. Supplementary exhibits should be numbered to correspond with the appendix they pertain to. More granular appendices are generally preferable so that referencing and numbering remains relatively uncomplicated.

There have been many attempts to standardized some of these elements, such as the STROBE and CONSORT reporting checklists. Journals will let you know if they expect these exact templates to be followed. Even if they are not required, such templates can still be used directly or to provide inspiration or structure for the materials you might want to include.