Biomedical Science


Rigor and Reproducibility in Health and Biomedical Research and Scholarship

Home

Introduction

This guide addresses methods to support open science, including general approaches to transparency and reproducibility such as pre-registration and registered reports. In addition it references specific guidance from the NIH and useful tools for laboratory sciences.

Pre-registration

Pre-registration is the process of describing a research question and analysis plan before observing the research outcomes (1). Specifically, this can entail writing out specific details such as data collection methods, analysis plans, and rules for data exclusion (2). The goal with pre-registration is to reduce practices related to publication bias such as HARKing (Hypothesizing After the Results are Known), “p-hacking” (manipulating data analyses in order to obtain significant effects, and the tendency to publish only positive or novel results. 

Pre-registering Systematic Reviews

PROSPERO is an international database of prospectively registered reviews in health and social care. PROSPERO allows registration of systematic reviews, rapid reviews and umbrella reviews, but not scoping reviews or literature scans. Reviews should be registered at the protocol stage. Sharing topics and methods can allow comparison of reviews, transparency about methods, and help avoid duplication with future reviews.

Pre-registering Clinical Research

Clinical trial registration "is the practice of documenting clinical trials before they are performed in a clinical trials registry so as to combat publication bias and selective reporting" (4). 

Clinicaltrials.gov is the largest database of clinical trials, hosting descriptions and results from privately and publicly funded clinical studies. Many funders and publishers require registration of clinical trials.

General Pre-Registration Tools and Databases

Research studies of all types can be pre-registered using the tools below:

Tools

Databases

Read More:

1. Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2600-2606. doi: 10.1073/pnas.1708274114. PMID: 29531091; PMCID: PMC5856500.

2. Standin E. 2019. Open science, pre-registration and striving for better research practices. American Psychological Association:Psychological Science Agenda | November 2019  

3. Gonzales JE, Cunningham CA. 2015. The promise of pre-registration in psychological research. American Psychological Association: Psychological Science Agenda | August 2015.

4. Wikipedia. Preregistration (Science): Clinical trial registration

Registered Reports

Registered Reports represent a change to the traditional publishing paradigm. Publishing a Registered Report involves submitting a research protocol to the journal before experiments are conducted. Protocols are reviewed, and once accepted, the journal commits to publishing the outcomes. This commitment to publishing helps to avoid cherry-picking results and foster sharing of all types of outcomes, even negative results.

The process often involves several steps, including:

  • Submitting your research question and study design to a journal or publisher
  • Stage 1 peer review on your plan and methods
  • In principle acceptance (IPA) by the journal for publication
  • Conducting the study and writing up the results
  • Stage 2 peer review (whether study matches plan)
  • Publication

Like pre-registration, Registered Reports supports research transparency and can prevent publication bias and address questionable research practices like p-hacking and HARKing.

Registered Reports Tools

Center for Open Science | Open Science Framework Registered Reports - create a Registered Report (typically done after you receive an IPA)

Sample Registered Reports:

Plos ONE Registered Report Protocols

Sample protocol: Text messaging as a tool to improve cancer screening programs (M-TICS Study): A randomized controlled trial protocol

Nature Human Behavior Registered Reports

Sample report: Dissociating memory accessibility and precision in forgetting (sample of collection of materials that might accompany a registered report) 

Read More:

Chambers C. What's next for Registered Reports? Nature. 2019 Sep;573(7773):187-189. doi: 10.1038/d41586-019-02674-6. PMID: 31506624.

Smith GD, Penny KI. Enhancing research integrity in academic nursing: The introduction of registered reports. J Clin Nurs. 2019 Apr;28(7-8):1037-1038. doi: 10.1111/jocn.14770. Epub 2019 Feb 8. PMID: 30618059.

Journal of Clinical Nursing (JCN) submission guidelines

Gray R, Thompson DR, Tong Chien W, Jones M, Jones A, Moyo N, Waters A, Brown E. Why has the Journal of Psychiatric and Mental Health Nursing stopped publishing registered reports? J Psychiatr Ment Health Nurs. 2020 Dec 18. doi: 10.1111/jpm.12721. Epub ahead of print. PMID: 33336865.

The editorial board notes the above discontinuation by the editorial team. The team respond that the primary reasons for closing registered reports are the lack of these papers submitted to the journal and subsequently the lack of good quality reports published.

Research-related Guidelines and Resources

Effective January 25, 2016, NIH has established a formal initiative that requires most NIH and AHRQ research grant applications to comply with the NIH Policy on Rigor and Reproducibility.

For more information, see the official notice: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-16-011.html

These resources and tools will help biomedical researchers to meet Rigor and Reproducibility Requirements from NIH. 

Science Signaling

Statistics and Computational Analysis Specific for Biologists

Chemical Probes and Biomarkers

Resources for Considering Sex as a Biological Variable

  

Cell Line Authentication

NIH suggests "applicants proposing to use cell lines could describe the method they plan to use to verify the identity and purity of the lines, which might include short tandem repeat (STR) profiling and mycoplasma testing." Below are some resources for writing an authentication plan and procedures for authenticating cell lines.

Guidelines and Resources from Professional Organizations

Literature Review

Specific for Non-Human Cell Lines

Antibody Validation

Checklists and Validation Methods

Resources from Manufacturers

Literature Review

Image Guidelines

The following resources and guidelines are for researchers who acquire images and use software image like Photoshop.

Skip to Main Content

Claude Moore Health Sciences Library
1350 Jefferson Park Avenue P.O. Box 800722
Charlottesville, VA 22908 (Directions)

facebook twitter instagram
© 2021 by the Rector and Visitors of the University of Virginia
Copyright & Privacy