This guide covers resources to support open science, including practices for better transparency and reproducibility in biomedical research. It addresses:

• Publishing paradigms (pre-registration, registered reports, and preprints)
• Research Data Management
• Description, Authentication and Validity (protocols, cell lines, antibodies)

Pre-registration is the process of describing a research question and analysis plan before observing the research outcomes (1). Specifically, this can entail writing out specific details such as data collection methods, analysis plans, and rules for data exclusion (2). The goal with pre-registration is to reduce practices related to publication bias such as HARKing (Hypothesizing After the Results are Known), “p-hacking” (manipulating data analyses in order to obtain significant effects, and the tendency to publish only positive or novel results. 

Pre-registering Systematic Reviews

PROSPERO is an international database of prospectively registered reviews in health and social care. PROSPERO allows registration of systematic reviews, rapid reviews and umbrella reviews, but not scoping reviews or literature scans. Reviews should be registered at the protocol stage. Sharing topics and methods can allow comparison of reviews, transparency about methods, and help avoid duplication with future reviews.

Pre-registering Clinical Research

Clinical trial registration "is the practice of documenting clinical trials before they are performed in a clinical trials registry so as to combat publication bias and selective reporting" (4). 

Clinicaltrials.gov is the largest database of clinical trials, hosting descriptions and results from privately and publicly funded clinical studies. Many funders and publishers require registration of clinical trials.

General Pre-Registration Tools and Databases

Research studies of all types can be pre-registered using the tools below:

Tools

• Center for Open Science | Open Science Framework Pre-registration 
• As Predicted | Wharton School of the Univ of Pennsylvania Wharton Credibility Lab. Pre-registration

Databases

• Curated list of about 10 pre-registration examples (OSF)
• Search or browse OSF Registries
  • Example nursing pre-registration: The validity of nurse surveys to estimate the nurse-to-patient ratio (skill mix) in general medical and surgical settings: protocol for an observational study (Jan 2021)
  • Example health and medicine pre-registration: Uncovering the causes of increased cardiovascular related mortality in individuals with schizophrenia
• As Predicted (see above) pre-registrations are private; a public copy may be made that produces a URL, but there is not a database of completed registrations.

Read More:

1. Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2600-2606. doi: 10.1073/pnas.1708274114. PMID: 29531091; PMCID: PMC5856500.

2. Standin E. 2019. Open science, pre-registration and striving for better research practices. American Psychological Association:Psychological Science Agenda | November 2019  

3. Gonzales JE, Cunningham CA. 2015. The promise of pre-registration in psychological research. American Psychological Association: Psychological Science Agenda | August 2015.

4. Wikipedia. Preregistration (Science): Clinical trial registration

Registered Reports represent a change to the traditional publishing paradigm. Publishing a Registered Report involves submitting a research protocol to the journal before experiments are conducted. Protocols are reviewed, and once accepted, the journal commits to publishing the outcomes. This commitment to publishing helps to avoid cherry-picking results and foster sharing of all types of outcomes, even negative results.

The process often involves several steps, including:

• Submitting your research question and study design to a journal or publisher
• Stage 1 peer review on your plan and methods
• In principle acceptance (IPA) by the journal for publication
• Conducting the study and writing up the results
• Stage 2 peer review (whether study matches plan)
• Publication

Like pre-registration, Registered Reports supports research transparency and can prevent publication bias and address questionable research practices like p-hacking and HARKing.

Registered Reports Tools

Center for Open Science | Open Science Framework Registered Reports - create a Registered Report (typically done after you receive an IPA)

Sample Registered Reports:

Plos ONE Registered Report Protocols

Sample protocol: Text messaging as a tool to improve cancer screening programs (M-TICS Study): A randomized controlled trial protocol

Nature Human Behavior Registered Reports

Sample report: Dissociating memory accessibility and precision in forgetting (sample of collection of materials that might accompany a registered report) 

Read More:

Chambers C. What's next for Registered Reports? Nature. 2019 Sep;573(7773):187-189. doi: 10.1038/d41586-019-02674-6. PMID: 31506624.

Smith GD, Penny KI. Enhancing research integrity in academic nursing: The introduction of registered reports. J Clin Nurs. 2019 Apr;28(7-8):1037-1038. doi: 10.1111/jocn.14770. Epub 2019 Feb 8. PMID: 30618059.

Journal of Clinical Nursing (JCN) submission guidelines

Gray R, Thompson DR, Tong Chien W, Jones M, Jones A, Moyo N, Waters A, Brown E. Why has the Journal of Psychiatric and Mental Health Nursing stopped publishing registered reports? J Psychiatr Ment Health Nurs. 2020 Dec 18. doi: 10.1111/jpm.12721. Epub ahead of print. PMID: 33336865. Notes the discontinuation by the editorial team. The team responds that the primary reasons for closing registered reports are the lack of these papers submitted to the journal and subsequently the lack of good quality reports published.

Preprints are defined as “a scholarly manuscript posted by the author(s) in an openly accessible platform, usually before or in parallel with the peer review process” (Committee on Publication Ethics [COPE], 2018). They are a version of a scientific paper uploaded by the authors to a public server before it is published in a peer-review journal. The preprint contains complete data and methodologies. 

Benefits of Preprints:

• Increase openness and accessibility of scientific findings
• Increase the speed of dissemination of knowledge
• Enhance collaboration among researchers
• Facilitate proactive sharing
• Improve with feedback from the community.

Preprint Servers and Repositories

Discipline-based Collections

Multidisciplinary Collections and Search Tools

Learn more about Preprints

• Visit our Guide to Preprints to learn more about advantages of preprints for your scholarship, and how to prepare and submit a preprint

Utilizing common practices to manage your research data will help your own workflow and also meet requirements for data management, such as the upcoming 2023 NIH Data Management and Sharing Policy.

See our guide on Research Data Management for practical tips, tricks, and resources.

There are many benefits to employing reproducible practices around your software- and computing-related research processes. They include:

• Batching often-repeated steps, for example with data cleaning multiple files
• Making re-running an analysis with new or changed data much easier
• Changing an analysis (e.g. based on reviewer comments)
• Answering questions about your analysis
• Increasing the reach of your work through re-use 

Practices don't have to be overly complicated. You can:

• Consider using macros in your spreadsheets to document (and repeat) steps in data processing
• Investigate using OpenRefine for cleaning and processing tabular data. Advantages include leaving your raw data untouched, providing robust undo/redo capabilities, and providing the ability to copy and export your change history (all edits made to your data. Read more about OpenRefine.
• Use a programming language like R or Python
• No matter what tool you use, DOCUMENT! Use commenting or markup to note what you've done, and why.

NIH suggests "applicants proposing to use cell lines could describe the method they plan to use to verify the identity and purity of the lines, which might include short tandem repeat (STR) profiling and mycoplasma testing." Below are some resources for writing an authentication plan and procedures for authenticating cell lines.

Guidelines and Resources from Professional Organizations

• International Cell Line Authentication Committee (ICLAC)
• ATCC

Literature Review

• Know Thy Cells: Improving Biomedical Research Reproducibility by Leonard P. Freedman
• Changing the Culture of Cell Culture: Applying Best Practices and Authentication to Ensure Scientific Reproducibility by Leonard P. Freeman, Mark C. Gibson and Richard M. Neve
• Cell Line Authentication Demystified by Vivien Marx
• Standards for Cell Line Authentication and Beyond Jamie L. Almeida, Kenneth D. Cole and Anne L. Plant
• Recommendation of Short Tandem Repeat Profiling for Authenticating Human Cell Lines, Stem Cells, and Tissues by Rita Barallon, et. al.
• A Resource for Cell Line Authentication, Annotation and Quality Control by Mamie Yu, et. al.

Specific for Non-Human Cell Lines

• Authentication for non-human cell lines from ICLAC
• Mouse Cell Line Authentication by Jamie L. Almeida, Carolyn R. Hill and Kenneth D. Cole

Unique IDs for your Antibodies, Model Organisms, and Tools

• Find or generate an Research Resource Identifier (RRID) for named resources in your manuscript to promote research resource identification, discovery, and reuse

Checklists and Validation Methods

• Global Biological Standards Institute (GBSI)
• Approaches to Validation (GBSI)

Resources from Manufacturers

• Improving Reproducibility: Best Practices for Antibodies from Sigma Aldrich

Literature Review

• Antibody Validation" by Jennifer Bordeaux, et. al.
• Standardize Antibodies used in Research by Andrew Bradbury and Andreas Plückthun
• A Proposal for Validation of Antibodies by Mathias Uhlen et. al.

The following resources and guidelines are for researchers who acquire images and use software image like Photoshop.

• "Avoiding Twisted Pixels: Ethical Guidelines for the Appropriate use and Manipulation of Scientific Digital Images" By Douglas Cromey
• "Seeing is Believing? A Beginners' Guide to Practical Pitfalls in Image Acquisition" by Alison North
• Online Learning Tool for Research Integrity and Image Processing from ORI
• "What's in a Picture? The Temptation of Image Manipulation"by Mike Rossner and Kenneth M. Yamada [Specific for blots, gels, and micrographs]

Read more about rigor and reproducibility

• Barriers to replicating cancer biology research: challenges or opportunities? (slides from Tim Errington, Center for Open Science (Cancer Biology Reproducibility Project)

Interested in supporting reproducibility in biomedical research? Here are some ways you can get started (thanks to Tim Errington, Center for Open Science):

• Share your code
• Share reagents (e.g. to addgene)
• Share protocols
• Create a registered report 

Test yourself - take the PLOS Reproducibility Assessment and consider their recommendations such as:

• Use an electronic lab notebook
• Utilize project management approach (e.g. have raw data, analysis, scripts, readme files, methods, manuscripts in an organized directory)
• Have a data management plan
• Pre-register your research
• Publish all findings, including negative and null results
• Think ahead! 10 years from now could you confidently reproduce your work based on materials you made public?

Want to learn more? Watch the webinar "A Guide to Supporting Early Career Researchers in Open Science". Some take-home messages from early-career researchers:

• Start small - you don't have to be doing everything "open". Find one or two practices that work for you, e.g. write preprint or share your codebook or procedures. What feels easiest? (Jesse Fleming, UVA) 

Prepare to meet requirements in January 2023 to submit a Data Management and Sharing Plan with your NIH grant proposal.

Background

[coming soon]

Resources

DMPTool: Web-based tool that provides a template and guidance for creating your plan.

Additional assistance is available to UVA Health System faculty, staff, and students. Just Ask Us - we are happy to provide help via email, Zoom, phone, or in-person.