Practices to increase the diversity of the citations (aka references) in published works are needed because of the bias in citation practices where papers by women and people of color are under-cited relative to the number of papers in a field or discipline.

This guide provides links to resources and tools to help support inclusive citation practices to address citation inequities.

These tools can help you audit your citations. To use them, your file of citations/references must be in BibTeX format (.bib) Citation managers like Zotero, EndNote, etc. can export citations as a .bib file. Other literature databases like Web of Science can as well. For those that don't have that option, like PubMed, you can export to citation manager style, import into your citation manager, and export as .bib.

If you have a manuscript with references that aren't in a citation manager, you can use this Citation Finder to scrape the references from your document and retrieve them in PubMed. You can import the PubMed references into your citation manager and then export as a .bib.

• Gender Diversity Statement and Code Notebook aka CleanBib (gender and race/ethnicity)
  • If you have your list of references in your citation manager (e.g. Zotero), you should be able to export them as a BibTex file and follow the instructions to run the code
  • If you want to analyze a past paper of yours, find the paper in Web of Science. Click on the link to the right for the References. Export these as a BibTeX file. NOTE as of 12/14, this method required a work-around to work in CleanBib. Please reach out and we will assist.
  • Uses Gender API and ethnicolr data
• GCBI-alyzer  (gender)
  • Uses genderize.io. You can also search genderize.io itself for the probabilistic gender of first names, either across the globe or in a specific country.
• Citation Transparency browser plugin (gender)

What about generative AI? Can it help?

For gender: Goyanes, de-Marcos, and Dominguez-Diaz (2024) offer a step-by-step research guide, with prompt examples and detailed clarifications, to automatically classify the gender from names through ChatGPT (as well as two partially free gender detection tools). They refer to previous works noting ChatGPT's stability and performance with this task.

ChatGPT may note reluctance to produce results for race and ethnicity, and state "It's great that you want to support underrepresented scholars in your research citations. However, inferring an author’s race based solely on names or publication lists raises ethical and privacy issues, and it can be highly inaccurate." It's wise to consider this response. If you do choose to use ChatGPT or another generative AI tool, follow up on its responses. Ask it where it is getting its data to formulate that response, and can it share any probabilities for its results.

How can you find additional citations and be more representative of all scholars? Consider these tips:

• Examine your content, especially your Background and Introduction. Have you used the same citations for years? Are there others you could consider adding?
• Have you conducted a fresh literature search to reveal different or additional publications?
  • Try literature databases such as Web of Science, Scopus, or Embase
    • From your results page, use the database features to sort or facet your results to reveal different publications (see below for tips)
• Can you identify under-represented authors through their communities? (see below)
• What other practices are useful? Please share!

Many library databases have options to display your results differently. This can help highlight additional papers and provide a path to diversify your citations. Consider these tips:

1. Perform a search in a database below and use the tips to display your results.

Scopus

After performing your search, you can use the Analyze results to get a visual representation of your results:

From there, the "cards" display gives you options to see your results: click on the double-headed arrow at the top right of the card you are interested in:

 

Embase

After performing your search, use the menus at left to refine your results:

• Click on the Sources option. Select only Embase references by clicking on the Embase circle then Apply. This will omit Medline/PubMed references (assuming you've already done a PubMed search). Note: Embase also includes Conference Abstracts - if you'd like to omit them you can use the Publication Types menu, or add NOT 'conference abstract'/it to your search string.
  
  

Web of Science

After performing your search, use the menus at left to refine your results.

• Consider menus such as those for affiliation, country, and/or funder. Use the see all to see options for refining your search (or exporting results to analyze further.
  
  

2. After you've found additional citations, use the Tools above to audit those sources for author characteristics.

Guides

• Citation Justice - Diversity, Equity, and Inclusion in Research (U Maryland Libraries)
  • See their section on Finding Diverse Voices” for more how-to’s
  • Their FAQ section addresses common questions
• Citation Diversity (Dalhousie University)
• Inclusive Citation (Rowan University)

• The Importance of Citational Justice (U Arizona Libraries)

Readings

Charpignon ML, Matos J, Nakayama L, Gallifant J, Alfonso PGI, Cobanaj M, Fiske A, Gates AJ, Ho FDV, Jain U, Kashkooli M, McCoy LG, Shaffer J, Link Woite N, Celi LA. Does diversity beget diversity? A scientometric analysis of over 150,000 studies and 49,000 authors published in high-impact medical journals between 2007 and 2022. medRxiv [Preprint]. 2024 Mar 22:2024.03.21.24304695. doi: 10.1101/2024.03.21.24304695. PMID: 38562711; PMCID: PMC10984076.

Coalter, Jodi H. "Citation Power: Overcoming Marginalization One Citation at a Time." Perspectives on Justice, Equity, Diversity, and Inclusion in Libraries. IGI Global, 2023. 62-77.

Domínguez-Díaz A, Goyanes M, de-Marcos L, Prado-Sánchez VP. Comparative analysis of automatic gender detection from names: evaluating the stability and performance of ChatGPT versus Namsor, and Gender-API. PeerJ Computer Science. 2024 Oct 17;10:e2378.

Else H, Perkel JM. The giant plan to track diversity in research journals. Nature. 2022 Feb 24;602(7898):566-70.

Fioranelli GA, Lo Y, Jesch AK, Laluzerne MJ, Donnelly D, Lyon SM, Dingle AM. Diversity Drives Representation: An Internal Audit of Gender Representation in Citation Practices of a Single Surgical Laboratory. Plast Reconstr Surg Glob Open. 2024 Jun 14;12(6):e5823. doi: 10.1097/GOX.0000000000005823. PMID: 38881963; PMCID: PMC11177816.

Kozlowski D, Larivière V, Sugimoto CR, Monroe-White T. Intersectional inequalities in science. Proc Natl Acad Sci U S A. 2022 Jan 11;119(2):e2113067119. doi: 10.1073/pnas.2113067119. PMID: 34983876; PMCID: PMC8764684.

Kozlowski D, Murray DS, Bell A, Hulsey W, Larivière V, Monroe-White T, Sugimoto CR. Avoiding bias when inferring race using name-based approaches. PLoS One. 2022 Mar 1;17(3):e0264270. doi: 10.1371/journal.pone.0264270. PMID: 35231059; PMCID: PMC8887775.

Lockhart JW, King MM, Munsch C. Name-based demographic inference and the unequal distribution of misrecognition. Nat Hum Behav. 2023 Jul;7(7):1084-1095. doi: 10.1038/s41562-023-01587-9. Epub 2023 Apr 17. PMID: 37069295.

Michelle A, Kelly L, Kaushar M, Marcus Emmanuel B, Rogan G. Gender Inference: Can ChatGPT Outperform Common Commercial Tools?. In Proceedings of the 33rd Annual International Conference on Computer Science and Software Engineering 2023 Sep 11 (pp. 161-166).

Zurn P, Teich EG, Simon SC, Kim JZ, Bassett DS. Supporting academic equity in physics through citation diversity. Communications Physics. 2022 Sep 27;5(1):240.