Analysis of practices to promote reproducibility and transparency in anesthesiology research: Are important aspects

Abstract

Introduction: Reliable, high quality research is essential to the field of anesthesiology. Investigating reproducibility and transparency has been accomplished broadly in the biomedical domain and in the social sciences; however, practices that promote reproducibility and transparency have never been evaluated in the anesthesiology research community. In this study, we applied 14 indicators of reproducibility to evaluate the current climate of the anesthesiology research community.

Methods: We used the National Library of Medicine (NLM) catalog to search for all journals using the subject terms tag Anesthesiology[ST]. The inclusion criteria required that journals provided full-text publications in English and were MEDLINE indexed. The list of journals in the NLM catalog fitting the inclusion criteria were then extracted using the electronic International Standard Serial Number (ISSN). This series of ISSN were used in a PubMed search to identify all publications within these journals. We then limited the sample to publications from January 1, 2014 to December 31, 2018. Subsequently, we randomly sampled 300 publications that fit the inclusion criteria for our analysis. Data extraction was then conducted in a blinded, duplicate fashion using a pilot-tested Google form.

Results: The PubMed search of these journals identified 171,441 publications, with 28,310 being within the time-frame. From the 300 publications sampled, 296 (296/300, 98% [97% to 99%]) full text publications were obtained, while 4 (4/300, 1% [0% to 3%]) only showed the abstract or could not be accessed. Most (104/107, 97% [95% to 99%]) of the studies did not include material availability statements or protocol availability statements. For the analysis scripts, the majority of publications (121/122, 99% [98% to 100%]) did not provide a data analysis script statement. The majority (94/122, 77% [72% to 81%]) of the publications did not contain a pre-registration statement. Other study characteristics were found to be insufficient.

Conclusion: Anesthesiology research needs to drastically improve with regards to reproducibility and transparency. By making research easily accessible online and by improving the accessibility of detailed components (raw data, materials and protocols, analysis scripts) primary research can be reproduced in subsequent studies and help contribute to the development of new practice guidelines, helping change patient care through evidence-based conclusions.