Statistical significance and orthopaedic traumatology

Abstract

Purpose of Study: A recent proposal suggests changing the threshold for statistical significance from a P value of .05 to .005 to minimize bias and increase reproducibility of future studies. The present study explores how lowering the P value threshold would affect the interpretation of previously published trauma orthopaedic randomized clinical trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold.

Methods: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion.

Results: Of 75 articles retrieved, 49 were included. We identified 117 primary endpoints from 49 trials: 41 endpoints (35.0%) had a P value less than .05 and 76 (65.0%) had a P value greater than .05. Overall, 41.5% (17/41) of statistically significant primary endpoints were less than .005. Of the 117 primary endpoints, only 17 (14.5%) of the endpoints were less than .005, and would hold significance with the proposed threshold. Only 6.12% (3/49) of the included studies had all primary endpoints that met the new threshold of .005.

Conclusion: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the impact it may have on orthopaedic practice.