Predicting Injuries: Are we using the appropriate statistical approach?

Purpose: Musculoskeletal injury prediction is complex and multifactorial. Insights from elite sports suggest prediction will never be possible and that “screening is dead”. In sport, after more than 20 years of attempts to identify risk factors for injury and implement preventive countermeasures, few studies have identified sustained injury reductions. Injury risk is contextual, and changeable with time and subsequent events, so it is unlikely that analyses of isolated variables will yield meaningful predictions. Recent utilization of machine learning tools has demonstrated the predictive capacity of these sophisticated analysis methods. As military organisations gear up to address the injury burden, lessons need to be learned. Is screening dead or do we need to step up the ante? Methods: A structured review of prediction modelling studies indexed on PubMed in the last 5 years was conducted. Comparative analysis of traditional reductionist modelling and non-linear complex modelling was performed. Traditional reductionist modelling included conditional logistic regression and correlation analyses. Complex modelling included recursive partitioning, decision tree, random forest, nearest neighbour algorithm (kNN), Support Vector Machine (SVM), naïve bayes, artificial neural networks, and boosting algorithms. Results of these studies were compared and contrasted in terms of gains in classification accuracy, sensitivity and specificity. Results: Of 327 papers found in the initial search, 30 were selected for analysis. Studies aimed to predict a wide range of complex phenomena from trauma or disease survival, Dementia and Alzheimer’s onset, to motor accident prediction. 26 of 30 papers (89%) demonstrated improved predictive accuracy of complex models over reference reductionist models. Accuracy reportedly improved by an average of 8% (range -4% to 33%). Manipulations of models to tune sensitivity and specificity requirements frequently resulted in variations of accuracy. Interpretability of some of the complex modelling techniques was frequently reported by authors as a limitation to clinical utility. Conclusion: Predictive accuracy was improved by up to 33% when using complex modelling techniques in the included studies. The first step towards targeted injury prevention is to accurately predict who is at risk. The complexity of injury may require the use of modelling methods that can enhance pattern recognition within a web of non-linear interactions. Complex modelling methods should be explored if future injury prevention efforts are to be more successful.