Externally validated machine learning algorithm accurately predicts medial tibial stress syndrome in military trainees: a multicohort study

Objectives Medial tibial stress syndrome (MTSS) is a common musculoskeletal injury in both sporting and military settings. No reliable treatments exist, and reoccurrence rates are high. Prevention of MTSS is critical to reducing operational burden. Therefore, this study aimed to build a decision-making model to predict the individual risk of MTSS within officer cadets and test the external validity of the model on a separate military population. Design Prospective cohort study. Methods This study collected a suite of key variables previously established for predicting MTSS. Data were obtained from 107 cadets (34 women and 73 men). A follow-up survey was conducted at 3 months to determine MTSS diagnoses. Six ensemble learning algorithms were deployed and trained five times on random stratified samples of 75% of the dataset. The resultant algorithms were tested on the remaining 25% of the dataset, with models then compared for accuracy. The most accurate new algorithm was tested on an unrelated data sample of 123 Australian Navy recruits to establish external validity of the model. Results Calibrated random forest modelling was the most accurate in identifying a diagnosis of MTSS; (area under curve (AUC)=98%, classification accuracy (CA)=96%). External validation on a sample of Navy recruits resulted in comparable accuracy; (AUC=95%, CA=94%). When the model was tested on the combined datasets, similar accuracy was achieved; (AUC=92%, CA=91%). Conclusion This model is highly accurate in predicting those who will develop MTSS. The model provides important preventive capacity which should be trialled as a risk management intervention.