Generalizability of clinical prediction models in mental health

Richter, M.; Emden, D.; Leenings, R.; Winter, N. R.; Mikolajczyk, R.; Massag, J.; Zwiky, E.; Borgers, T.; Redlich, R.; Koutsouleris, N.; Falguera, R.; Thanarajah, S. E.; Padberg, F.; Reinhard, M. A.; Back, M. D.; Morina, N.; Buhlmann, U.; Kircher, T.; Dannlowski, U.; PRONIA consortium; FOR2107 consortium; MBB consortium; Hahn, T.; Opel, N.

Zusammenfassung

Concerns about the generalizability of machine learning models in mental health arise, partly due 46 to sampling effects and data disparities between research cohorts and real-world populations. We 47 aimed to investigate whether a machine learning model trained solely on easily accessible and low- 48 cost clinical data can predict depressive symptom severity in unseen, independent datasets from 49 various research and real-world clinical contexts. This observational multi-cohort study included 50 3021 participants (62.03% females, MAge= 36.27 years, range 15 - 81) from ten European research 51 and clinical settings, all diagnosed with an affective disorder. We firstly compared research and 52 real-world inpatients from the same treatment center using 76 clinical and sociodemographic 53 variables. An elastic net algorithm with ten-fold cross-validation was then applied to develop a 54 sparse machine learning model for predicting depression severity based on the top five features 55 (global functioning, extraversion, neuroticism, emotional abuse in childhood, and somatization). 56 Model generalizability was tested across nine external samples. The model reliably predicted 57 depression severity across all samples (r = 0.60, SD = 0.089, p < 0.0001) and in each individual 58 external sample, ranging in performance from r = 0.48 in a real-world general population sample 59 to r = 0.73 in real-world inpatients. These results suggest that machine learning models trained on 60 sparse clinical data have the potential to predict illness severity across diverse settings, offering 61 insights that could inform the development of more generalizable tools for use in routine 62 psychiatric data analysis.