Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: a machine learning approach

Background: At present, no tools exist to estimate objectively the risk of poor treatment outcomes in patients with first-episode psychosis. Such tools could improve treatment by informing clinical decision-making before the commencement of treatment. We tested whether such a tool could be successfully built and validated using routinely available, patient-reportable information.

Methods: By applying machine learning to data from 334 patients in the European First Episode Schizophrenia Trial (EUFEST; International Clinical Trials Registry Platform number, ISRCTN68736636), we developed a tool to predict poor versus good treatment outcome (Global Assessment of Functioning [GAF] score ≥65 vs GAF <65, respectively) after 4 weeks and 52 weeks of treatment. To enable the unbiased estimation of the predictive system's generalisability to new patients, we used repeated nested cross-validation to prevent information leaking between patients used for training and validating the models. In pursuit of everyday clinical applicability, we retrained the 4-week outcome predictor with only the top ten predictors of the pooled prediction system and then tested this tool in 108 independent patients with 4-week outcome labels. Discontinuation and readmission to hospital events in patients with predicted poor versus good outcomes were assessed with Kaplan-Meier log-rank analyses, whereas generalised linear mixed-effects models were used to investigate the GAF-based predictions against several clinically meaningful outcome indicators, including treatment adherence, symptom remission, and quality of life.

Findings: The generalisability of our outcome predictions were estimated with cross-validation (test-fold balanced accuracy [BAC] of 75·0% for 4-week outcomes and 73·8% for and 52-week outcomes), and leave-site-out validation across 44 European sites (BAC of 72·1% for 4-week outcomes and 71·1% for 52-week outcomes). We identified a smaller group of ten predictors still providing a BAC of 71·7% in 108 patients never used for model discovery. Unemployment, poor education, functional deficits, and unmet psychosocial needs predicted both endpoints, whereas previous depressive episodes, male sex, and suicidality additionally predicted poor 1-year outcomes. 52-week predictions identified patients at risk for symptom persistence, non-adherence to treatment, readmission to hospital and poor quality of life. Specifically among these patients, amisulpride and olanzapine showed superior efficacy versus haloperidol, quetiapine, and ziprasidone.

Interpretation: Our results suggest that prognostic models operating on brief, patient-reportable pre-treatment data might provide useful insight into individualised outcome trajectories, optimising treatment selection, and targeted clinical trial designs. To embed these tools into real-world care, replication is needed in external first-episode samples with overlapping variables, which are not available in the field at present.

Funding: The European Group for Research in Schizophrenia.