Predicting Treatment Completion Among Individuals with Opioid Dependence Using Machine Learning

Background: Opioid use disorder (OUD) remains a major public health concern in the United States, affecting approximately 2–3 million individuals and significantly contributing to roughly 80,000 annual drug overdose deaths (CDC, 2026). Treatment dropout is common and undermines recovery outcomes. Identifying individuals at risk of non-completion is critical for improving retention. Traditional statistical approaches (e.g., logistic regression) rely on prespecified relationships and may not fully capture complex, nonlinear interactions in large administrative datasets. Various machine learning (ML) methods can flexibly model such patterns and have shown promise in substance use research (Acion et al., 2017; Ellis et al., 2019).

Purpose: This study compares multiple ML models to predict treatment completion versus dropout among individuals with opioid dependence, using logistic regression as a benchmark.

Methods: Data were drawn from the 2023 Treatment Episode Data Set—Discharges (TEDS-D), a national administrative dataset of substance use treatment discharges in the United States. The unit of analysis was the treatment episode. The analytic sample included 160,584 episodes among individuals with opioid dependence, with outcomes classified as completion or dropout.

Predictors included demographics (age group, sex, race/ethnicity), socioeconomic indicators (education, employment, income source), and clinical/behavioral factors (frequency of use, prior treatment, co-occurring mental health problems, criminal justice involvement, living arrangement, medication-assisted treatment, and waiting time). Categorical variables were dummy coded, and missing was included as an independent category. Data were randomly split into training (70%) and test (30%) sets. Analyses were conducted in Python using pandas, NumPy, and scikit-learn.

Five models were evaluated: logistic regression, decision tree, random forest, histogram-based gradient boosting, and multilayer perceptron (MLP). Model performance was assessed using area under the receiver operating characteristic curve (AUC), balanced accuracy, F1 score, sensitivity, and specificity.

Results: The sample reflected substantial socioeconomic and clinical vulnerability, including a high prevalence of co-occurring mental health problems (~43%), prior treatment history, and unstable income sources. Across models, discrimination—the ability to distinguish completion from dropout—was consistently good (AUC = 0.85–0.88). Logistic regression achieved an AUC of 0.85, with sensitivity of approximately 0.74 (correctly identifying completion) and specificity of 0.72 (correctly identifying dropout), corresponding to a balanced accuracy of 0.73. ML models showed modest but consistent improvements: random forest, gradient boosting, and MLP achieved AUC values near 0.88 and balanced accuracy around 0.76. The decision tree showed minimal improvement.

Variable importance analyses indicated that education was the most influential predictor (over 60% of total importance in the decision tree), followed by treatment setting (~15%), income (~4%), medication-assisted treatment (~3%), and waiting time and living arrangement (~2–3%). Age, prior treatment, and arrest history each contributed less than 2%.

Discussion: ML models modestly outperformed logistic regression, suggesting benefits from capturing nonlinearities and interactions. However, performance remained moderate, indicating that key determinants of treatment completion may not be fully captured in the data examined.

Implications: ML models using large administrative datasets can support risk stratification and targeted interventions to improve retention. Future work should incorporate longitudinal and more extended contextual data, validate models externally, and explore integration into clinical decision-making.