Behavioral Health Referrals In Patients with Substance Use Disorder Reflect Patient Needs, or Clinician Avoidance As Expressed through Unprofessional Language Use?: A LLM-Augmented, Human-Validated Natural Language Processing Approach.

Background and Purpose

Stigma in healthcare settings remains a key barrier to care for patients with substance use disorders (SUDs). Unprofessional language in clinical documentation, including stigmatizing terms, passive judgments, and challenges to patient credibility, may shape clinician perceptions and care decisions. Whether behavioral health referral patterns reflect clinical need or clinician avoidance driven by documentation bias remains poorly understood. This study developed and validated a natural language processing pipeline, enhanced by large language models (LLMs) and human expert review, to detect unprofessional language; quantify its prevalence alongside behavioral health referrals; and examine their association among patients with SUD.

Methods

This retrospective cohort study analyzed discharge summaries from MIMIC-IV, a de-identified electronic health record (EHR) dataset from emergency department and intensive care unit admissions at a large academic medical center (2008-2019). Patients with SUD (N = 82,012) were identified from 260,347 adults using International Classification of Diseases codes. A rule-based natural language processing algorithm was expanded using three LLMs (GPT-4, Claude 3, Llama-3), with generated terms reviewed by a five-member expert panel requiring four-of-five endorsement on relevance and clinical realism. Multivariable logistic regression estimated adjusted odds ratios (aORs) with 95% confidence intervals (CIs), controlling for demographics, comorbidities, insurance, prior hospitalization, and note length.

Results

Unprofessional language was more prevalent in SUD notes than non-SUD notes (69.1% vs. 51.2%, p < 0.001). The LLM-enhanced model achieved an F-score of 0.91, a 2.5% improvement over baseline (F-score = 0.89), with 45 of 135 candidate terms meeting both criteria. Behavioral health referrals were documented for 23.5% of SUD patients versus 14.9% without (p < 0.001). After adjusting for covariates, unprofessional language was independently associated with higher odds of referral (aOR = 1.54; 95% CI: 1.47-1.61). Psychiatric medication use disorder (aOR = 1.65) and stimulant use disorder (aOR = 1.45) showed the strongest subtype associations. Widowed (aOR = 1.48) and single patients (aOR = 1.38) had higher referral odds than married patients. Psychoses (aOR = 2.38) and depression (aOR = 1.52) were the strongest clinical predictors (all p < 0.05).

Conclusions and Implications

Unprofessional language is prevalent in SUD documentation and independently predicts behavioral health referral. While referral is often appropriate for patients with SUD, its association with unprofessional language suggests referral may sometimes precede rather than follow assessment of what the treating team can provide internally, including medications for addiction treatment, brief intervention, and withdrawal management. When avoidance drives referral, patients may bypass evidence-based options within the treating team's scope. Biased documentation also travels across care transitions, priming subsequent providers and perpetuating inequitable treatment. These findings call for stigma-free documentation training, EHR-integrated alerts for biased language, and communication standards co-developed with patients with lived SUD experience. Future research should assess referral appropriateness and extend these methods to community settings where social workers are primary providers.