Detecting Early Suicide Risk from Social Media with a Multi-Agent LLM System

Background and Purpose: Early identification of suicide risk is potentially lifesaving but remains difficult in routine clinical and community contexts, especially when people disclose distress online before engaging formal services. For social work and mental health practice, scalable tools that support earlier, transparent risk screening may strengthen prevention and outreach. This study developed and evaluated a multi-agent large language model (LLM)-based system for detecting early suicide risk signals from social media data aligned with the Columbia-Suicide Severity Rating Scale (C-SSRS), and compared its performance and interpretability with established benchmarks.

Methods: We conducted a retrospective evaluation using a clinician-annotated Reddit dataset comprising 15,755 posts from 500 users sampled from the r/SuicideWatch community. Each user was assigned one of five C-SSRS-aligned labels: supportive, indicator, ideation, behavior, or attempt. The proposed system consisted of three LLM-based agents with distinct roles for screening, evidence extraction, and adjudication. Performance was evaluated on a held-out test set using weighted F1-score and accuracy and compared with single-model LLM baselines and a previously published CNN benchmark under identical experimental conditions.

Results: The multi-agent system achieved a weighted F1-score of 0.71 and an accuracy of 0.71, outperforming single-model LLM baselines (weighted F1-scores of 0.60 or lower) and the CNN benchmark (weighted F1 = 0.64; accuracy = 0.56). The system generally distinguished broad risk strata well, while misclassifications occurred primarily between adjacent risk levels, consistent with the gradual nature of suicide risk expression in social media. It also generated explicit intermediate outputs and evidence-based rationales, enabling reviewable user-level assessments.

Conclusions and Implications: This multi-agent framework improved suicide risk detection while increasing transparency relative to single-model approaches. By aligning automated classification with a clinically grounded taxonomy and producing reviewable evidence, it offers a scalable decision-support approach for social workers, mental health providers, and digital service teams seeking earlier and more equitable identification of people at elevated risk. The findings contribute to social work research on AI-enabled prevention and assessment in community and online settings and support rigorous, evidence-informed intervention for the common good.