Abstract: Community-Engaged AI for Youth Homeless Services: A Perspective on Data, Equity and Opportunity (Society for Social Work and Research 31st Annual Conference)

Schedule:

Saturday, January 23, 2027

Union Station 24 (Hilton San Francisco Union Square)

* noted as presenting author

Eric Rice, PhD, Professor, University of North Carolina, Chapel Hill, Chapel Hill, NC

Jaspreet Ranjit, PhD student, University of Southern California, CA

Peichen Liu, MSW, Doctoral student, University of Southern California, Los Angeles, CA

Swabha Swayamdipta, PhD, Assistant Professor, University of Southern California, CA

Lindsay Young, PhD, Assistant Professor, University of Southern California, Los Angeles, CA

Olga Koumoundouros, MFA, PhD Student, University of Southern California, Brooklyn, NY

Laura Onasch-Vera, MSW, Data and Impact Manager, My Friend's Place, CA

Bill Tang, PhD graduate, University of Southern California, CA

Nicole Haseman, MSW, PhD Student, University of North Carolina at Chapel Hill, NC

Hailey Winetrobe Nadel, MPH, CHES, Research Associate, University of North Carolina at Chapel Hill, NC

Brian Rice, MDCM, Clinical Associate Professor in Emergency Medicine, Stanford University, CA

Hsun-Ta Hsu, PhD, Associate Professor, University of North Carolina at Chapel Hill, Chapel Hill, NC

Background and Purpose:
Youth experiencing homelessness (YEH) face disproportionately high rates of depression, trauma exposure, suicidality, and substance use, yet youth homelessness-serving organizations (YHSOs) remain critically under-resourced and often lack the analytic infrastructure necessary to evaluate behavioral health services. Much of the information that could support evaluation and program improvement exists in unstructured case management notes, which cannot be analyzed with statistical methods. The objective of this study is to introduce and operationalize a community-engaged artificial intelligence (AI) framework that leverages large language models (LLMs) to transform narrative case management data into structured indicators of behavioral health engagement, service linkage, and well-being. We believe that community-engaged AI offers a model for using emerging computational tools to advance health equity while preserving the relational values central to social work practice.

Methods:
This project draws on a 20-yearlong ongoing collaboration with My Friend’s Place (MFP), a Los Angeles drop-in center for YEH.

The dataset consists of 4,386 de-identified case management notes collected between July 2022 and June 2023. Prior to analysis, all personally identifiable information was removed using automated anonymization procedures to ensure privacy and compliance with data protection standards.

We have developed a software application which pairs staff with open-source LLMs to iteratively develop structured codebooks aligned with MFP’s theory of change, which emphasizes relationship-building, engagement in behavioral health services, and improved well-being. Initial annotation involved trained human coders labeling a subset of 200 case notes to establish benchmark categories. The LLM then generated predictions for the additional notes, with experts reviewing incorrect outputs and refining coding instructions through successive iterations. Then the system extracts data from the remaining 4,186 notes.

Community participation was embedded throughout the process, including problem definition, taxonomy development, model validation, and interpretation of outputs. This human-in-the-loop workflow was designed to reduce bias, improve contextual accuracy, and ensure that outputs reflect real-world service processes and community priorities.

Results:
This project demonstrates the feasibility of using community-engaged LLM workflows to extract structured behavioral health indicators from narrative case notes within youth-serving organizations. Preliminary findings indicate that LLM-assisted annotation can identify both explicit and implicit references to key service processes, including therapeutic engagement, referrals to mental health services, and progress toward individualized goals.

Iterative collaboration between technical teams and frontline staff substantially improved the relevance and interpretability of extracted indicators. The development of customized codebooks, aligned with MFP’s theory of change, allows for the extraction of quantifiable data from existing free-text case management narratives, suitable for program evaluation and continuous quality improvement.

Conclusions and Implications:
Community participation at every stage of AI development, from problem conceptualization to data labeling and model refinement, is essential to ensure equity, accuracy, contextual relevance, and end-user/community buy-in. Ultimately, our approach to community-engaged AI offers a model for using emerging computational tools to advance health equity while preserving the relational values central to social work practice. This approach positions AI not as a replacement for human expertise but as a collaborative instrument for strengthening behavioral health services for vulnerable youth.