A Human-Centered Framework for Evaluating Llms for Generalized Anxiety Disorder

Background and Objective. Large language models (LLMs) are increasingly being used in mental health contexts, including anxiety support, where they can generate real-time, conversational responses. While prior work has focused on evaluating these systems in terms of accuracy, safety, and coherence, less attention has been given to how well they communicate emotional support, empathy, and ethical responsiveness. This gap is especially important for social work, where practice is grounded in relational, ethical, and context-sensitive communication. Existing evaluation approaches often rely on predefined annotation schemes and treat disagreements between annotators as errors to be resolved. However, in emotionally complex contexts like anxiety support, disagreement may reflect differences in interpretation rather than mistakes. The objective of this study was to develop a human-centered evaluation framework for LLM responses by examining annotator disagreement as a source of insight, and to consider how such a framework can inform social work practice and technology standards.

Methods. We conducted a qualitative secondary analysis of 1,367 on two LLM-generated responses (ChatGPT and LLaMA) to anxiety-related prompts that had been previously annotated across multiple dimensions, including empathy, tone, and helpfulness. While we resolved disagreements through majority voting initially, in this study we focused on cases where annotators disagreed. These cases were reviewed iteratively to identify patterns in how different annotators interpreted the same response. Reflexive thematic analysis was used to examine sources of ambiguity and inconsistency in the annotation process. Through this process, we refined existing constructs and developed clearer definitions and decision-making heuristics, guided by human-centered evaluation principles and social work values.

Results. Disagreement between annotators was most common in dimensions that required subjective interpretation, particularly empathy, appropriateness of advice, and practical relevance. Several recurring patterns emerged. These included (1) narrative confusion, where responses were unclear or internally inconsistent; (2) affective ambiguity, where tone could be interpreted as either supportive or dismissive or could be interpreted in multiple ways; and (3) tone misalignment, where the intended message did not match how it was perceived. These patterns highlighted gaps in the original annotation framework and informed the refinement of seven core constructs. The final framework includes revised definitions, guiding heuristics, and examples to support more consistent and context-sensitive evaluation, while still allowing room for interpretation.

Discussion and Implications. This study reframes annotator disagreement as a methodological asset and uses it to build a more rigorous, human-centered evaluation approach for LLMs in mental health contexts. The proposed framework moves beyond technical metrics to operationalize qualities that matter in practice, such as empathy, authenticity, and practical advice. For social work, this offers a way to assess whether emerging AI tools align with core professional values. The framework also has implications for policy, including informing updates to NASW technology standards by translating abstract ethical principles into actionable evaluation criteria. As AI tools become more embedded in mental health services, evaluation approaches that center human experience will be essential to ensuring these systems support, rather than undermine, quality of care.