Training Social Workers As Policy Actors: Evaluation of a Generative AI-Based Multi-Agent Simulator for Policy Advocacy

Background and Purpose: Social workers are professionally mandated to influence policy and advance social justice. However, traditional pedagogical methods, such as classroom lectures or peer role-plays, often lack the scalable capacity to provide repeated practice in realistic, high-pressure policy contexts. This study reports on the development and preliminary evaluation of a Generative Artificial Intelligence (GenAI)-based multi-agent simulator designed to train licensed social workers in formulating and defending position papers before policymakers.

Methods: Using a mixed-methods pre-post design, 40 licensed social workers enrolled in a master’s program participated in a single-session simulation. The simulator, deployed on the CESURA AI platform and powered by Gemini 2.5 Flash, comprised three virtual agents: a policy expert with a dynamic adaptive-challenge mechanism that adjusts resistance based on the rigor of the social worker’s arguments, a structured feedback agent, and a reflective guidance agent facilitating dialogic processing. Self-efficacy was measured pre- and post-intervention using a four-item composite scale. Motivation was assessed via a single-item measure. User experience was evaluated through perceived contribution ratings of each simulator component and learning preference questions. Qualitative data from three open-ended questions were analyzed via thematic analysis.

Results: A paired-samples t-test revealed a statistically significant increase in professional self-efficacy from pre-intervention (M = 6.38, SD = 1.61) to post-intervention (M = 6.95, SD = 2.00; t(39) = 2.36, p = .012, Cohen’s d = 0.37), corroborated by a Wilcoxon signed-rank test (Z = 2.22, p = .027). Motivation remained stable (Cohen’s d = 0.06), suggesting a possible ceiling effect given high baseline scores. User experience ratings indicated that 62.5% of participants rated the virtual policy expert interaction as high or very high in pedagogical contribution, followed by structured feedback (64.1%) and dialogic feedback (63.1%). Eighty-five percent expressed willingness to use this method again, and 60% preferred it over traditional lectures. Qualitative thematic analysis yielded three themes: (1) an emotional trajectory from initial frustration to professional growth; (2) the simulator as a diagnostic mirror for surfacing knowledge gaps; and (3) the pedagogical value of sustained critical resistance from the AI agent.

Conclusions and Implications: This study demonstrates how GenAI-based simulation can serve as a scalable, low-cost method for teaching policy advocacy skills to social workers. By providing a safe-to-fail environment, the simulator enables repeated, accessible practice that can potentially be adapted across languages and cultural contexts, broadening access to evidence-based training at scale. These findings carry implications for social work curriculum design, positioning AI-mediated simulation as a practical complement to traditional instruction in preparing social workers for active policy engagement.