Centering Human Judgment In Evidence Synthesis: A Multi-Agent AI Approach to Literature Screening for Social Work Research

Background and Purpose: Evidence-informed social work depends on timely and credible research synthesis to inform practice, policy, and education. Yet title and abstract screening in systematic reviews remains labor-intensive, difficult to standardize, and increasingly hard to sustain as evidence volume grows. Recent advances in artificial intelligence (AI) have created new opportunities to support screening, but social work research requires approaches that preserve transparency, deliberation, and human accountability. This study developed and evaluated SR-MAS, a multi-agent large language model workflow designed to support rigorous, human-centered literature screening.

Methods: SR-MAS was designed to mirror team-based review by generating three independent screening judgments for each record, prioritizing sensitivity, specificity, or balance, and then integrating their rationales into a final include or exclude recommendation. Records that remained uncertain were advanced to human full-text review rather than automatically excluded. The system was evaluated on a 500-article test set drawn from 5 published systematic reviews across health domains, with 100 records per review, including 20 studies originally included and 80 excluded by human reviewers.

Results: Across datasets, SR-MAS achieved weighted recall of 0.85, weighted precision of 0.90, and F1-score of 0.86 while processing each article in under 30 seconds. Performance advantages were most apparent when eligibility criteria were less clearly defined, where the multi-agent workflow maintained more consistent detection of relevant studies than simpler automated approaches. The system also retained more borderline records, reflecting a deliberate design choice to reduce false exclusion at the preliminary screening stage and preserve human oversight where judgment was needed most.

Conclusions and Implications: This study suggests that AI can function as supportive research infrastructure rather than a substitute for professional judgment. By reducing repetitive screening labor and making disagreement visible, SR-MAS may help social work researchers redirect time toward interpretation, ethical reflection, and the translation of evidence into practice and policy contexts. More broadly, human-centered screening workflows may strengthen equitable access to evidence synthesis capacity, especially for teams with limited resources, therefore support social work research that is more accountable and responsive to the common good.