Abstract: Evaluation of a Generative AI Chatbot (Rosie) on Maternal Health Outcomes: A Randomized Controlled Trial (Society for Social Work and Research 31st Annual Conference)

Schedule:

Saturday, January 23, 2027

Golden Gate 4 (Hilton San Francisco Union Square)

* noted as presenting author

Elizabeth Norell, PhD, MSW, LCSW-C, Associate Professor, University of Maryland at College Park, College Park, MD

Xiaohe Yue, Research Analyst, University of Maryland at College Park, College Park, MD

Heran Mane, Data Scientist, University of Maryland at College Park, MD

Francia Ximena Marin Gutierrez, MSW, Research Coordinator, University of Maryland at College Park, College Park, MD

Neha Srikanth, Doctoral Student, University of Maryland at College Park, College Park, MD

Daniel Smolyak, PhD, Postdoctoral Fellow, National Institute of Nursing Research, Bethesda, MD

Pankaj Dipankar, PhD, Postdoctoral Fellow, National Institutes of Health (NIH), Bethesda, MD

Carson Peters, PhD, Research Affiliate, University of Maryland at College Park, College Park, MD

Jordan Boyd-Graber, PhD, Professor, University of Maryland at College Park, College Park, MD

Thu Nguyen, PhD, Associate Professor, University of Maryland at College Park

Xin He, PhD, Associate Professor, University of Maryland at College Park, College Park, MD

Quynh Nguyen, PhD, Intramural Investigator, National Institute of Nursing Research, Bethesda, MD

Background/Objective: Generative artificial intelligence (AI) chatbots are increasingly deployed to support maternal health and reduce health inequities, yet randomized trial evidence of their effectiveness remains limited. We evaluated the effects of a generative AI chatbot application (Rosie) embedded within a health education application on minoritized women’s maternal mental health, health care utilization, pregnancy complications, and related outcomes, and assessed whether effects varied by user engagement and parental status at 12 months after trial enrollment.

Methods: We conducted a two-arm randomized controlled trial that enrolled 400 pregnant or postpartum women (parenting an infant 6 months or younger) from 49 states and D.C. Eligible participants were all members of racial and/or ethnic minoritized groups aged 14 or older who spoke English. Participants were randomized to receive either the Rosie intervention or a monthly children’s book. Primary outcomes included maternal anxiety and depression, measured using the Generalized Anxiety Disorder–7 (GAD‑7) and Patient Health Questionnaire–9 (PHQ‑9). Secondary outcomes included maternal health care utilization (emergency department) and pregnancy-related health complications. Linear, Poisson, and modified Poisson regression models were used to estimate intervention effects, adjusting for maternal age, race and ethnicity, education, income, post‑pregnancy health insurance, and baseline value of the outcome when available. Additional analyses examined differences in outcomes by chatbot engagement level and potential effect modifiers (i.e., parental status, marital status, and infant caregiving status).

Results: Rosie use was not significantly associated with improvements in maternal anxiety or depression at 12 months after enrollment. Mean GAD‑7 and PHQ‑9 scores did not differ between intervention and control groups in either crude or adjusted models, and there were no significant differences in the risk of moderate to severe anxiety or depression. Similarly, there were no statistically significant differences in maternal emergency department visits, or pregnancy-related health complications between groups. Among Rosie users, greater engagement was associated with longer sleep hours (β = 0.50; 95% CI: 0.01, 0.99), but higher anxiety scores (β = 1.63; 95% CI: 0.18, 3.09). Among never married and separated women, Rosie users had higher adjusted rates of emergency room visits than book club participants (aRR = 3.22, 95% CI: 1.36, 7.61), whereas no significant effect was observed among married women. Conversely, among married women, Rosie users were more likely to use effective contraception (aPR = 1.26, 95% CI: 1.06–1.51), with no significant effect among never married or separated women.

Conclusions: Access to a generative AI chatbot did not significantly improve maternal mental health, health care utilization, or pregnancy complications at 12 months after study enrollment. However, Rosie's intervention effects were heterogeneous, with engagement intensity, pregnancy and parenting stage, and social context shaping observed outcomes. Prior studies of Rosie indicate improved depression, anxiety, and stress at 3 months, indicating measurement timing importance in assessing intervention effects, which may provide short term benefits during a sensitive developmental period.