“I'd Say Don't Let It Get Too Much into Your Practice or In between You and Your Clients.” Considering Mental Health Professional’s Folk Theories of Automated Clinical Documentation Systems: A Qualitative Study.

Background and Purpose: AI-assisted clinical documentation tools built on large language models are rapidly expanding in mental health care, offering to reduce documentation burden and support clinical decision-making. However, professional training, regulatory frameworks, and ethical guidance have not kept pace with the deployment of these technologies. This leaves social workers without formal standards for evaluating and integrating AI-enabled documentation systems into practice. Drawing on the concept of "folk theories" or the intuitive, experience-based models people develop to make sense of complex systems as well as Gambrill's framework of "avoidable ignorance," this study investigates how social workers make decisions about whether and how to adopt AI-assisted documentation tools. Specifically, it examines how practitioners' varying levels of AI literacy shape the informal theories they develop about these systems and how those theories, in turn, influence their clinical behaviors around data privacy, consent, and confidentiality.

Methods: This qualitative study employed a constructivist grounded theory approach. Semi-structured interviews were conducted via Zoom with 17 licensed social workers, who had used an AI clinical note-taking tool at least once in the past year. Participants were recruited through a national professional association's message boards and a user group affiliated with an AI note-taking platform. The sample was predominantly female (65%), white (76%), and aged 36–50 (65%). Most had over six years of professional experience. Interviews were transcribed, de-identified, and analyzed using Dedoose software. Coding followed a two-phase process, initial line-by-line coding followed by focused coding to synthesize categories, supplemented by memo writing and constant comparison methods.

Results: Findings revealed three interrelated themes. First, participants demonstrated varying levels of AI literacy, particularly regarding data storage and usage, which created substantial uncertainty about the ethical implications of these tools. Second, in the absence of formal guidance from professional associations or regulatory bodies, practitioners developed folk theories about how AI systems operate, especially regarding data confidentiality, recording practices, and HIPAA compliance, that were often partial or inaccurate. Third, these folk theories actively shaped clinical behaviors: practitioners set boundaries around recording sensitive sessions, devised strategies for anonymizing client data (such as misspelling names or using initials), made deliberate language choices when discussing AI with clients, and developed individualized consent practices. Some practitioners kept AI invisible to clients, while others declined to use recording features altogether.

Conclusions and Implications: Therapists' folk theories represent adaptive professional responses to genuine uncertainty rather than simple misconceptions. However, reliance on informal reasoning in the absence of institutional guidance carries ethical risks, particularly around informed consent and data privacy. The findings highlight the need for a coordinated approach combining AI literacy training for practitioners with enforceable regulatory frameworks that ensure transparency, explainability, and developer accountability. Professional associations, educational programs, and policymakers must work together to provide structured guidance so that clinicians can critically evaluate AI tools rather than relying on improvised theories to navigate an increasingly AI-integrated clinical landscape.