A Psychometric Evaluation of AI Social Work Task Application Scale, a Task-Based Measure of AI Use In Social Work

Introduction: AI large language models (AI LLM) are increasingly used in social work practice, yet there is a lack of validated measures that assess task-specific acceptance of AI LLM in everyday social work activities, rather than general attitudes toward AI adoption. This study develops and validates the AI Social Work Task Application Scale (AISWTAS), which assesses the extent to which AI LLM are applied to specific social work tasks. The AISWTAS includes 24 items across three domains: documentation assistance (e.g., creating a handout of school attendance policies), communication assistance (e.g., drafting a professional email to a housing provider regarding a client referral), and expanded access to information (e.g., suggesting Excel formulas for data analysis). This study provides psychometric evidence of the validity and reliability of the AISWTAS.

Methods: Participants included 121 individuals: 36 faculty and staff(29.8%) 64 students(52.9%), 21 practicum supervisors(17.4%) recruited in 2026. Among them, 46.3% (N=56) reported currently practicing in the field, including 38.9% of faculty and staff and 32.8% of students. Validity measures included frequency of AI LLM use for routine tasks and constructs from the Technology Acceptance Model 2(TAM2): perceived usefulness, perceived ease of use, and intention to use. AISWTAS items were developed led by a school-level committee at a large public university in Arizona to ensure face validity. Responses were rated on a 5-point Likert scale(1 = not at all to 5 = absolutely) as to their willingness to use AI LLM for a specific social work task. Reliability was assessed using Cronbach’s alpha. Concurrent validity was examined using Pearson’s correlations with TAM2’s constructs. Known-groups validity was tested using independent t-test comparing frequent users(more than once per week) and less frequent users. Statistical significance was set at p < .05.

Results: The mean AISWTAS score was 2.79(SD=1.26), indicating slightly below moderate willingness to use AI LLM across tasks. Faculty and staff showed the highest average of AISWTAS(mean=2.98, SD=0.98) following by students(mean=2.77, SD=1.19) and practicum supervisors(mean=2.25; SD=1.16). Internal consistency was strong for the total scale(α=.98) and subdomains: documentation assistance(α=.94), communication assistance(α=.93), and expanded access to information(α=.95). AISWTAS was positively correlated with perceived usefulness(r=.77; p<.001), perceived ease of use(r=.33; p<.001), and intention to use(r=.78; p<.001), supporting concurrent validity. Frequent users(n=37) reported significantly higher AISWTAS scores(M=3.35; SD=1.02) than less frequent users(n=83; M=2.45; SD=1.00), supporting known-groups validity(p<.001).

Discussion and Conclusions: The AISWTAS demonstrates strong reliability and validity as a measure of willingness to apply AI LLM to social work tasks. The scale offers three core functional domains of AI use and provides a practical framework for assessing task-level adoption. It may also serve as a reference for guiding appropriate AI integration in social work practice and education.