Calculating and Interpreting Effect Sizes In Single Subject Research Design Using R

Rigorous single-subject research designs are increasingly recognized as foundational to building practitioner-led evidence for the efficacy of interventions implemented in practice settings (Byiers, Reichle, & Symons, 2012; Jindani & Newman, 2006; Kratochwill et al., 2013; Perdices & Tate, 2009). Additionally single-subject designs provide an underpinning for translational research, enabling practitioners to become involved in the research process at the client-level and add to the knowledge base.

In order to add to the relevance of single-subject research studies, it is important to consider two related issues: 1) the practical relevance of these studies, since small sample sizes are not likely to yield statistically significant results even for effective interventions, and 2) the ability to aggregate these studies into meta-analyses. Moving towards a solution for these challenges involve calculating meaningful effect sizes (Morgan, 2008; Parker, 2006; Schlosser & Wendt, 2008; Scruggs & Mastropieri, 2013). A number of effect size methods have emerged as useful in single-subject research. These include the more Cohen’s d and Hedge’s g, but also include methods specifically designed for single-subject research. Examples of these include non-parametric, non-overlapping methods such as the Percentage of All Non-Overlapping Data (PAND),the Percentage of Data Exceeding the Median (PEM), and Non-Overlap of All Pairs (NAP).

In this workshop, the presenters will use an R package, SSD for R, to illustrate how and when to use a host of effect size calculations based upon characteristics of the data. Many of these effect sizes have accompanying graphs, and a demonstration of how to produce, annotate, and export these graphs will be demonstrated. During the demonstration, the presenters will explain how to generate and interpret a variety of effect sizes, including some that have only been calculated by hand in the past, such as the Improvement Rate Difference. The advantage of using SSD for R is that, in some cases, effect sizes for larger datasets have been cumbersome to manually calculate accurately despite the fact that these effect sizes are useful and appropriate for use in single-subject research.

R is a robust, free, open-source statistical programming language (The R Project for Statistical Computing, n.d.) which can be in lieu of proprietary statistical software packages such as SPSS, SAS, and Stata. R can be installed on almost any computing platform, making it easy for researcher-practitioners to use regardless of the type of computing environment in which they are working. Another benefit to using R is that, because it is freely available, it can easily be introduced into agency settings, increasing research capacity within agencies by reducing access barriers.

In this session, participants will be given instructions for freely downloading all required software, including Rand RStudio, a free graphical user interface that is compatible with R. Participants will also be provided access to datasets and R commands used in the demonstration.