Maric et al. (2015). Evaluating statistical and clinical significance of intervention effects in single-case experimental designs: An SPSS method to analyze univariate data.” - Article summary

Single-case experimental designs (SCED) are useful methods in clinical research to investigate individual client progress. It could also be used to determine whether an intervention works. In single-case experimental designs, a single participant is repeatedly assessed on one or multiple symptoms during various phases (e.g. baseline). The statistical analysis of this data is difficult.

Advantages of the single-case experimental designs are that it can be used to test novel interventions before RCTs are conducted (1), it may be the only way to investigate treatment outcomes in heterogeneous groups (2) and it offers the possibility to systematically document the knowledge of researchers and clinicians, preventing loss of information (3).

The most common single-case experimental design is the AB design. It consists of two phases (i.e. baseline and treatment). It is similar to an interrupted time series. In order to obtain an adequate analysis of the differences, the overall pattern in the time series should be modelled adequately (1) and there should be adequate modelling of potential correlations between residuals (2).

A common assumption of adequate models is that there is a linear function for both the baseline and the treatment phase. Both of these linear functions should be described by an intercept and a slope. Modelling of potential correlations between residuals means that there should be adequate modelling after the overall pattern has been accounted for. The correlation between residuals of the observations is the autocorrelation. It implies that the residuals are not independent. If residuals are correlated, the correlations are likely to decrease with increasing separation between timepoints.

The tests on the intercepts and the slopes of the linear functions will be unreliable if the correlations between the residuals are not modelled adequately (e.g. incorrectly assuming that the residuals are uncorrelated). If positively correlated residuals are assumed to be uncorrelated, the chances of finding significant results will be too high.

Modelling the overall pattern by the intercept and the slope for each phase (i.e. each timepoint) does not yield a direct test of intercept and slope differences. Therefore, it is useful to re-parameterize the model in terms of an intercept and slope for the baseline phase and baseline-treatment differences in the intercepts and slopes. This takes the following formula:

 Y(i) denotes the outcome variable score at time point i. Phase(i) denotes the phase in which time point i is contained. Time_in_phase denotes time points within each phase. E(i) denotes the residual at time point i.

The parameter b0 is interpreted as the baseline intercept. B1 is interpreted as the treatment-baseline difference in intercepts. B2 is interpreted as the baseline slope and b3 is interpreted as the treatment-baseline differences in slopes. These parameters can also be interpreted as effect sizes.

The b0 and b1 refer to symptom scores when time_in_phase is zero. Therefore, when coding the variables, time_in_phase zero should denote the start of each phase. However, when time_in_phase zero denote the end of each phase, then a test on b1 would be a test of treatment efficacy.

The parameter (i.e. rho) used in the first-order autoregressive model may be underestimated for short time series. This can be solved by applying a small sample correction (1), by testing parameters by implementing a permutation-based procedure (2) and carrying out the test at more stringent levels (i.e. reducing the likelihood of finding spurious results (3).

The reliable change index (RCI) uses the following formula:

X1 and X2 represent a participant’s pre- and post-treatment scores. Sdiff denotes the standard deviation of the difference between the two test scores. It may be useful to use the observed endpoints of phases and this leads to the following formula.

There are several assumptions of the first-order autoregressive model (i.e. AB design):

• The overall pattern in the data is modelled correctly.
• The error correlation structure is modelled adequately.
• Missing data should be missing at random.
• The number of timepoints is sufficient to estimate the AR(1) parameter.

The autoregressive model is reliable if the number of timepoints per phase is 10 or more. If there are only 5 timepoints per phase, a 1% alpha level should be used to avoid type-I errors. How to do this in SPSS is explained in the following video http://youtu.be/sYGOynx-J8M.

The AB design method using the autoregressive model can be generalized to more than two phases (e.g. ABAC) (1), the method can be used if some session data are missing at random (2), the method can be used if symptoms are probed at irregular timepoints (3), longer follow-up data can be included in the method by viewing it as a separate phase (4) and results across participants can be combined by using meta-analytic techniques (e.g. weighing intercepts).  

In the absence of significant results, it may be that the questionnaire used had low levels of sensitivity to treatment change. Reliability information regarding the measures should ideally be selected from norm groups that are as similar to the client groups as possible.