Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care
American Psychologist, 63, 146-159.

Introduction  

A central issue is the extent to which findings from research can be applied to clinical practice.

Empirically support or evidence-based treatment (EBT) refers to the interventions or techniques that have produced therapeutic change in controlled trials. Evidence-based practice (EBP) is a broader tem and refers to clinical practice that is informed by evidence about interventions, clinical expertise, and patient needs, values, and preferences and their integration in decision making about individual care.

Evidence-based treatments and clinical practice: illustrative concerns

Concerns about evidence-based treatments

An concern about EBTs is that key conditions and characteristics of treatment research depart markedly from those in clinical practice and bring into question how and whether to generalize the results to practice.

Another concern about research in psychotherapy pertains to the focus on symptoms and disorders as the primary way of identifying participants and evaluation treatment outcomes. In clinical practice, much of psychotherapy is not about reaching a destination (eliminating symptoms), but it is about the ride (the process of coping with life). Psychotherapy research rarely addresses the broader focus of coping with multiple stressors and negotiating the difficult shoals of life, both of which are aided by speaking with a trained professional.

There are concerns about the methods of analysis or the results among several studies. They question whether these are satisfactory bases for concluding that treatment is effective or efficacious. These concerns are: 1) Conclusions about treatment that are based on studies showing statistical differences are difficult to translate into effects on the lives of participants in the study, let alone generalize to patients seen in practice. 2) The outcome measures in most psychotherapy studies raise fundamental concerns. Changes on rating scales are difficult to translate into changes in everyday life. Many valid and reliable measures of psychotherapy are ‘arbitrary metrics’, we do not know how changes on standardized measures translate to functioning in everyday life. 3) Typically, in a single study, multiple measures are used to evaluate outcome, and only some of these show that the treatment and control conditions are statistically different. An EBT may have support for its effects, but within individual studies and among multiple studies, the results are often mixed.

There are inherent limitations in the ways EBTs are discussed. Large segments of the literature usually are grouped together.

A central concern about EMBTs involves the generalization of the results

Measures of clinical significance. 
Kraemer, Morgan, Leech, Gliner, Vaske & Harmon (2003)
Journal of the American Academic of Child & Adolescent Psychiatry

Introduction

Behavioural scientists are interested in answering three questions when examining the relationships between variables

• Statistical significance
   Is an observed result real or should it be attributed to chance?
• Effect size
  If the result is real, how large is it?
• Clinical or practical significance
  Is the result large enough to be meaningful and useful?

Researchers suggest that using one of three type of effect size measures assist in interpreting clinical significance

• r family effect size measures
  The strength of association between variables
• d family effect size measures
  The magnitude of the difference between treatment and comparison groups
• Measures of risk potency
  • Odds ratio
  • Risk ratio
  • relative risk reduction
  • risk difference
  • number needed to treat

Problems with statistical significance

A statistical significant outcome indicates that there is likely to be at least one relationship between the variables. p indicates the probability that an outcome this extreme could happen, if the null hypothesis is true. It doesn’t provide information about the strength of the relationship or whether it is meaningful.

It is possible, with a large sample, to have a statistically significant result from a weak relationship between variables. Outcomes with lower p values are sometimes misinterpret as having stronger effects than those with higher p’s.

Non-statistically significant results do not ‘prove’ the null hypothesis. These might be due to determinants of low power.

The presence or absence of statistical significance does not give information about the size or importance of the outcome. This makes it critical to know the effect size.

Effect size measures

The r family

One method of expressing effect sizes is in terms of strength of association. This can be done with statistics such as the Pearson product moment correlation coefficient, r, used when both the independent and the dependent measures are ordered. Such effect sizes vary between  -1.0 and + 1.0. 0 represents no effect.

The d family

Used when the independent variable is binary (dichotomous) and the dependent variable is ordered.

When comparing two groups, the effect size d can be computed by

Statistics
Chapter 16
Mixed designs

Mixed designs

Situations where we combine repeated-measures and independent designs.

Mixed designs: when a design includes some independent variables that were measured using different entities and others that used repeated measures.
A mixed design requires at least two independent variables.

Because by adding independent variables we’re simply adding predictors to the linear model, you can have virtually any number of independent variables if your sample size is gin enough.

We’re still essentially using the linear model.
Because there are repeated measures involved, people typically use an ANOVA-style model. Mixed ANOVA

Assumptions in mixed designs

All the sources of potential bias in chapter 6 apply.

• homogeneity of variance
• sphericity

You can apply the Greenhouse-Geisser correction and forget about sphericity.

Mixed designs

• Mixed designs compare several means when there are two or more independent variables, and at least one of them has been measured using the same entities and at least one other has been measured using different entiteis.
• Correct for deviations from sphericity for the repeated-measures variable(s) by routinely interpreting the Greenhouse-Geisser corrected effects.
• The table labelled Tests of Within-Subject Effects shows the F-statistic(s) for any repeated-measures variables and all of the interaction effects. For each effect, read the row labelled Greenhouse-Geisser or Huynh-Feldt. If the values in the Sig column is less than 0.05 then the means are significantly different
• The table labelled Test of Between-Subjects Effects shows the F-statistic(s) for any between-group variables. If the value in the Sig column is less than 0.05 then the means of the groups are significantly different
• Break down the mean effects and interaction terms using contrasts. These contrasts appear in the table labelled Tests of Within-Subjects Contrasts. Again, look at the column labelled sig.
• Look at the means, or draw graphs, to help you interpret contrasts.

Calculating effect sizes

Effect sizes are more useful when they summarize a focused effect.

A straightforward approach is to calculate effect sizes for your contrasts.

Statistics
Chapter 11
Moderation, mediation, and multi-category predictors

Moderation: interactions in the linear model

The conceptual model

Moderation: for a statistical model to include the combined effect of two or more predictor variables on an outcome.
This is in statistical terms an interaction effect.

A moderator variable: one variable that affects the relationship between two others.
Can be continuous or categorical.
We can explore this by comparing the slope of the regression plane for X ad low and high levels of Y.

The statistical model

Moderation is conceptually.

Moderation in the statistical model. We predict the outcome from the predictor variable, the proposed variable, and the interaction of the two.
It is the interaction effect that tells us whether moderation has occurred, but we must include the predictor and moderator for the interaction term to be valid.

Outcome_i = (model) + error_i

or

Y_i = (b₀ + b_1iX_1i + b_2iX_2i + … + b_nX_ni) + Ɛ_i

To add variables to a linear model we literally just add them in and assign them a parameter (b).
Therefore, if we had two predictors labelled A and B, a model that tests for moderation would be expressed as:

Y_i = (b₀ + b₁A_i + b₂B_i + b₃AB_i) + Ɛ_i

The interaction is AB_i

Centring variables

When an interaction term is included in the model the b parameters have a specific meaning: for the individual predictors they represent the regression of the outcome on that predictor when the other predictor is zero.

But, there are situation where it makes no sense for a predictor to have a score of zero. So the interaction term makes the bs for the main predictors uninterpretable in many situations.
For this reason, it is common to transform the predictors using grand mean centring.
Centring: the process of transforming a variable into deviations around a fixed point.
This fixed point ca be any value that you choose, but typically it’s the grand mean.
The grand mean centring for a given variable is achieved by taking each score and subtracting from it the mean of all scores (for that variable).

Centring the predictors has no effect on the b for highest-order predictor, but will affect the bs for the lower-order predictors.
Order: how many variables are involved.
When we centre variables, the bs represent the effect of the predictor when the other predictor is at its mean value.

Centring is important when your model contains an interaction term because it makes the bs for lower-order effects interpretable.
There are good reasons for not caring about the lower-order effects when the higher-order

An introduction to Meta-analysis

Chapter 1 How a meta-analysis works

Individual studies

Effect size

The effect size is a value which reflects the magnitude of the treatment effect or the strength of a relationship between two variables. It can represent any relationship between two variables. It is the unit of currency in a meta-analysis.

You compute the effect size for each study, and then work with the effect sizes to assess the consistency of the effect across studies and to compute a summary effect.

In graphs, the effect size for each study is represented by a square, with the location of the square representing both the direction and magnitude of the effect.

Precision

In a schematic, the effect size for each study is bounded by a confidence interval. This reflects the precision with which the effect size has been estimated in that study.

Study weights

In a schematic, the size of each square reflects the weight that is assigned to the corresponding study. There is a relationships between a study’s precision and that study’s weight in the analysis. Studies with good precision are assigned more weight. Precision is primarily driven by simple size.

Other elements can be used as well to assign weights.

p-values

For each study, a p-value for a test of the null is shown. A p-value will fall under .005 only if the 95% confidence interval does not include the null value.

The summary effect

Typically, you report the effect size of a summary effect, as well as a measure of precision and a p-value.

Effect size

In a plot, a summary effect is shown on the bottom line. It is the weighted mean of the individual effects. The mechanism used to assign the weights depends on our assumptions about the distribution of effect sizes form which the studies were sampled. 1) Fixed-effect model, there is assumed that all studies in the analysis share the same true effect size. The summary effect is the estimate of this common effect size 2) Random-effects model, there is assumed that the true effect sizes vary from study to study. The summary effect is our estimate of the mean of the distribution of effect sizes

Precision

The summary effect is represented by a diamond. The location

Evaluating statistical and clinical significance of intervention effects in single-case experimental designs: an SPSS method to analyse univariate data
Marija, de Haan, Hogendoorn, Wolters and Huizenga (2015)

Abstract

Single-case experimental designs are useful to investigate individual client progress. This can help the clinician to investigate whether an intervention works as compared with a baseline period or another intervention, and whether symptom improvement is clinically significant.

Introduction

In single-case experimental designs (SCEDs), a single participant is repeatedly assessed on one or multiple indices (symptoms) during various phases. Advantages of this are: 1) It can be used to test novel interventions. 2) In heterogenous populations, it can be the only way to investigate treatment outcomes. 3) It offers the possibility to systematically document knowledge of researchers and clinicians.

Method

The AB design consist of two phases; baseline and a treatment.

The AB method can be conceptualized as an interrupted time series. To analysis the differences between baseline and treatment two requirements must be fulfilled. These are:  1) the overall pattern in the time series has to be modelled adequately, an adequate model consists of two linear functions, one for baseline and one for the treatment. Each of these functions is described by an intercept and a slope. 2) adequate modelling of potential correlations between residuals is needed, this is adequate modelling for that which remains after the overall pattern has been accounted for. Autocorrelation is the correlation between residuals of the observations. The residuals are not independent. If residuals are correlated, the correlations are likely to decrease with increasing separation between time points.

Analyses investigating treatment efficacy

 Y(i) = b0 +b1 * phase(i) + b2*time_in_phase (i) + b3 * phase(i) * time_in_phase(i) + error(i)

Y(i) is the outcome variable at time point i.
Phase(i) denotes the phase in which time point I is contained (0 for baseline and 1 for treatment).
Time_in_phase (i)is the time points in each phase.
Error (i) is the residual at point i.
b0 is the baseline intercept
b1 is the treatment-baseline difference in intercepts
b2 is the baseline slope
b3 is the treatment-baseline difference in slopes

Intercept differences between phases can be assessed by testing whether b1 differs from 0. Slope differences can be assessed by testing b3.

B0 and b1 refer to symptom scores when time_in_phase is zero. This depends on the coding of time_in_phase.

It is best to both describe the general trend adequately and to account for remaining correlations. The

The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials
Psychological Bulletin, 130

The assumptions underlying ESTs

ESTs are empirically supported therapies.

ESTs are typically designed for a single Axis I disorder, and patients are screened to maximize homogeneity of diagnosis and minimize co-occurring conditions that could increase variability of treatment response. Treatments are manualized and of brief and fixed duration to minimize within-group variability. Outcome assessment focuses primarily on the symptom that is the focus of the study.

RCT methodologies to validate ESTs require a set of additional assumptions that are neither well validated nor applicable to most disorders because: 1) psychopathology is highly malleable 2) most patients can be treated for a single problem or disorder 3) psychiatric disorders can be treated independently of personality factors 4) experimental methods provide a gold standard for identifying useful psychotherapeutic packages

Psychological processes are highly malleable

A substantial body of data shows that, with or without treatment, relapse rates for all but a handful of disorders are high. There is also a dose-response relationship. Longer treatments are more effective.

Most psychopathological vulnerabilities are highly resistant to change. They tend to be rooted in personality and temperament. The modal patient treated with brief treatments for most disorders relapses or seeks additional treatment.

Most patients have one primary problem or can be treated as if they do

In RCTs, including patients with substantial comorbidities would vastly increase the sample size necessary to detect treatment differences if comorbidity bears any systematic relation to outcome.

Three issues are: 1) The empirical and pragmatic limits imposed by reliance on DSM diagnoses 2) The problem of comorbidity 3) the way the different functions of assessing comorbidity in controlled trials and clinical practice may place limits on generalizability

The pragmatics of DSM-IV diagnosis

Three costs of the DSM are 1) the diagnoses are themselves created by committee consensus on the basis of the available evidence rather than by strictly empirical methods 2) The assumption that patients typically present with symptoms of a specific Axis I diagnosis and identify