Evidence-based Clinical Practice – Lecture 1 (UNIVERSITY OF AMSTERDAM)

Evidence-based treatment refers to interventions or techniques that have therapeutic changes in controlled trials. Disadvantages are that key conditions and characteristics of treatment research are different from clinical practice and that research tends to focus on symptoms, rather than the patient as a whole. Determining whether a treatment is evidence-based or empirically supported has different criteria (1), makes use of arbitrary rating scales (2) and often has to deal with mixed results (3).

Evidence-based practice refers to clinical practice informed by evidence about interventions, clinical expertise, patient needs, values and preferences. Concerns regarding this are that not everything has been researched (e.g. rare disease) (1), clinical decision making is unreliable (2), generalization of results is difficult (3), determining which variables make the difference in a treatment is difficult (4) and clinical progress is often evaluated based on clinical impressions (5).

The goals of research are to optimally develop the knowledge base (1), provide the best information to improve patient care (2) and reduce the divide between research and practice. These goals can be achieved by shifting the emphasis to give a greater priority to the study of mechanisms of therapy (1), study the moderators of change (2) and conducting more qualitative research (3).

Conducting more qualitative research could connect metrics that are not arbitrary (e.g. connecting objective measures to a disorder). Furthermore, it could also generate hypotheses. Clinical practice should use more systematic measures of patient progress because it helps to provide high-quality care (1), it helps make decisions regarding treatment (e.g. continuation or not) (2) and it helps to complete clinical judgement (3). Clinical practice should also add accumulated knowledge to the knowledge-base.

The outcome of treatment can be improved by identifying effective interventions (1), understanding why and how an effective treatment works (2) and identifying moderators of treatment (3).

There are three distinctions for the classification of variables:

1. Nominal vs. continuous
   The nominal scale is a categorical scale (e.g. ADHD or no ADHD). The continuous scale includes more values (e.g. age).
2. Independent vs. dependent
   Independent variables are the interventions or the variables that are believed to influence the dependent variable. The dependent variable is the outcome variable and the variable which is thought to be influenced by the independent variable (e.g. effect on memory).
3. Manifest vs. latent
   A manifest variable is any variable that can be measured (e.g. shoe size). A latent variable is any variable that cannot be measured directly but can be derived from the data (e.g. resilience).

There are variables that can be both a manifest or a latent variable, depending on how it’s being used (e.g. intelligence).

The foundation of most statistical tests (i.e. general linear model) are comparing two or more conditions and checking whether the difference between the conditions are significant. This is done by calculating the test statistic (e.g. t-statistic, F-statistic). This is typically calculated by having the alternative hypothesis (e.g. mean 1 – mean 2) minus the null hypothesis (e.g. 0) divided by the standard error.

The null hypothesis is not rejected if the data is consistent with it. However, it is rejected if the data are not consistent with the null hypothesis. The null hypothesis is always rejected or not rejected but it is never accepted. The p-value is the probability of obtaining the data given that the null hypothesis is true.

Regression analysis fits a line through the data to predict the value of one variable from the value of another variable. The predictor variable (i.e. ‘x’) is used to predict another variable (i.e. ‘y’). The values for the predictor variable are fixed. The regressor is measured for all variables of the predictor variable. Regression analysis fits a line through the data that minimizes the deviations in y from that line.

There are three potential variations in the regression analysis:

1. Line of best fit
   This line reflects how strongly x and y covary.
2. Line with b ≈ 0
   The degree to which x and y covary is negligible and the value of y does not depend on the value of x.
3. Line with b ≠ 0
   The degree to which x and y covary is significant and the higher the x the higher or lower the y.

The residuals reflect the scatter of points above and below the best-fitting regression line. It measures the difference between the predicted and the measured Y. The variance in the residuals quantifies the spread of the scatter points and indicates the goodness of fit. A low variance of residuals indicates a good fit of the regression line.

A null hypothesis test with a regression line evaluates whether the observed slope equals the value under the null hypothesis, which is typically zero. It makes use of a t-distribution and has several assumptions:

1. Homoscedasticity
   The variance in the variable of interest does not differ between groups. The variance should be equal for all values of x.
2. Independence
   The participants should be independent of each other.
3. Normal distribution
   The residuals should be normally distributed.

The ANOVA compares multiple conditions to see whether they differ. However, the ANOVA does not tell us which conditions differ significantly from the null hypothesis. It makes use of the F-statistic. In an ANOVA, there are two sources of variance; between-groups variance (1) and within-groups variance (2). The null hypothesis is rejected if the between-groups variance is larger than the within-group variance (i.e. F-statistic).

The ANOVA uses several assumptions:

1. Equal variance
   The variance should be equal in all groups.
2. Independence
   The participants should be independent of each other.
3. Normal distribution
   The data should be normally distributed within groups.

The ANOVA and the GAM can be used to check for interaction (e.g. significant b1 * b2) and main effects (e.g. significant b1). The ANCOVA corrects for non-random allocation (i.e. influence of covariates) (1) and reduces variance within groups (i.e. by controlling for a source of variance). The ANCOVA potentially promotes power. It makes use of several assumptions:

1. Equal variance
   The variance should be equal in all groups.
2. Independence
   The participants should be independent of each other.
3. Normal distribution
   The data should be normally distributed within groups.
4. Parallel regression lines
   The regression lines should be parallel (including the covariate).

The ANCOVA is conducted in three steps:

1. Test assumptions
   The first step is to test whether the regression lines run parallel (i.e. this can be checked by having an insignificant interaction effect between treatment and covariate).
2. Influence of covariate
   The second step is to test whether the covariate influences the dependent variable (1) and to test the influence of the effect of treatment on the dependent variable (2).
3. Comparison
   The final step is to compare the outcome of the influence of the treatment on the dependent variable when controlling for the covariate with the regular ANOVA which does not control for the covariate.  It checks whether adding the covariate alters the treatment effects.

The multivariate approach makes use of pair-wise contrasts and compares them all. This does not make use of assumptions regarding which pairs differ. The univariate approach is more common and makes use of specific pair-wise contrasts, based on specific predictions. The within-subjects design ANOVA makes use of several assumptions:

1. Equal variance
   The variance should be equal in all groups.
2. Independence
   The participants should be independent of each other.
3. Normal distribution
   The data should be normally distributed.
4. Sphericity (only in univariate approach)
   The variance of the differences between all possible pairs of within-subject conditions should be equal (i.e. Mauchly’s test of sphericity).

In a t-test, the difference between two groups (e.g. exercise vs. no exercise) is scrutinized. If the difference is statistically significant, then the null hypothesis will be rejected in favour of the alternative hypothesis. The t-test requires several assumptions:

1. Homoscedasticity
   The variance in the variable of interest does not differ between groups. On SPSS, this is seen as “equal variances assumed” (e.g. in the t-test).
2. Independence
   The participants should be independent of each other.
3. Normal distribution
   The data should be normally distributed within groups.