Evidence-based working in clincial practice
- 1725 reads
The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trialsPsychological Bulletin, 130 ESTs are empirically supported therapies. ESTs are typically designed for a single Axis I disorder, and patients are screened to maximize homogeneity of diagnosis and minimize co-occurring conditions that could increase variability of treatment response. Treatments are manualized and of brief and fixed duration to minimize within-group variability. Outcome assessment focuses primarily on the symptom that is the focus of the study.RCT methodologies to validate ESTs require a set of additional assumptions that are neither well validated nor applicable to most disorders because: 1) psychopathology is highly malleable 2) most patients can be treated for a single problem or disorder 3) psychiatric disorders can be treated independently of personality factors 4) experimental methods provide a gold standard for identifying useful psychotherapeutic packages A substantial body of data shows that, with or without treatment, relapse rates for all but a handful of disorders are high. There is also a dose-response relationship. Longer treatments are more...
JoHo can really use your help! Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world
Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care
American Psychologist, 63, 146-159.
A central issue is the extent to which findings from research can be applied to clinical practice.
Empirically support or evidence-based treatment (EBT) refers to the interventions or techniques that have produced therapeutic change in controlled trials. Evidence-based practice (EBP) is a broader tem and refers to clinical practice that is informed by evidence about interventions, clinical expertise, and patient needs, values, and preferences and their integration in decision making about individual care.
Concerns about evidence-based treatments
An concern about EBTs is that key conditions and characteristics of treatment research depart markedly from those in clinical practice and bring into question how and whether to generalize the results to practice.
Another concern about research in psychotherapy pertains to the focus on symptoms and disorders as the primary way of identifying participants and evaluation treatment outcomes. In clinical practice, much of psychotherapy is not about reaching a destination (eliminating symptoms), but it is about the ride (the process of coping with life). Psychotherapy research rarely addresses the broader focus of coping with multiple stressors and negotiating the difficult shoals of life, both of which are aided by speaking with a trained professional.
There are concerns about the methods of analysis or the results among several studies. They question whether these are satisfactory bases for concluding that treatment is effective or efficacious. These concerns are: 1) Conclusions about treatment that are based on studies showing statistical differences are difficult to translate into effects on the lives of participants in the study, let alone generalize to patients seen in practice. 2) The outcome measures in most psychotherapy studies raise fundamental concerns. Changes on rating scales are difficult to translate into changes in everyday life. Many valid and reliable measures of psychotherapy are ‘arbitrary metrics’, we do not know how changes on standardized measures translate to functioning in everyday life. 3) Typically, in a single study, multiple measures are used to evaluate outcome, and only some of these show that the treatment and control conditions are statistically different. An EBT may have support for its effects, but within individual studies and among multiple studies, the results are often mixed.
There are inherent limitations in the ways EBTs are discussed. Large segments of the literature usually are grouped together.
A central concern about EMBTs involves the generalization of the results
.....read moreA power primer.
Cohen (1992)
Psychological Bulletin
The tables of this article are missing
Effect-size indexes and conventional values for these are given for operationally defined small, medium, and large effects.
Statistical power analysis exploits the relationships among the four variables involved in statistical inference.
Each is a function of the other three. It is most useful to determine the N necessary to have a specified power for given α and ES.
The significance criterion α
α represents the maximum risk of mistakenly rejecting the null hypothesis (committing a Type I error). This is usually .05. α risk may be defined as one or two sided.
Power
The statistical power of a significance test is the long-term probability, given the population ES, α, and N of rejection the H0. When the ES is nit equal to zero, H0 is false, so failure to reject it also incurs an error (Type II error). For any given ES, α, and N, its probability of occurring is β. Power is 1 – β, the probability of rejecting a false H0.
Taken the conventional α = .05, power of .80, there is a α:β ratio of 4:1 of the two kinds of risks.
Sample size
In research planning, the investigator needs to know the N necessary to attain the desired power for the specified α and hypothesized ES. N increases with an increase in the power desired, a decrease in the ES and in α.
For statistical tests involving two or more groups, N is the necessary size for each group.
Effect size
The effect size (ES) is the degree to which the H0 is believed to be false.
In the Neyman-Pearson method of statistical inference, an alternative hypothesis H1 is counterpoised against H0. The degree to which H0 is false is indexed by the discrepancy between H0 and H1 and is called the ES. Each statistical test has its own ES index. All the indexes are scale free and continuous, ranging upward from zero. For all, the H0 is that ES = 0.
To convey the meaning of any given ES index, it is necessary to have some idea of its scale.
The ES index for the t test of the difference between independent means is d, the difference expressed in units of the within-population standard deviation.
.....read more
Measures of clinical significance.
Kraemer, Morgan, Leech, Gliner, Vaske & Harmon (2003)
Journal of the American Academic of Child & Adolescent Psychiatry
Behavioural scientists are interested in answering three questions when examining the relationships between variables
Researchers suggest that using one of three type of effect size measures assist in interpreting clinical significance
A statistical significant outcome indicates that there is likely to be at least one relationship between the variables. p indicates the probability that an outcome this extreme could happen, if the null hypothesis is true. It doesn’t provide information about the strength of the relationship or whether it is meaningful.
It is possible, with a large sample, to have a statistically significant result from a weak relationship between variables. Outcomes with lower p values are sometimes misinterpret as having stronger effects than those with higher p’s.
Non-statistically significant results do not ‘prove’ the null hypothesis. These might be due to determinants of low power.
The presence or absence of statistical significance does not give information about the size or importance of the outcome. This makes it critical to know the effect size.
The r family
One method of expressing effect sizes is in terms of strength of association. This can be done with statistics such as the Pearson product moment correlation coefficient, r, used when both the independent and the dependent measures are ordered. Such effect sizes vary between -1.0 and + 1.0. 0 represents no effect.
The d family
Used when the independent variable is binary (dichotomous) and the dependent variable is ordered.
When comparing two groups, the effect size d can be computed by
.....read moreStatistics
Chapter 13
Comparing means adjusted for other predictors (analysis of covariance)
The linear model to compare means can be extended to include one or more continuous variables that predict the outcome (or dependent variable).
Covariates: the additional predictors.
ANCOVA: analysis of covariance.
Reasons to include covariates in ANOVA:
For example:
Happinessi = b0 + b1Longi + b2Shorti + b3Covariatei + Ɛi
We can add a covariate as a predictor to the model to test the difference between group means adjusted for the covariate.
With a covariate present, the b-values represent the differences between the means of each group and the control adjusted for the covariate(s).
Independence of the covariate and treatment effect
When the covariate and the experimental effect are not independent, the treatment effect is obscured, spurious treatment effects can arise, and at the very least the interpretation of the ANCOVA is seriously compromised.
When treatment groups differ on the covariate, putting the covariate into the analysis will not ‘control for’ or ‘balance out’ those differences.
This problem can be avoided by randomizing participants to experimental groups, or by matching experimental groups on the covariate.
We can see whether this problem is likely to be an issue by checking whether experimental groups differ on the covariate before fitting the model.
If they do not significantly differ then we might consider it reasonable to use it as a covariate.
Homogeneity of regression slopes
When a covariate is used we loot at its overall relationship with the outcome variable:; we ignore the group to which a person belongs.
We assume that this relationship between covariate and outcome variable holds true for all groups of participants: homogeneity of regression slopes.
There are situations where you might expect regression slopes to differ across groups and that variability may be interesting.
What to do when assumptions are violated
But bootstrap won’t help for the F-tests.
There is a robust variant of ANCOVA.
The main analysis
The format of the ANOVA table is largely the same as without the covariate, except that there is an additional
.....read moreStatistics
Chapter 16
Mixed designs
Situations where we combine repeated-measures and independent designs.
Mixed designs: when a design includes some independent variables that were measured using different entities and others that used repeated measures.
A mixed design requires at least two independent variables.
Because by adding independent variables we’re simply adding predictors to the linear model, you can have virtually any number of independent variables if your sample size is gin enough.
We’re still essentially using the linear model.
Because there are repeated measures involved, people typically use an ANOVA-style model. Mixed ANOVA
All the sources of potential bias in chapter 6 apply.
You can apply the Greenhouse-Geisser correction and forget about sphericity.
Effect sizes are more useful when they summarize a focused effect.
A straightforward approach is to calculate effect sizes for your contrasts.
Discovering statistics using IBM SPSS statistics
Chapter 20
Categorical outcomes: logistic regression
This summary contains the information from chapter 20.8 and forward, the rest of the chapter is not necessary for the course.
Logistic regression is a model for predicting categorical outcomes from categorical and continuous predictors.
A binary logistic regression is when we’re trying to predict membership of only two categories.
Multinominal is when we want to predict membership of more than two categories.
The linear model can be expressed as: Yi = b0 + b1Xi + errori
b0 is the value of the outcome when the predictors are zero (the intercept).
The bs quantify the relationship between each predictor and outcome.
X is the value of each predictor variable.
One of the assumptions of the linear model is that the relationship between the predictors and outcome is linear.
When the outcome variable is categorical, this assumption is violated.
One way to solve this problem is to transform the data using the logarithmic transformation, where you can express a non-linear relationship in a linear way.
In logistic regression, we predict the probability of Y occurring, P(Y) from known (logtransformed) values of X1 (or Xs).
The logistic regression model with one predictor is:
P(Y) = 1/(1+e –(b0 +b1X1i))
The value of the model will lie between 1 and 0.
You need to test for
Multinomial logistic regression predicts membership of more than two categories.
The model breaks the outcome variable into a series of comparisons between two categories.
In practice, you have to set a baseline outcome category.
Statistics
Chapter 11
Moderation, mediation, and multi-category predictors
The conceptual model
Moderation: for a statistical model to include the combined effect of two or more predictor variables on an outcome.
This is in statistical terms an interaction effect.
A moderator variable: one variable that affects the relationship between two others.
Can be continuous or categorical.
We can explore this by comparing the slope of the regression plane for X ad low and high levels of Y.
The statistical model
Moderation is conceptually.
Moderation in the statistical model. We predict the outcome from the predictor variable, the proposed variable, and the interaction of the two.
It is the interaction effect that tells us whether moderation has occurred, but we must include the predictor and moderator for the interaction term to be valid.
Outcomei = (model) + errori
or
Yi = (b0 + b1iX1i + b2iX2i + … + bnXni) + Ɛi
To add variables to a linear model we literally just add them in and assign them a parameter (b).
Therefore, if we had two predictors labelled A and B, a model that tests for moderation would be expressed as:
Yi = (b0 + b1Ai + b2Bi + b3ABi) + Ɛi
The interaction is ABi
Centring variables
When an interaction term is included in the model the b parameters have a specific meaning: for the individual predictors they represent the regression of the outcome on that predictor when the other predictor is zero.
But, there are situation where it makes no sense for a predictor to have a score of zero. So the interaction term makes the bs for the main predictors uninterpretable in many situations.
For this reason, it is common to transform the predictors using grand mean centring.
Centring: the process of transforming a variable into deviations around a fixed point.
This fixed point ca be any value that you choose, but typically it’s the grand mean.
The grand mean centring for a given variable is achieved by taking each score and subtracting from it the mean of all scores (for that variable).
Centring the predictors has no effect on the b for highest-order predictor, but will affect the bs for the lower-order predictors.
Order: how many variables are involved.
When we centre variables, the bs represent the effect of the predictor when the other predictor is at its mean value.
Centring is important when your model contains an interaction term because it makes the bs for lower-order effects interpretable.
There are good reasons for not caring about the lower-order effects when the higher-order
Voorbij het oordeel van de dodo
Huibers (2015)
Tijdschrift voor Psychotherapie
DOI: 10.1007/s12485-015-0027-6
Om de effectiviteit van psychotherapie te verhogen zijn er drie routes. Dit zijn: 1) Het bedenken van nieuwe therapieën 2) Inzicht vinden in de onderliggende werkingsmechanismen 3) Onderzoek doen naar het profiel van patiënten die baad hebben van therapie.
The great psychotherapy debate
Over de werkingsmechanismen van psychotherapie is weinig bekend.
Een mediator is de term voor het werkingsmechanisme of het veranderproces waarmee we het effect van de therapie willen verklaren.
De temporele relatie is wat er vooraf gaat aan het andere. Om dit te kunnen bepalen moeten we tijdens de behandeling vaak zowel de mediator als de uitkomstvariabele meten. Zo kun je ontdekken welke eraan vooraf gaat. Indien de mediator aan de uitkomstvariabele vooraf gaat, weet je nog niet zeker of de mediator werkelijk het veranderingsmechanisme is. De verandering op de mediator kan op zijn beurt weer worden veroorzaakt door een andere factor.
In onderzoek naar werkingsmechanismen zijn twee elementen belangrijk. Dit zijn: 1) Statistische mediatie, het effect van de behandeling op de uitkomst loopt via de mediator 2) Temporele relatie, de verandering van de mediator moet eerder optreden dan de verandering in uitkomstvariabele.
Een moderator wijst op verschillende uitkomsten in verschillende behadelingen. Het is een interessante factor om te bepalen voor welke mensen een bepaalde soort therapie werkt. bn
An introduction to Meta-analysis
Individual studies
Effect size
The effect size is a value which reflects the magnitude of the treatment effect or the strength of a relationship between two variables. It can represent any relationship between two variables. It is the unit of currency in a meta-analysis.
You compute the effect size for each study, and then work with the effect sizes to assess the consistency of the effect across studies and to compute a summary effect.
In graphs, the effect size for each study is represented by a square, with the location of the square representing both the direction and magnitude of the effect.
Precision
In a schematic, the effect size for each study is bounded by a confidence interval. This reflects the precision with which the effect size has been estimated in that study.
Study weights
In a schematic, the size of each square reflects the weight that is assigned to the corresponding study. There is a relationships between a study’s precision and that study’s weight in the analysis. Studies with good precision are assigned more weight. Precision is primarily driven by simple size.
Other elements can be used as well to assign weights.
p-values
For each study, a p-value for a test of the null is shown. A p-value will fall under .005 only if the 95% confidence interval does not include the null value.
The summary effect
Typically, you report the effect size of a summary effect, as well as a measure of precision and a p-value.
Effect size
In a plot, a summary effect is shown on the bottom line. It is the weighted mean of the individual effects. The mechanism used to assign the weights depends on our assumptions about the distribution of effect sizes form which the studies were sampled. 1) Fixed-effect model, there is assumed that all studies in the analysis share the same true effect size. The summary effect is the estimate of this common effect size 2) Random-effects model, there is assumed that the true effect sizes vary from study to study. The summary effect is our estimate of the mean of the distribution of effect sizes
Precision
The summary effect is represented by a diamond. The location
.....read moreMeta-analysis in mental health reserach
Cuipters, P (2016)
Advantages of meta-analysis are: 1) The statistical power to detect effects is higher than for individual studies,this makes a more precise and accurate estimation of true effects possible 2) It is possible to explore inconsistencies between studies and to examine whether the effects of the intervention differs among specific subgroups of studies. 3) It is possible to make an estimate of the number of studies that were conducted but not published
Problems with meta-analyses are: 1) Garbage in, garbage out, they can never be better than the studies they summarize 2) They combine apples and oranges, there are always differences between studies. 3) The file drawer problem, not all relevant studies are published and are often not included in meta-analyses 4) Researcher allegiance, the agenda-driven bias of researchers who conduct the meta-analyses.
Publication bias is the problem that not all the studies that are conducted in a certain area are actually published. Publication of studies which show significant effect and large effects of intentions are favoured. This can lead to an over-estimation of the effect.
There are several other types of reporting bias 1) Time lag bias, some studies are published later than others, depending on the nature and direction of the results 2) Outcome reporting bias 3) Language bias, when studies in another language are not identified and these studies differ in terms of nature and direction of the results.
Testing for publication bias with indirect methods: the funnel plot
In some cases it is possible to examine publication bias directly.
If it is not possible to compare published with unpublished trials, it is possible to get an indirect impression whether there is publication bias or not. These estimates are based on the assumption that large studies can make a more precise estimate of the effect size. Random variations of the effect sizes are larger in studies with few participants, a difference that can be represented graphically in a ‘funnel plot’. In this plot the effect size is represented at the horizontal axis and the size of the study on the vertical axis. If the effect sizes differ from the mean effect size only by chance, they should divert in both directions, both positive and negative.
There are several tests for the asymmetry of the funnel plot.
There is also a method to impute the missing studies and estimate the effect size after imputation of these missing studies.
.....read moreEvaluating statistical and clinical significance of intervention effects in single-case experimental designs: an SPSS method to analyse univariate data
Marija, de Haan, Hogendoorn, Wolters and Huizenga (2015)
Single-case experimental designs are useful to investigate individual client progress. This can help the clinician to investigate whether an intervention works as compared with a baseline period or another intervention, and whether symptom improvement is clinically significant.
In single-case experimental designs (SCEDs), a single participant is repeatedly assessed on one or multiple indices (symptoms) during various phases. Advantages of this are: 1) It can be used to test novel interventions. 2) In heterogenous populations, it can be the only way to investigate treatment outcomes. 3) It offers the possibility to systematically document knowledge of researchers and clinicians.
The AB design consist of two phases; baseline and a treatment.
The AB method can be conceptualized as an interrupted time series. To analysis the differences between baseline and treatment two requirements must be fulfilled. These are: 1) the overall pattern in the time series has to be modelled adequately, an adequate model consists of two linear functions, one for baseline and one for the treatment. Each of these functions is described by an intercept and a slope. 2) adequate modelling of potential correlations between residuals is needed, this is adequate modelling for that which remains after the overall pattern has been accounted for. Autocorrelation is the correlation between residuals of the observations. The residuals are not independent. If residuals are correlated, the correlations are likely to decrease with increasing separation between time points.
Analyses investigating treatment efficacy
Y(i) = b0 +b1 * phase(i) + b2*time_in_phase (i) + b3 * phase(i) * time_in_phase(i) + error(i)
Y(i) is the outcome variable at time point i.
Phase(i) denotes the phase in which time point I is contained (0 for baseline and 1 for treatment).
Time_in_phase (i)is the time points in each phase.
Error (i) is the residual at point i.
b0 is the baseline intercept
b1 is the treatment-baseline difference in intercepts
b2 is the baseline slope
b3 is the treatment-baseline difference in slopes
Intercept differences between phases can be assessed by testing whether b1 differs from 0. Slope differences can be assessed by testing b3.
B0 and b1 refer to symptom scores when time_in_phase is zero. This depends on the coding of time_in_phase.
It is best to both describe the general trend adequately and to account for remaining correlations. The
.....read moreN=1 studies in onderzoek en praktijk.
De Psycholoog, 3, 11-20
Door frequent te meten bij een individu ontstaat er een N =1 studie. De informatie kan worden gebruikt om de behandeling optimaal in te richten en maximaal resultaat te verkrijgen.
N =1 studies zijn studies waarbij de ernst van de klachten worden gemeten bij één cliënt tijdens de baseline en de behandeling. Dit kan informatie opleveren of en wanneer de behandeling werkt.
Kenmerken van N=1 studies zijn: 1) Het is een studie op één deelnemer. Het is mogelijk een reeks N =1 studies uit te voeren waarbij meerdere deelnemers met dezelfde onderzoeksvraag worden onderzocht. De gevonden resultaten worden bij elkaar gevoegd. 2) Er wordt regelmatig en betrouwbaar gemeten 3) Er zijn duidelijk gedefinieerde fasen. Fase A is een baselinefase Fase B is een fase waarin er behandeling plaatsvindt. In een ABC design krijgt een cliënt twee verschillende behandelingen (B en C).
Werkzaam
N = 1 studies kunnen gebruikt worden voor het testen van nieuwe behandelprocedures en om de doeltreffendheid van bestaande behandelingen te testen in de praktijk.
In N=1 studies kunnen veranderingsprocessen worden onderzocht. Werkingsmechanismen zijn processen waardoor de behandeling effect behaalt.
Voor het onderzoek naar werkingsmechanismen zijn een aantal voorwaarden waaraan moet worden voldaan: 1) de deelnemer moet de behandeling hebben ontvangen 2) tijdens de behandeling moet er verbetering op zijn getreden in de onderzochte klachten 3) er moet worden aangetoond dat er verandering is opgetreden op het voorgestelde werkingsmechanisme 4) de verandering op het werkingsmechanisme moet voorafgaan aan het behandeleffect.
De N=1 methodologie sluit aan bij de klinische praktijk.
Een nadeel van de N=1 methodologie is: gegevens verkregen via een N=1 studie hebben een lagere bewijskracht dan gegevens verkregen uit een RCT.
N=1 studies en RCT studies kunnen elkaar aanvullen.
Voordelen N=1 studies zijn: 1) De omgeving van de studie moet minder aan methodologische eisen voldoen. De cliënt is zijn eigen controle Het kan worden onderzocht in heterogene groepen 2) Analyses worden op individueel niveau uitgevoerd, waardoor er geen informatie verloren gaat 3) N=1 studies worden op kleine schaal uitgevoerd.
De werkingsmechanismes die naar voren komen in N=1 studies kunnen daarna in RCTs worden onderzocht.
Routinematig meten
Routine Outcome motinitoring (ROM) maakt herhaaldelijk meten mogelijk. Het kan gebruikt worden om gegevens te verzamelen over stoornis, cliënt en behandeluitkomsten.
.....read moreThe empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials
Psychological Bulletin, 130
ESTs are empirically supported therapies.
ESTs are typically designed for a single Axis I disorder, and patients are screened to maximize homogeneity of diagnosis and minimize co-occurring conditions that could increase variability of treatment response. Treatments are manualized and of brief and fixed duration to minimize within-group variability. Outcome assessment focuses primarily on the symptom that is the focus of the study.
RCT methodologies to validate ESTs require a set of additional assumptions that are neither well validated nor applicable to most disorders because: 1) psychopathology is highly malleable 2) most patients can be treated for a single problem or disorder 3) psychiatric disorders can be treated independently of personality factors 4) experimental methods provide a gold standard for identifying useful psychotherapeutic packages
A substantial body of data shows that, with or without treatment, relapse rates for all but a handful of disorders are high. There is also a dose-response relationship. Longer treatments are more effective.
Most psychopathological vulnerabilities are highly resistant to change. They tend to be rooted in personality and temperament. The modal patient treated with brief treatments for most disorders relapses or seeks additional treatment.
In RCTs, including patients with substantial comorbidities would vastly increase the sample size necessary to detect treatment differences if comorbidity bears any systematic relation to outcome.
Three issues are: 1) The empirical and pragmatic limits imposed by reliance on DSM diagnoses 2) The problem of comorbidity 3) the way the different functions of assessing comorbidity in controlled trials and clinical practice may place limits on generalizability
The pragmatics of DSM-IV diagnosis
Three costs of the DSM are 1) the diagnoses are themselves created by committee consensus on the basis of the available evidence rather than by strictly empirical methods 2) The assumption that patients typically present with symptoms of a specific Axis I diagnosis and identify
.....read moreIn this beautiful magazine, the summaries you need for the course evidence-based working in clinical practice in the third year of the uva, are bundled.
JoHo kan jouw hulp goed gebruiken! Check hier de diverse studentenbanen die aansluiten bij je studie, je competenties verbeteren, je cv versterken en een bijdrage leveren aan een tolerantere wereld
There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.
Do you want to share your summaries with JoHo WorldSupporter and its visitors?
Field of study
Add new contribution