An introduction to Meta-analysis - summary of chapter 1, 2, 3, 4, 8, 10, 11, 12, 13 and 20

An introduction to Meta-analysis

Chapter 1 How a meta-analysis works

Individual studies

Effect size

The effect size is a value which reflects the magnitude of the treatment effect or the strength of a relationship between two variables. It can represent any relationship between two variables. It is the unit of currency in a meta-analysis.

You compute the effect size for each study, and then work with the effect sizes to assess the consistency of the effect across studies and to compute a summary effect.

In graphs, the effect size for each study is represented by a square, with the location of the square representing both the direction and magnitude of the effect.

Precision

In a schematic, the effect size for each study is bounded by a confidence interval. This reflects the precision with which the effect size has been estimated in that study.

Study weights

In a schematic, the size of each square reflects the weight that is assigned to the corresponding study. There is a relationships between a study’s precision and that study’s weight in the analysis. Studies with good precision are assigned more weight. Precision is primarily driven by simple size.

Other elements can be used as well to assign weights.

p-values

For each study, a p-value for a test of the null is shown. A p-value will fall under .005 only if the 95% confidence interval does not include the null value.

The summary effect

Typically, you report the effect size of a summary effect, as well as a measure of precision and a p-value.

Effect size

In a plot, a summary effect is shown on the bottom line. It is the weighted mean of the individual effects. The mechanism used to assign the weights depends on our assumptions about the distribution of effect sizes form which the studies were sampled. 1) Fixed-effect model, there is assumed that all studies in the analysis share the same true effect size. The summary effect is the estimate of this common effect size 2) Random-effects model, there is assumed that the true effect sizes vary from study to study. The summary effect is our estimate of the mean of the distribution of effect sizes

Precision

The summary effect is represented by a diamond. The location represents the effect size while its width reflects the precision of the estimate. The precision addresses the accuracy of the summary effects as an estimate of the true effect.

P-value

The p-value reflects both the magnitude of the summary effect size and the volume of information of which the estimate is based.

Heterogeneity of effect sizes

The treatement effect is not always consistent across studies. If the effect size is consistent, we usually focus on the summary effect. If the effect sizevaries, we might still report the summary effect but note that the true effect in any given study could be somewhat lower or higher than this value. If it varies to heavily, the attention will go to the dispersion itself.

The dispersion in observed effects is partly spurios (it includes both real difference in effects and random error). Before trying to interpret this, we need to dermine what part of the observed variation is real.

Chapter 2 Why perform a meta-analysis

Introduction

The goal of a synthesis is to understand the results of any study in the context of all the other studies. We need no know whether the effect size is consistent across the body of data. If it is, we want to estimate the effect size as accureately as possible. If it isn’t we want to quantify the extent of variance and consider the implications.

Meta-analysis work with effect-sizes to determine whether or not the effect is consistent across studies. All of the are included in a single statistical synthesis.

Chapter 3 Overview

Treatment effects and effect sizes

The term effect size is appropriate when the index is used to quantify the relationship between two variables or a difference between two groups.

The term treatment effect is appropriate only for an index used to quantify the impact of a deliberate intervention.

Single group summary is when a meta-analysis has the goal of estimating a mean or risk or rate in a single population.

How to choose an effect size

Three considerations drive the choice of an effect size index are: 1) The effect sizes from different studies should be comparable to one another. They should measure (at least approximately) the same thing. 2) Estimates of the effect size should be computable from the information that is likely to be reported in published research reports. It should not require a re-analysis of the raw data 3) The effect size should have good technical properties

The effect size should be substantively interpretable.

The kind of data used in the primary studies will usually lead to a pool of two or three effect sizes that meet these criteria. 1) Based on means and standard deviations in two groups, the effect size will be raw difference in means, the standardized difference in means, or the response ratio. 2) Based on a binary outcome, the effect size will be the risk ratio, odds ratio, or risk difference. 3) Based on a correlation, the correlation can serve as an effect size

Parameters and estimates

An underlying effect size parameter is q. The sample estimate of that parameter is Y.

If a study has an infinitely large sample size, the it would yield an effect size Y that is identical to q. Because this isn’t possible, the sample estimate always differs. The value of Y  will vary from sample to sample. The distribution of these values is the sampling distribution of Y.

Chapter 4 Effectsizes based on means

Computing d and g from studies that use independent groups

We can estimate the standardized mean difference (d) from studies that used two independent groups as:

d = (Ẍ1 - Ẍ2) / swithin

1 and Ẍ2 are the sample means in the two groups. The swithin  is the within-groups standard deviation, pooled across groups.

Swithin = √ ((n1 – 1) s21  + (n2 – 1) s22 )/n1+n2 - s). 

N1 and n2 are the sample sizes in the two groups.
s1 and s2 are the standard deviations in the two groups.

The sample estimate of the standardized mean difference is Cohen’s d.

The variance of d is given by:

Vd = (n1 + n2/ n1n2) + d2 /(2(n1 + n2))

The standard error of d is the square root of Vd

SEd = √Vd

d tends to overestimate the absolute value of d in small samples. This can be removed by a correction that yields an unbiased estimate of d, this is sometimes called Hedges’g. To convert from d to g we use a correction factor J.

J = 1 - 3/4df -1.

Df is the degrees of freedom used to estimate swithin.
For two dependent groups, this is n1 + n2 - 2.

G = J * d

Vg = J2 x Vd

SEg = √Vd

J is always less than 1.0.
g will always be less than d in absolute value and variance.
J will be very close to 1.0 unless df is very small.

Chapter 8 Factors that affect precision

Variance, standard error and confidence intervals

The variance is a measure of the mean squared deviation from the mean effect. The variance for an effect size Y is Vy.

The computation of the variance is different for every effect size index. It metric is based on squared value.

The standard error is on the same scale as the effect size.
SEy = √Vy.

If we assume that the effect size is normally distributed, we can compute a 95% confidence interval using

LLy = Ῡ - 1.96 x SEy

ULy = Ῡ + 1.96 x SEy

Zy =  Ῡ /SEy

Factors that affect precision

Factors that affect precision are: 1) sample size, larger studies yield more precise estimates than smaller samples. 2) Study design

Chapter 10 overview

Nomenclature

A study’s true effect size is the effect size in the underlying population. A study’s observed effect size is the effect size that is actually observed.

In the schematics this is represented as 1) Individual studies, circle for the true effect and a square for the observed effect 2) summary effects triangle for the true effect and a diamond for the observed effect

Fixed-effect model

The true effect size

Under the fixed-effect model, we assume that all studies in the meta-analysis share a common (true) effect size.

Impact of sampling error

The observed effect size varies from one study to the next only because of the random error inherent in each study.

The observed effect Yi for any study is given by the population mean plus the sampling error in that study.

Yi = q + errori

We can estimate the sampling distribution of the errors.

Performing a fixed-effect meta-analysis

In a meta-analysis, we start with the observed effects and try to estimate the population effect. In order to obtain the most precise estimate of the population effect (to minimize the variance) we compute a weighted mean. The weight assigned to each study is the inverse of that study’s variance.

Wi = 1/ Vyi

Vyi is the within-study variance for study (i).

The weighted mean (M) is computed as:

M = ∑ki=1wiyi / ki=1wi

The sum of the products WiYi divided by the sum of the weights.

The variance of the summary effect is estimated as the reciprocal of the sum of the weights

Vm = 1 /∑ki=1wi

SEM = √VM

LLy = M - 1.96 x SEM

ULy = M + 1.96 x SEM

Zy =  M /SEM

For a one-tailed test the p value is given by:

p = 1 - φ(±|Z|)

We use + if the difference is in the expected direction and - otherwise.
For a two-tailed test we use:

p = 2[1 - φ(|Z|) ]

 φ(Z) is the standard normal cumulative distribution.

Chapter 12 Random-effects model

The true effect sizes

In a random-effects meta-analysis, we assume that the true effects are normally distributed.

Impact of sampling error

If the true effect for a study is qI, then the observed effect for that study will be less or greater than qI, because of sampling error. The distance between the overall mean and the observed effect in any given study consists of two parts: true variation effect sizes zi and the sampling error ei

The observed effect Yi for ay study is given by grand mean, the deviation of the study’s true effect from the grand mean, and the deviation of the study’s effect from the study’s true effect.

Yi = m + zI + ei

The distance from m to each qI depends on the standard deviation of the distribution of the true effects across studies, called tau (t). The same value of t2 applies to all studies in the meta-analysis.

The distance from qI to Yi depends on the sampling distribution of the sample effects about qI.  This depends on the variance of the observed effect size of each study.

Performing a random-effects meta-analysis

In a meta-analysis, we start with the observed effects and try to estimate the population effect. We use the collection of Yi to estimate m. To obtain the most precise estimate of the overall mean we compute a weighted mean, where the weight assigned to each study is the inverse of that study’s variance.

We need to now the within-study variance and t2.

Estimating tau-squared

T2 is the between-studies variance. If we somehow new the true effect size of each study, and computed the variance of these effect sizes, this would be T2. We estimate this with:

T2 = (Q-df)/C

Q = ∑ki=WiY2i - ((= ki=WiY2)2/= ki=Wi

Df = k -1

K is the number of studies

C = = ∑Wi -∑W2i/ ∑Wi

Estimating the mean effect size

Each study will be weighted by the inverse of its variance. The variance includes the original (within-studies) variance plus the estimate of the between-studies variance T2.

The weight assigned to each study is:
* is for the random effects model.

Wi *= 1/V*Yi

V*Yi = VYi + T2

M* = ∑ki=1wiyi / ki=1wi

Vm* = 1 /∑ki=1wi*

SEM* = √VM*

LLM* = M* - 1.96 x SEM*

ULM* = M* + 1.96 x SEM*

Z* =  M* /SEM*

P* = 1 - φ(±|Z*|)

We use + if the difference is in the expected direction and - otherwise. For a two-tailed test we use:

P* = 2[1 - φ(|Z*|) ]

Chapter 13 Fixed-effect versus random-effects models

Definition of a summary effect

In the fixed-effect analysis, we assume that the true effect size is the same in all studies, and the summary effect tis our estimate of this common effect size.

In random-effects analysis, we assume that the true effect size varies from one study to the next, and that the studies in our analysis represent a random sample of effect sizes that could have been observed. The summary effect is an estimate of the mean of these effects.

Estimating the summary effect

Under the fixed-effect model, we assume that the only reason the effect size varies between studies is the sampling error. When assigning weights to different studies, we can largely ignore the information in the smaller studies.

Under random-effect model, we want to be sure that all effect sizes are represented in the summary estimate. The goal is to estimate the mean effect in a range of studies, we don’t want that overall estimate to be overly influenced by any one of them.

In graphs, the weight assigned to each study is reflected in the size of the box for that study. The fixed-effect model has a wide range of weights. The random-effects model has a narrow range.

Extreme effect size in a large study or a small study

Whenever T2 is nonzero, the relative weights assigned under random effects will be more balanced than those assigned under fixed effects.

Confidence interval

Under the fixed-effect model, the only source of uncertainty is the within-study (sampling or estimation) error.

Under the random-effects model, the sources of uncertainty are the within-study error and the between-studies variance.

The variance, standard error, and confidence interval for the summary effect will be larger under the random-effects model.

Under the fixed-effect model, the standard error of the summary effect is given by:

SEM = √(s2/k x n )

Under the random,effects model, the standard error of the summary effect is given by:

SEM = √((s2/k x n) + (T2/k))

The null hypothesis

Under the fixed-effect model, the null hypothesis is that there is zero effect in every study.

Under the random-effects model, the null hypothesis is that the mean effect is zero.

Which model should we use?

The selection is based on the expectation about whether the studies share a common effect size.

Fixed effect

It makes sense to sue the fixed-effect model if two conditions are met: if you believe that all studies included in the analysis are functionally identical and the goal is to compute the common effect size for the identified population, and not to generalize

Random effects

The goal here is to generalize to a range of scenarios.

A cavecat

If the number of studies is very small, then the estimate of the between-studies variance T2 will have poor precision. One option here is to report the separate effects and not report a summary effect. Another option is to perform fixed-effect analysis. This doesn’t allow to make inferences about the wider population. A third option is to take a Bayesian approach.

Model should not be based on the test for heterogeneity

The decision should be based on our understanding of whether or not all studies share a common effect size, and not the outcome of a statistical test.

Chapter 20 Meta-regression

Introduction

Meta-regression is the use of regression in a meta-analysis. The covariates are at the level of the study. The dependent variable is the effect size in the studies.

There is a need to assign a weight to each study and to select the appropriate model. The R2 index must be modified for use in meta-analysis.

In order for the analysis to be meaningful, there is a need for an appropriately large ratio of studies to covariates.

Fixed-effect model

Assessing the impact of the slope

If we are working with a primary study and wanted to test the coefficient for significance we might use a t-test of the form

t = B/SEB

B is the covariate.

In meta-analysis, the coefficient for any covariate and its standard error are based on groups of studies.

Z = B/SEB

 

Image

Access: 
Public

Image

Join WorldSupporter!
This content is used in:

Evidence-based working in clincial practice

Search a summary

Image

 

 

Contributions: posts

Help other WorldSupporters with additions, improvements and tips

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Image

Spotlight: topics

Check the related and most recent topics and summaries:
Institutions, jobs and organizations:
Activities abroad, study fields and working areas:
This content is also used in .....

Image

Check how to use summaries on WorldSupporter.org

Online access to all summaries, study notes en practice exams

How and why use WorldSupporter.org for your summaries and study assistance?

  • For free use of many of the summaries and study aids provided or collected by your fellow students.
  • For free use of many of the lecture and study group notes, exam questions and practice questions.
  • For use of all exclusive summaries and study assistance for those who are member with JoHo WorldSupporter with online access
  • For compiling your own materials and contributions with relevant study help
  • For sharing and finding relevant and interesting summaries, documents, notes, blogs, tips, videos, discussions, activities, recipes, side jobs and more.

Using and finding summaries, notes and practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the summaries home pages for your study or field of study
  2. Use the check and search pages for summaries and study aids by field of study, subject or faculty
  3. Use and follow your (study) organization
    • by using your own student organization as a starting point, and continuing to follow it, easily discover which study materials are relevant to you
    • this option is only available through partner organizations
  4. Check or follow authors or other WorldSupporters
  5. Use the menu above each page to go to the main theme pages for summaries
    • Theme pages can be found for international studies as well as Dutch studies

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Quicklinks to fields of study for summaries and study assistance

Main summaries home pages:

Main study fields:

Main study fields NL:

Follow the author: SanneA
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Statistics
2115