Meta-analysis in Jasp
Setting up your Meta-Analysis (MA) data is manually done
Meta-analysis menu → Classical Meta-Analysis Menu
Here you can see variables in the dataset on the left side
Select Hedges_g for effect size
select SE for Effect sge Standard error
Underneath Method you can select what kind of meta-analysis you want to perform, for example Fixed Effect
forest plots: most common form used to convey the results in meta-analysis. The pooled effect is noted with a diamond (RE Model). The Weight of the studies is seen via the thickness of the square.
JASP Meta-analysis menu → Classical Meta-Analysis → statistics. Select Forest plot underneath model fit
Publication bias: your meta-analysis is as good as the data. So when the data (studies) are biased your results will also be biased. Some attempts to minimize this is by purposively looking in pre-print repositories and theses
Funnel plot: studies with a small n that are statistically significant, must have a high effect size and SE. A funnel plot is a scatter plot showing the relationship between SE and observed effect size.
If there is no publication bias the points must be fairly symmetrical, and form an upside-down tunnel. There must be more studies at the top and fewer at the bottom
When there is publication bias the points are asymmetrical
JASP Meta-analysis menu --> Classical Meta-Analysis --> statistics--> funnel plot (underneath model fit)
other issues that can cause asymmetry in the Funnel Plot
between-study heterogeneity: effect size plotted by only, does not control for different true underlying effects
Smaller studies could have higher effect sizes because study quality was better (e.g., treatment fidelity)
Lower quality studies tend to have larger effect sizes (other sources of bias)
Egger's Regression Test: A quantitative way to test for asymmetry in the funnel plot Tests whether the Y intercept = 0 (no bias) in a linear regression of normalized effect estimates (observed estimate divided by its standard error) against precision (inverse of the standard error of the estimate)
If there is no publication bias, then the intercept in Egger's regression test should be close to 0
JASP Meta-analysis menu → Classical Meta-Analysis → statistics→ Regression test for funnel plot asymmetry (underneath model fit)
Trimm & Fill Method:
Trimming: – removing outlying studies that cause the asymmetry recalculate the pooled effect
Filling: using the recalculated pooled effect as the center, imputing hypothetical studies to mirror trimmed studies. The recalculated pooled effect this time is the corrected pooled effect
JASP Meta-analysis menu → Classical Meta-Analysis → diagnostics → Trim fill analysis (underneath plots).
In plots you will observe studies that are now labeled filled.
Pooled effect in a random-effects meta-analysis: mean true effect of a range of true effects. Says nothing about how large range is
Between-study heterogeneity: extent that true effects vary Most common way of capturing between-study heterogeneity: Cochran’s Q and I2 When heterogeneity is high – could be explained by subgroups of studies showing differing effects
Outliers: observed effect sizes that are quite a bit different to the others. can have a large impact on the pooled effect. Make sure to recalculate the pooled effect once excluding studies where the confidence intervals does not overlap with the pooled effect confidence intervals.
Influential studies: When the pooled effect relates particularly on a influential study. To check if this is the case or if the pooled effect is robust you can refund the meta-analysis k times, excluding 1 study each time = leave out-analysis
JASP: Meta-analysis menu → Classical Meta-Analysis → diagnostics → case-wise diagnostics (underneath robustness).
You will now see a table named influence measures. Make sure to look at Std. Residuals, DFFITS (Difference in Fits) and Qe. Also look at the forest plot
Between-Subgroup/Moderator analyses
Numerous subgroup analyses: decide a priori based on theory and previous literature. This is also done when between-study heterogeneity is not high and in fixed effects meta-analysis
Fixed effect subgroup analyses: Make inference about subgroups included – sometimes other subgroups are not possible. For example sex: male vs female
Random effects subgroup analyses: Make inference about a range of possible subgroups, where you have a sample of some.
→ usually, same between-study heterogeneity assumed across subgroups. Therefore: approach to meta-analysis and subgroup analysis do not need to match.
JASP Meta-analysis menu → Classical Meta-Analysis. Select your wanted Method for example Restricted underneath Method and also select your interested factor you want to compare underneath Factors
Subgroup analysis output
Line one of Table 1: = if statistically significant, at least one effect size (i.e., one subgroup) is significantly different (from ref.) So when it is significant it means one estimate differs significant from 0
Table 2: Intercept = effect for tonal stimuli (note: ref. chosen alphabetically. It shows the estimate of each factor, and whether they are different from 0
Other lines: if other stimuli were significantly different from your selected stimuli
Table 3: The influence on the between-study heterogeneity for accounting for different stimuli.
You can also use a continuous moderator
JASP meta-analysis menu → Classical Meta-Analysis → model. Select your components and Model Terms (the same) and make sure to include intercept
Limitations subgroup analysis
A lot of power is needed to detect differences in subgroups
anticipated differences are normally small. If not statistically
significant, does not mean that the pooled effects are the
same. (Total k <10, subgroup analysis not advised)
Analysis is observational – studies differ for many reasons, confounders may be present
Add new contribution