Join Log in Profile Search

aantekeningen lecture 2 Academic Research Methods and Statistics (ARMS) module 1 Meta-analysis 22/23 Universiteit Utrecht

Meta-analysis in Jasp
 
Setting up your Meta-Analysis (MA) data is manually done
  • study labels are used
  • potential moderators are determined
 
Meta-analysis menu →  Classical Meta-Analysis Menu
  • Here you can see variables in the dataset on the left side
  • Select Hedges_g for effect size
  • select SE for Effect sge Standard error
  • Underneath Method you can select what kind of meta-analysis you want to perform, for example Fixed Effect
 
forest plots: most common form used to convey the results in meta-analysis. The pooled effect is noted with a diamond (RE Model). The Weight of the studies is seen via the thickness of the square. 
JASP Meta-analysis menu →  Classical Meta-Analysis →  statistics. Select Forest plot underneath model fit
 
Publication bias: your meta-analysis is as good as the data. So when the data (studies) are biased your results will also be biased. Some attempts to minimize this is by purposively looking in pre-print repositories and theses
 
Funnel plot: studies with a small n that are statistically significant, must have a high effect size and SE. A funnel plot is a scatter plot showing the relationship between SE and observed effect size. 
  • If there is no publication bias the points must be fairly symmetrical, and form an upside-down tunnel. There must be more studies at the top and fewer at the bottom
  • When there is publication bias the points are asymmetrical
JASP Meta-analysis menu --> Classical Meta-Analysis --> statistics--> funnel plot (underneath model fit)
 
other issues that can cause asymmetry in the Funnel Plot
  • between-study heterogeneity: effect size plotted by only, does not control for different true underlying effects
  • Smaller studies could have higher effect sizes because study quality was better (e.g., treatment fidelity)
  • Lower quality studies tend to have larger effect sizes (other sources of bias)
 
Egger's Regression Test: A quantitative way to test for asymmetry in the funnel plot Tests whether the Y intercept = 0 (no bias) in a linear regression of normalized effect estimates (observed estimate divided by its standard error) against precision (inverse of the standard error of the estimate)
If there is no publication bias, then the intercept in Egger's regression test should be close to 0
JASP Meta-analysis menu →  Classical Meta-Analysis → statistics→  Regression test for funnel plot asymmetry (underneath model fit)
 
Trimm & Fill Method:
  • Trimming: – removing outlying studies that cause the asymmetry recalculate the pooled effect
  • Filling: using the recalculated pooled effect as the center, imputing hypothetical studies to mirror trimmed studies. The recalculated pooled effect this time is the corrected pooled effect
JASP Meta-analysis menu →  Classical Meta-Analysis →  diagnostics →  Trim fill analysis (underneath plots). 
In plots you will observe studies that are now labeled filled. 
 
Pooled effect in a random-effects meta-analysis: mean true effect of a range of true effects. Says nothing about how large range is
  • Between-study heterogeneity: extent that true effects vary Most common way of capturing between-study heterogeneity: Cochran’s Q and I2 When heterogeneity is high – could be explained by subgroups of studies showing differing effects
  • Outliers: observed effect sizes that are quite a bit different to the others. can have a large impact on the pooled effect. Make sure to recalculate the pooled effect once excluding studies where the confidence intervals does not overlap with the pooled effect confidence intervals.
 
Influential studies: When the pooled effect relates particularly on a influential study. To check if this is the case or if the pooled effect is robust you can refund the meta-analysis k times, excluding 1 study each time = leave out-analysis 
JASP: Meta-analysis menu →  Classical Meta-Analysis →  diagnostics →  case-wise diagnostics (underneath robustness). 
You will now see a table named influence measures. Make sure to look at Std. Residuals, DFFITS (Difference in Fits) and Qe. Also look at the forest plot
 
Between-Subgroup/Moderator analyses
  • Numerous subgroup analyses: decide a priori based on theory and previous literature. This is also done when between-study heterogeneity is not high and in fixed effects meta-analysis
  • Fixed effect subgroup analyses: Make inference about subgroups included – sometimes other subgroups are not possible. For example sex: male vs female
  • Random effects subgroup analyses: Make inference about a range of possible subgroups, where you have a sample of some.
→  usually, same between-study heterogeneity assumed across subgroups. Therefore: approach to meta-analysis and subgroup analysis do not need to match.
JASP Meta-analysis menu →  Classical Meta-Analysis. Select your wanted Method for example Restricted underneath Method and also select your interested factor you want to compare underneath Factors
 
Subgroup analysis output
  • Line one of Table 1: = if statistically significant, at least one effect size (i.e., one subgroup) is significantly different (from ref.) So when it is significant it means one estimate differs significant from 0
  • Table 2: Intercept = effect for tonal stimuli (note: ref. chosen alphabetically. It shows the estimate of each factor, and whether they are different from 0
  • Other lines: if other stimuli were significantly different from your selected stimuli
  • Table 3: The influence on the between-study heterogeneity for accounting for different stimuli.
 
You can also use a continuous moderator
JASP meta-analysis menu →  Classical Meta-Analysis →  model. Select your components and Model Terms (the same) and make sure to include intercept
 
Limitations subgroup analysis
  • A lot of power is needed to detect differences in subgroups 
  • anticipated differences are normally small. If not statistically 
  • significant, does not mean that the pooled effects are the 
  • same. (Total k <10, subgroup analysis not advised)
  • Analysis is observational – studies differ for many reasons, confounders may be present
 

Image

Access: 
Public

Image

Check more of topic:
Going abroad?
Study with summaries
Search a summary

Image

 

 

Contributions: posts

Help other WorldSupporters with additions, improvements and tips

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Image

Spotlight: topics

Check the related and most recent topics and summaries:

Image

Check how to use summaries on WorldSupporter.org

Back to top

Online access to all summaries, study notes en practice exams

Back to top

How and why use WorldSupporter.org for your summaries and study assistance?

  • For free use of many of the summaries and study aids provided or collected by your fellow students.
  • For free use of many of the lecture and study group notes, exam questions and practice questions.
  • For use of all exclusive summaries and study assistance for those who are member with JoHo WorldSupporter with online access
  • For compiling your own materials and contributions with relevant study help
  • For sharing and finding relevant and interesting summaries, documents, notes, blogs, tips, videos, discussions, activities, recipes, side jobs and more.
Back to top

Using and finding summaries, notes and practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the summaries home pages for your study or field of study
  2. Use the check and search pages for summaries and study aids by field of study, subject or faculty
  3. Use and follow your (study) organization
    • by using your own student organization as a starting point, and continuing to follow it, easily discover which study materials are relevant to you
    • this option is only available through partner organizations
  4. Check or follow authors or other WorldSupporters
  5. Use the menu above each page to go to the main theme pages for summaries
    • Theme pages can be found for international studies as well as Dutch studies

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Back to top

Quicklinks to fields of study for summaries and study assistance

Main summaries home pages:

Main study fields:

Main study fields NL:

Back to top
WorldSupporter: what are the features, functionalities and rules on WorldSupporter.org?
WorldSupporter NL: hoe vind je samenvattingen en studiehulp op WorldSupporter.org en JoHo.org
Summaries and Study Assistance - Start
Follow the author: Yara Claassen
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Statistics
1019