aantekeningen lecture 2 Academic Research Methods and Statistics (ARMS) module 1 Meta-analysis 22/23 Universiteit Utrecht
Meta-analysis in Jasp
 
Setting up your Meta-Analysis (MA) data is manually done
  • study labels are used
  • potential moderators are determined
 
Meta-analysis menu →  Classical Meta-Analysis Menu
  • Here you can see variables in the dataset on the left side
  • Select Hedges_g for effect size
  • select SE for Effect sge Standard error
  • Underneath Method you can select what kind of meta-analysis you want to perform, for example Fixed Effect
 
forest plots: most common form used to convey the results in meta-analysis. The pooled effect is noted with a diamond (RE Model). The Weight of the studies is seen via the thickness of the square. 
JASP Meta-analysis menu →  Classical Meta-Analysis →  statistics. Select Forest plot underneath model fit
 
Publication bias: your meta-analysis is as good as the data. So when the data (studies) are biased your results will also be biased. Some attempts to minimize this is by purposively looking in pre-print repositories and theses
 
Funnel plot: studies with a small n that are statistically significant, must have a high effect size and SE. A funnel plot is a scatter plot showing the relationship between SE and observed effect size. 
  • If there is no publication bias the points must be fairly symmetrical, and form an upside-down tunnel. There must be more studies at the top and fewer at the bottom
  • When there is publication bias the points are asymmetrical
JASP Meta-analysis menu --> Classical Meta-Analysis --> statistics--> funnel plot (underneath model fit)
 
other issues that can cause asymmetry in the Funnel Plot
  • between-study heterogeneity: effect size plotted by only, does not control for different true underlying effects
  • Smaller studies could have higher effect sizes because study quality was better (e.g., treatment fidelity)
  • Lower quality studies tend to have larger effect sizes (other sources of bias)
 
Egger's Regression Test: A quantitative way to test for asymmetry in the funnel plot Tests whether the Y intercept = 0 (no bias) in a linear regression of normalized effect estimates (observed estimate divided by its standard error) against precision (inverse of the standard error of the estimate)
If there is no publication bias, then the intercept in Egger's regression test should be close to 0
JASP Meta-analysis menu →  Classical Meta-Analysis → statistics→  Regression test for funnel plot asymmetry (underneath model fit)
 
Trimm & Fill Method:
  • Trimming: – removing outlying studies that cause the asymmetry recalculate the pooled effect
  • Filling: using the recalculated pooled effect as the center, imputing hypothetical studies to mirror trimmed studies. The recalculated pooled effect this time is the corrected pooled effect
JASP Meta-analysis menu →  Classical Meta-Analysis →  diagnostics →  Trim fill analysis (underneath plots). 
In plots you will observe studies that are now labeled filled. 
 
Pooled effect in a random-effects meta-analysis: mean true effect of a range of true effects. Says nothing about how large range is
  • Between-study heterogeneity: extent that true effects vary Most common way of capturing between-study heterogeneity: Cochran’s Q and I2 When heterogeneity is high – could be explained by subgroups of studies showing differing effects
  • Outliers: observed effect sizes that are quite a bit different to the others. can have a large impact on the pooled effect. Make sure to recalculate the pooled effect once excluding studies where the confidence intervals does not overlap with the pooled effect confidence intervals.
 
Influential studies: When the pooled effect relates particularly on a influential study. To check if this is the case or if the pooled effect is robust you can refund the meta-analysis k times, excluding 1 study each time = leave out-analysis 
JASP: Meta-analysis menu →  Classical Meta-Analysis →  diagnostics →  case-wise diagnostics (underneath robustness). 
You will now see a table named influence measures. Make sure to look at Std. Residuals, DFFITS (Difference in Fits) and Qe. Also look at the forest plot
 
Between-Subgroup/Moderator analyses
  • Numerous subgroup analyses: decide a priori based on theory and previous literature. This is also done when between-study heterogeneity is not high and in fixed effects meta-analysis
  • Fixed effect subgroup analyses: Make inference about subgroups included – sometimes other subgroups are not possible. For example sex: male vs female
  • Random effects subgroup analyses: Make inference about a range of possible subgroups, where you have a sample of some.
→  usually, same between-study heterogeneity assumed across subgroups. Therefore: approach to meta-analysis and subgroup analysis do not need to match.
JASP Meta-analysis menu →  Classical Meta-Analysis. Select your wanted Method for example Restricted underneath Method and also select your interested factor you want to compare underneath Factors
 
Subgroup analysis output
  • Line one of Table 1: = if statistically significant, at least one effect size (i.e., one subgroup) is significantly different (from ref.) So when it is significant it means one estimate differs significant from 0
  • Table 2: Intercept = effect for tonal stimuli (note: ref. chosen alphabetically. It shows the estimate of each factor, and whether they are different from 0
  • Other lines: if other stimuli were significantly different from your selected stimuli
  • Table 3: The influence on the between-study heterogeneity for accounting for different stimuli.
 
You can also use a continuous moderator
JASP meta-analysis menu →  Classical Meta-Analysis →  model. Select your components and Model Terms (the same) and make sure to include intercept
 
Limitations subgroup analysis
  • A lot of power is needed to detect differences in subgroups 
  • anticipated differences are normally small. If not statistically 
  • significant, does not mean that the pooled effects are the 
  • same. (Total k <10, subgroup analysis not advised)
  • Analysis is observational – studies differ for many reasons, confounders may be present
 
Access: 
Public
Join WorldSupporter!

Join with an account for more service, or become a member with full access and support of WordSupporter

Check more of this topic?
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Search other summaries?

Image

Follow the author: Yara Claassen
More contributions of WorldSupporter author: Yara Claassen:
Comments, Compliments & Kudos:

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.
Promotions
vacatures

JoHo kan jouw hulp goed gebruiken! Check hier de diverse studentenbanen die aansluiten bij je studie, je competenties verbeteren, je cv versterken en een bijdrage leveren aan een tolerantere wereld

Check how to use summaries on WorldSupporter.org


Back to top

Online access to all summaries, study notes en practice exams

Back to top

Using and finding summaries, study notes en practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the menu above every page to go to one of the main starting pages
    • Starting pages: for some fields of study and some university curricula editors have created (start) magazines where customised selections of summaries are put together to smoothen navigation. When you have found a magazine of your likings, add that page to your favorites so you can easily go to that starting point directly from your profile during future visits. Below you will find some start magazines per field of study
  2. Use the topics and taxonomy terms
    • The topics and taxonomy of the study and working fields gives you insight in the amount of summaries that are tagged by authors on specific subjects. This type of navigation can help find summaries that you could have missed when just using the search tools. Tags are organised per field of study and per study institution. Note: not all content is tagged thoroughly, so when this approach doesn't give the results you were looking for, please check the search tool as back up
  3. Check or follow your (study) organizations:
    • by checking or using your study organizations you are likely to discover all relevant study materials.
    • this option is only available trough partner organizations
  4. Check or follow authors or other WorldSupporters
    • by following individual users, authors  you are likely to discover more relevant study materials.
  5. Use the Search tools
    • 'Quick & Easy'- not very elegant but the fastest way to find a specific summary of a book or study assistance with a specific course or subject.
    • The search tool is also available at the bottom of most pages

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Back to top

Quicklinks to fields of study for summaries and study assistance

Field of study

Back to top
Check related topics:
Activities abroad, studies and working fields
Institutions and organizations
Access level of this page
  • Public
  • WorldSupporters only
  • JoHo members
  • Private
Statistics
718