Statistics
Systematic Review and Meta-Analysis for Beginners: PRISMA, Forest Plots and Heterogeneity
A systematic review answers a focused question by finding, appraising and summarising all relevant studies using a predefined, reproducible method. A meta-analysis goes one step further and statistically combines the results of similar studies into one pooled estimate.
Resora Research Team · 15 Nov 2026 · 6 min read
A systematic review answers a focused question by finding, appraising and summarising all relevant studies using a predefined, reproducible method. A meta-analysis goes one step further and statistically combines the results of similar studies into one pooled estimate. The process runs from a PICO question and registered protocol through a comprehensive multi-database search, dual screening, data extraction and risk-of-bias assessment, to synthesis and a PRISMA-compliant report. Results are shown in a forest plot; heterogeneity (I²) tells you whether studies are similar enough to pool; a funnel plot checks for publication bias; and GRADE rates the certainty of the evidence. Systematic reviews sit at the top of the evidence pyramid, but only if the underlying studies and methods are sound.
Systematic review vs narrative review
| Narrative (traditional) review | Systematic review | |
|---|---|---|
| Question | Broad | Focused (PICO) |
| Search | Not stated; may be selective | Comprehensive, multi-database, reproducible |
| Study selection | Author's choice | Predefined eligibility criteria, two reviewers |
| Quality assessment | Often absent | Formal risk-of-bias assessment |
| Synthesis | Descriptive | Structured, with or without meta-analysis |
| Bias | High risk | Minimised by method |
A meta-analysis is a statistical technique that can be part of a systematic review. A systematic review does not always include one: when studies are too different, results are summarised in words (narrative synthesis).
Steps in a systematic review
- Frame the question with PICO. Population, Intervention (or exposure), Comparator, Outcome. Example: In adults with type 2 diabetes (P), does metformin (I) compared with lifestyle change alone (C) reduce HbA1c (O)? For prevalence questions, use PEO or CoCoPop (Condition, Context, Population).
- Check that no recent review already exists, then register your protocol on PROSPERO (for health-related reviews) or OSF before you search. This prevents changing your methods after seeing results.
- Define eligibility criteria: study designs, populations, interventions, outcomes, languages and years.
- Search comprehensively. Use at least two or three databases (PubMed/MEDLINE, Embase, Cochrane CENTRAL, Scopus, Web of Science; CINAHL or PsycINFO for specific topics). Combine MeSH terms and free-text words with AND/OR, search trial registries and grey literature, and check reference lists. Record the full search strategy so others can repeat it.
- Remove duplicates and screen. Two reviewers independently screen titles and abstracts, then full texts, resolving disagreements by discussion or a third reviewer. Tools such as Rayyan or Covidence help.
- Extract data with a pilot-tested form: study characteristics, participants, interventions, outcomes and results, again by two people.
- Assess risk of bias in each study (see below).
- Synthesise the findings narratively and, where appropriate, statistically.
- Assess certainty of the evidence with GRADE.
- Report following PRISMA 2020, with a flow diagram showing records identified, screened, excluded (with reasons) and included.
A good search, in practice
("Diabetes Mellitus, Type 2"[MeSH] OR "type 2 diabetes"[tiab])
AND ("Metformin"[MeSH] OR metformin[tiab])
AND (randomized controlled trial[pt] OR randomized[tiab])Synonyms joined with OR, concepts joined with AND. Pilot it, check whether known key studies are found, and refine it.
Risk of bias: judging study quality
Pooling poor studies gives a precise but wrong answer ("garbage in, garbage out"). Use a tool that fits the design:
| Study design | Common tool |
|---|---|
| Randomised trials | Cochrane RoB 2 |
| Non-randomised intervention studies | ROBINS-I |
| Cohort and case–control | Newcastle–Ottawa Scale |
| Cross-sectional / prevalence | JBI critical appraisal checklist |
| Diagnostic accuracy | QUADAS-2 |
Two reviewers assess each study independently. Do not just exclude low-quality studies. Report them, and test whether they change your result in a sensitivity analysis.
Meta-analysis: combining the results
Each study gives an effect estimate (odds ratio, relative risk, mean difference, prevalence) with a confidence interval. The meta-analysis computes a weighted average, in which larger and more precise studies get more weight (inverse variance weighting).
Fixed-effect vs random-effects model
| Fixed-effect | Random-effects | |
|---|---|---|
| Assumes | One true effect shared by all studies | True effects vary between studies |
| Use when | Studies are very similar | Studies differ in population or methods (the usual situation) |
| Result | Narrower CI | Wider, more cautious CI |
In medicine, random-effects is the usual default because populations and methods differ between studies.
Reading a forest plot
The forest plot is the signature picture of a meta-analysis. Each row is one study:
- A square marks the study's estimate; its size shows its weight.
- A horizontal line through it shows the 95% CI.
- A vertical line marks "no effect" (1 for ratios, 0 for differences).
- A diamond at the bottom shows the pooled result; its width is the pooled CI.
- If the diamond does not cross the vertical line, the pooled effect is statistically significant.
Heterogeneity: can the studies be pooled?
Heterogeneity is variation in results between studies beyond chance.
- I² is the percentage of variation due to real differences, not chance. A rough guide: 0–40% might not be important, 30–60% moderate, 50–90% substantial, and 75–100% considerable.
- The Cochran Q test (p < 0.10 suggests heterogeneity) is less reliable with few studies.
- If I² is high, explore the reasons with subgroup analysis or meta-regression (by age, dose, region, study quality), or do not pool at all.
Sensitivity analysis
Repeat the analysis leaving out one study at a time, or only high-quality studies, to see whether the conclusion holds.
Publication bias
Studies with positive results are more likely to be published, so a review of published studies may overstate effects.
- A funnel plot shows each study's effect against its precision. Without bias it looks like a symmetric upside-down funnel; a missing corner suggests unpublished negative studies.
- Egger's test checks asymmetry statistically (and needs about 10 or more studies).
- Reduce it by searching trial registries, conference abstracts and grey literature.
GRADE: how certain are we?
GRADE rates the certainty of evidence for each outcome as high, moderate, low or very low. Randomised trials start high and observational studies start low. Certainty is lowered for risk of bias, inconsistency, indirectness, imprecision and publication bias, and raised for large effects or a dose–response gradient.
Meta-analysis of prevalence and of diagnostic tests
These need special methods. Prevalence data are pooled on a transformed scale (such as logit or Freeman–Tukey), usually with random effects. Diagnostic accuracy uses bivariate or hierarchical models for sensitivity and specificity together. Ask a biostatistician or use the dedicated software.
Software
| Tool | Cost | Use |
|---|---|---|
| RevMan (Cochrane) | Free | Standard for Cochrane-style reviews, forest and funnel plots |
R (meta, metafor) | Free | Flexible, reproducible, most analyses |
| Jamovi (MAJOR module) | Free | Point-and-click |
| Rayyan / Covidence | Free tier / paid | Screening and extraction |
| Stata, CMA | Paid | Full-featured |
Common mistakes
- No protocol or registration, so methods can change after seeing the data.
- A search in only one database.
- Single-reviewer screening and extraction, which raises the error rate.
- Pooling very different studies ("apples and oranges") despite high heterogeneity.
- Ignoring risk of bias, or not using it in the interpretation.
- Treating a narrative review as a systematic review.
- Not reporting excluded studies and the reasons.
- Overstating conclusions from a pooled result built on weak studies.
- Skipping the PRISMA checklist and flow diagram.
Keywords
systematic review, meta-analysis, systematic review for beginners, PRISMA, PRISMA flow diagram, forest plot, heterogeneity I2, funnel plot, publication bias, PROSPERO, PICO question, risk of bias tools, GRADE, random effects model, systematic review vs narrative review
References
- Page MJ, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.
- Higgins JPT, et al., eds. Cochrane Handbook for Systematic Reviews of Interventions. Cochrane; training.cochrane.org/handbook.
- Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–560.
- Guyatt GH, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–926.
- Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. Wiley; 2009.