Protocol Development

Protocol Registration

Register the protocol on a platform like PROSPERO or OSF, which allows others to view and verify the review's planned approach, ensuring transparency and minimizing the risk of research duplication.

Purpose and Rationale

Clearly define the review's objectives and justify why it is necessary. This section should describe the knowledge gap or clinical/practical problem the review aims to address.

Example: “This review aims to assess the effectiveness of aerobic exercise in reducing anxiety symptoms in adults, filling a gap in comparative effectiveness research.”

Research Question

Formulate a focused question using a framework like PICO (Population, Intervention, Comparator, Outcome). This structure helps maintain clarity and focus throughout the review.

Example: “In adults with anxiety (Population), does aerobic exercise (Intervention) compared to no exercise (Comparator) improve anxiety symptoms (Outcome)?”

Inclusion and Exclusion Criteria

Define criteria for study eligibility, detailing aspects like study design, population, interventions, comparisons, and outcomes. Set additional limits for language, publication date, and study setting if applicable.

Example: “Only randomized controlled trials on adults with clinically diagnosed anxiety reporting on symptom reduction will be included.”

Search Strategy

Outline the search strategy to be used for identifying studies, specifying databases (e.g., PubMed, Cochrane, Embase), search terms, Boolean operators, and any planned manual search methods.

Example: “A comprehensive search will be conducted in PubMed, Embase, and PsycINFO, using terms like ‘aerobic exercise’ AND ‘anxiety’ with Boolean operators and field tags.”

Screening and Selection Process

Describe the process for screening and selecting studies, including who will screen titles and abstracts, who will resolve disagreements, and the tools or software that will be used.

Example: “Two reviewers will independently screen titles and abstracts, with disagreements resolved by a third reviewer.”

Data Extraction

Provide a detailed plan for extracting relevant data, such as study characteristics, population demographics, intervention details, and outcome measures.

Example: “Data extraction will capture information on sample size, intervention frequency, duration, and primary outcome measures.”

Risk of Bias Assessment

Specify the tools or frameworks that will be used to assess the quality and risk of bias in included studies, like the Cochrane Risk of Bias tool for RCTs or ROBINS-I for non-randomized studies.

Example: “The Cochrane Risk of Bias (RoB 2) tool will be used to assess included RCTs, with two reviewers independently scoring each study.”

Data Synthesis and Analysis

Describe how data will be synthesized, either narratively or through a meta-analysis if pooling is possible. Include plans for subgroup analyses, sensitivity analyses, or any statistical tests.

Example: “Studies will be pooled using a random-effects model. Subgroup analysis will explore exercise duration as a potential moderator of effect size.”

1. Define Key Concepts Using Frameworks

Identify the main elements of your research question. Frameworks like PICO (Population, Intervention, Comparator, Outcome) can help break down the question into manageable concepts.

2. Identify Keywords and Synonyms

For each key concept, identify potential keywords and synonyms. Consider variations in spelling, abbreviations, and related terms to avoid missing studies.

3. Use Boolean Operators

Combine keywords using Boolean operators:

• AND: Combines different concepts to narrow the search.
• OR: Includes synonyms or related terms, broadening the search.
• NOT: Excludes unwanted terms.

4. Use Field Tags and Truncation

Many databases allow field tags (e.g., [tiab] in PubMed for title and abstract) to restrict searches to specific fields. Truncation (e.g., exercise* for exercise, exercises, exercising) helps include variations of a term.

5. Develop Separate Strategies for Each Database

Adapt the search strategy to each database's unique indexing system. For instance, PubMed uses MeSH terms (Medical Subject Headings), while Embase uses Emtree terms.

Phase 1: Initial Title/Abstract Screening Process

Begin with title and abstract screening to identify potentially relevant studies. This initial phase helps eliminate clearly irrelevant papers while flagging promising candidates for full-text review.

Phase 2: Full-Text Review

Conduct detailed assessment of full-text articles that passed initial screening. Apply inclusion/exclusion criteria rigorously and document reasons for exclusion.

Documentation

Maintain detailed records of the screening process, including decisions made and reasons for exclusion. This documentation is essential for creating the PRISMA flow diagram and ensuring transparency.

Randomized Controlled Trials (RCTs)

Use the Cochrane Risk of Bias (RoB 2) tool to assess:

• Random sequence generation
• Allocation concealment
• Blinding (participants, personnel, outcome assessment)
• Incomplete data
• Selective reporting
• Other biases

Judgments: Low, high, or unclear risk

Non-Randomized Studies (NRSI)

Use ROBINS-I to evaluate:

• Confounding
• Participant selection
• Intervention classification
• Deviations from intended interventions
• Missing data
• Outcome measurement
• Reported result selection

Judgments: Low, moderate, serious, or critical risk

Assessment Process

• Independent Assessment: Two reviewers independently assess each study
• Consensus: Disagreements resolved through discussion or third reviewer
• Documentation: Clear documentation of judgments and supporting evidence
• Reporting: Results presented in risk of bias tables and graphs

Purpose and Benefits

• Organize study data systematically
• Facilitate comparison across studies
• Support quality assessment and synthesis
• Aid in identifying patterns and trends across studies

Key Components

A typical evidence table includes the following elements:

• Study descriptives (author, year, country)
• Study design and methods
• Participant characteristics
• Intervention and comparison details
• Outcome measures
• Results
• Notes or comments

Building the Table

When constructing evidence tables:

• Develop a standardized template based on your review question and included study designs
• Extract data consistently across studies, using predefined data extraction forms
• Include all relevant outcomes, even if not reported by all studies
• Present numerical data clearly, including sample sizes, effect sizes, and confidence intervals where available
• Summarize qualitative findings concisely but informatively

Software Tools

Several tools can assist in creating evidence tables:

• Our in house Evidence Table Builder
• Spreadsheet programs (e.g., Excel, Google Sheets)
• Specialized data extraction tools (e.g., EPPI-Reviewer)

Steps in Conducting a Meta-Analysis

• Define research question and inclusion criteria
• Conduct systematic literature search
• Screen and select studies
• Extract data from included studies
• Calculate effect sizes
• Assess heterogeneity
• Conduct statistical analysis
• Interpret and report results

Effect Size Calculation

A key step is calculating standardized effect sizes for each study. Common effect size measures include:

• Standardized mean difference (Cohen's d, Hedges' g) for continuous outcomes
• Odds ratios or risk ratios for dichotomous outcomes
• Correlation coefficients for associations

Assessing Heterogeneity

Heterogeneity refers to variability in the true effects across studies. It's assessed using:

• Cochran's Q test
• I² statistic - percentage of variability due to heterogeneity rather than chance
• Tau² - estimate of between-study variance

High heterogeneity may indicate the need for subgroup analyses or meta-regression.

Statistical Analysis

The main approaches are:

• Fixed-effect model - assumes a common true effect size
• Random-effects model - allows for variation in true effects between studies

A random-effects model is often preferred when heterogeneity is present.

Forest Plot Interpretation

Forest plots visually display the results, showing:

• Individual study effect sizes and confidence intervals
• Overall pooled effect size
• Heterogeneity statistics

The plot allows assessment of the consistency and precision of effects across studies.

Considerations

• Publication bias should be assessed (e.g., funnel plots, Egger's test)
• Sensitivity analyses can test the robustness of findings
• Results should be interpreted in context of study quality and limitations

By systematically combining and analyzing results across studies, meta-analysis provides a powerful tool for synthesizing evidence and estimating overall effects in a research area. Proper conduct and interpretation are crucial for drawing valid conclusions.

Narrative Synthesis

Narrative synthesis is a qualitative approach that involves:

• Organizing studies into logical categories
• Exploring relationships within and between studies
• Assessing the robustness of the synthesis

This method is particularly useful when studies are too heterogeneous for meta-analysis.

Meta-Analysis

Meta-analysis is a statistical method for combining quantitative results from multiple studies. Key steps include:

• Calculating effect sizes for individual studies
• Pooling effect sizes using fixed-effect or random-effects models
• Assessing heterogeneity (e.g., using I² statistic)
• Conducting sensitivity analyses

Meta-analysis provides a precise estimate of the overall effect and increases statistical power.

Thematic Analysis

For qualitative studies, thematic analysis involves:

• Coding relevant data from primary studies
• Developing descriptive themes
• Generating analytical themes

This method allows for the integration of findings across qualitative studies.

Meta-Ethnography

Meta-ethnography is an interpretive method for synthesizing qualitative research. It involves:

• Determining how studies are related
• Translating studies into one another
• Synthesizing translations

This approach aims to produce new interpretations that go beyond the primary studies.

Content Analysis

Content analysis can be used to systematically categorize and quantify information from qualitative studies. It involves:

• Developing a coding framework
• Applying codes to study findings
• Analyzing the frequency and patterns of codes

Subgroup Analysis and Meta-Regression

These techniques explore how study characteristics influence outcomes:

• Subgroup analysis compares effects between predefined groups of studies
• Meta-regression examines the relationship between study-level covariates and effect sizes

Best Practices

• Choose analysis methods appropriate to your review question and included studies
• Prespecify analysis plans in your review protocol
• Assess the quality and strength of evidence (e.g., using GRADE approach)
• Consider both statistical significance and clinical importance
• Address heterogeneity and potential biases
• Conduct sensitivity analyses to test the robustness of findings

Reporting

When reporting your analysis:

• Clearly describe methods used
• Present results in both narrative and visual formats (e.g., forest plots for meta-analyses)
• Discuss limitations and potential biases
• Interpret findings in the context of existing knowledge and practice

By carefully analyzing and synthesizing data from included studies, systematic reviews can provide valuable insights and inform evidence-based decision-making in various fields.

Title and Authors

Abstract and Keywords

Introduction

• Background and rationale
• Objectives
• Research question (often using PICO format)

Methods

Results

• Study Selection (PRISMA flow diagram)
• Study Characteristics
• Risk of Bias Within Studies
• Results of Individual Studies
• Synthesis of Results
• Additional Analysis

Discussion

• Summary of Evidence
• Limitations (study, outcome, and review-level)
• Conclusions and implications for future research

Supporting Sections

• References (following journal-specific formatting)
• Appendix (supplementary materials, detailed search strategies, data extraction forms)
• Additional tables or figures not included in main text