Conduct a systematic literature review by defining a question, writing and registering a protocol, searching relevant sources, screening records and full texts, extracting data, assessing risk of bias, synthesizing findings and rating certainty, then reporting with PRISMA 2020. AI can assist, but reviewers must verify its suggested decisions and evidence.
TL;DR
A systematic literature review needs a protocol, reproducible searches and documented decisions.
AI screening and risk-of-bias judgments are drafts, not replacements for independent reviewers.
Choose appraisal tools by study design, assessment target and instrument version.
AnswerThis shows screening reasons and draft appraisal explanations for reviewers to check.
What is a systematic literature review?
A systematic literature review uses explicit, planned methods to identify, select, appraise and synthesize research answering a defined question. Unlike an informal overview based on selected reading, a systematic review follows a planned, repeatable strategy. The aim is to reduce bias and make the methods and conclusions open to inspection, not to guarantee that the conclusions are correct. See the fuller explanation of what a systematic review is.
A systematic review does not have to include a meta-analysis. Its credibility depends on the question, searches, eligibility decisions, extracted data and interpretation, not on a pooled number or polished diagram. This guide focuses on intervention reviews while explaining where study design changes the appraisal approach.
Step 1: Define the research question
Begin with a specific, focused and researchable question. For example, narrow “How is AI impacting education?” to “What effects do AI-based tutoring systems have on student performance in higher education?” Define the intervention, comparator and meaning of performance before translating that question into search terms.
For an intervention question, use PICO: population, intervention, comparator and outcomes. Specify who the review concerns, what intervention it evaluates, what it compares against and which outcomes matter. Other review questions need a framework appropriate to their purpose.
For an illustrative question, consider remote blood-pressure monitoring compared with usual care in adults with hypertension. Decide whether monitoring must include clinician feedback and which blood-pressure outcomes and follow-up periods qualify. This is a planning example, not a finding about effectiveness.
Step 2: Set inclusion and exclusion criteria and register the protocol
Turn the question into eligibility rules before screening. Define populations, interventions, comparators, study designs, settings and outcome requirements. Justify restrictions rather than adding language or publication-date limits for convenience. A recent-publication window or an English-only rule is not a default requirement: either can omit relevant evidence. Distinguish eligible study designs from publication types such as opinion pieces, and write rules clearly enough for reviewers to apply consistently.
Write a protocol covering searches, screening, extraction, appraisal, synthesis and disagreement resolution. Register it prospectively with an appropriate registry where eligible, or make a dated protocol publicly accessible. Record amendments and their reasons rather than silently changing the rules after seeing results.
In AnswerThis, eligibility criteria can be suggested by AI or entered by the reviewer. Treat suggested rules as drafts. Check that they express the actual question and do not exclude evidence merely because it is difficult to process.
Step 3: Search for relevant studies
Choose databases, registers and other sources that fit the question. Combine synonyms within a concept using OR and connect essential concepts using AND. Adapt vocabulary and syntax to each database interface, then test retrieval of known relevant papers. For the education example, a starting concept combination is ("AI tutoring" OR "intelligent tutoring") AND ("student performance" OR "academic achievement"). This is an illustrative fragment, not a complete or validated search strategy. Use NOT cautiously because it can remove relevant records.
The AnswerThis Search String Builder drafts editable queries. It does not execute searches. Review terminology, run the query in the destination interface and save the exact strategy, filters, search date and retrieved count.
Preserve exports and document deduplication. Link separate reports of the same study rather than discarding them as duplicate records. Cochrane Handbook chapter 4 explains why search design and study identification need documented methods.
Step 4: Screen titles and abstracts, then full texts
Assess titles and abstracts against the eligibility criteria, then retrieve potentially eligible reports for full-text assessment. Record reports that could not be retrieved separately from reports excluded after assessment. Uncertainty at abstract screening should prompt further inspection rather than an unsupported exclusion. Apply the protocol’s eligibility rules, rather than excluding a relevant study simply because it appears methodologically weak. Assess risk of bias separately and apply any prespecified appraisal-based exclusion rule transparently.
Cochrane standard C39 requires at least two people working independently to determine study eligibility, with disagreement resolution defined in advance. Inclusion decisions should use full texts where possible, usually after initial title-and-abstract screening. An AI suggestion or a feature called dual review does not itself satisfy that human-review requirement.
Gartlehner and colleagues’ 2020 randomized trial found that single-reviewer abstract screening missed 13% of relevant studies on average. It supports checking exclusions rather than accepting them uncritically.
AnswerThis supports title, abstract and full-text screening with reviewer override and reasons behind suggestions. If full text is missing, upload an authorized PDF and rerun the assessment. Keep the report identifier, screening stage, reviewer decisions, final decision and exclusion reason together, then reconcile disagreements against the protocol and source.
Step 5: Extract data from the included studies
Data extraction means collecting prespecified information from included studies into a structured record. Record the study identifier, authors and publication date alongside design, setting, population, sample size, interventions, comparators, outcomes, measurement methods, follow-up, results, funding and conflicts of interest as relevant. A spreadsheet can hold this record, provided fields are defined consistently and reviewers preserve source links and verification decisions. Retain the quotation, table or page behind consequential information.
For numerical results, keep the outcome, time point, comparison, effect measure, units and uncertainty together. A value detached from its denominator can be transcribed correctly but interpreted incorrectly. Link multiple reports of one study to avoid counting participants repeatedly.
Cochrane Handbook chapter 5 requires at least two people working independently to extract outcome data, with a process for resolving disagreement. Pilot the extraction form before applying it throughout the review. AI-assisted extraction does not supply those independent people.
The research paper data extractor with quotations and locations accepts text-layer PDF, TXT, Markdown or JATS XML up to 25 MB. Its eight fields cover limitations, methods, sample and population, effect sizes, outcomes, datasets, funding and conflicts of interest. Findings have quotations, locations and found, missing or ambiguous states, with CSV export.
The extractor does not recalculate effect sizes. Missing means information was not located in the supplied text, not that the full study necessarily omitted it. Review original tables and context before accepting a result.
The website tool includes two free papers per browser, shared across extraction presets and the summarizer. Text goes to the AnswerThis backend and OpenAI. The application does not store paper text or results, but provider retention terms apply: use authorized material and download reviewed results before leaving.
Step 6: Assess risk of bias
Risk of bias concerns whether study design, conduct, analysis or reporting could systematically distort a finding. It differs from reporting completeness, journal reputation and certainty across a body of evidence. Choose the instrument and assessment target before answering its questions.
Randomized intervention trial
Instrument to consider: RoB 2
Important boundary: Assess the specified result. AnswerThis’s website tool uses the 22 August 2019 individually randomized parallel-group version, not cluster or crossover variants.
Non-randomized intervention effect
Instrument to consider: ROBINS-I
Important boundary: Define the intervention effect and relevant confounding. AnswerThis’s website tool implements ROBINS-I 2016, not ROBINS-I V2.
Diagnostic test accuracy
Instrument to consider: QUADAS-3, successor to QUADAS-2
Important boundary: The University of Bristol recommends QUADAS-3. AnswerThis’s website tool implements QUADAS-2 2011, not QUADAS-3.
Cohort or case-control study
Instrument to consider: Newcastle-Ottawa Scale
Important boundary: Use the matching form. Do not treat a star total as a universal low-risk threshold.
Other eligible designs
Instrument to consider: Design-specific JBI checklist
Important boundary: Match the checklist to the design and record its edition. Preserve item-level appraisal.
This is a starting point, not permission to use the same tool for every observational study. ROBINS-I addresses intervention effects, while the Newcastle-Ottawa Scale has distinct cohort and case-control forms. Consult official guidance and your review’s methods requirements.
The AnswerThis risk of bias tool assesses one study per run and produces evidence-linked draft answers and explanations. It checks quotations against supplied text and removes unverifiable quotations. A quotation match establishes provenance, not that the interpretation is correct.
Read each judgment’s reason, consult the report and companion materials, and confirm or revise the assessment. Missing information is not proof of poor conduct. Retain unresolved questions, reviewer reasoning and the instrument version alongside the judgment.
The website tool includes one successful free study per browser and exports SVG, PNG, CSV and JSON. Study text goes to the AnswerThis backend and OpenAI and is not saved to a study database. Every result requires reviewer confirmation and does not replace independent reviewers.
Step 7: Synthesize the findings and rate certainty
Group studies by populations, interventions, comparators and outcomes. Consider differences in design, follow-up, measurement and risk of bias before pooling. Cochrane Handbook chapter 10 explains why meta-analysis can mislead when these decisions are inappropriate.
Use meta-analysis only when combining sufficiently comparable studies answers a meaningful question. Otherwise, provide a structured narrative synthesis explaining findings, differences and uncertainty. Compare what studies actually measured rather than simply assembling summaries. Organize the synthesis around comparisons, themes or methods, then explain where findings agree, conflict or remain uncertain. Identify evidence gaps precisely, distinguishing an absence of eligible studies from limitations in the studies available.
GRADE rates certainty in a body of evidence for an outcome as high, moderate, low or very low. This differs from an individual study’s risk-of-bias judgment. Explain each certainty decision rather than displaying a label without justification.
AnswerThis’s workflow includes extraction, a meta-analysis section and certainty ratings. Reviewers must check data, analysis choices and reasons behind suggested ratings. Software output alone does not establish that pooling or a certainty judgment is appropriate.
Step 8: Write and report the review with PRISMA 2020
Report the question, protocol, searches, selection, study characteristics, appraisal, synthesis and limitations. Explain deviations from the plan and where supporting records can be inspected. Use PRISMA 2020 to report what the team did, not to imply software certification of methods. Write the results around the review’s comparisons, themes and findings rather than giving each paper an isolated summary. Attach supporting citations to substantive claims and check that the cited evidence supports the interpretation.
AnswerThis’s systematic review workflow can produce a written review draft and export review steps. Treat that draft as a starting point: verify citations, reconcile it with the reviewed data and appraisal records, and retain responsibility for the final text.
Build the flow diagram from reconciled logs. Distinguish records, reports and studies, since one study can have several reports. Keep non-retrieved reports separate from full-text exclusions and preserve exclusion reasons.
The PRISMA flow diagram guide explains template selection and counting. AnswerThis produces a PRISMA flow diagram. It also keeps an audit trail of every step, so your report can document each PRISMA 2020 item and Cochrane step.
Using AI in a systematic review: what the November 2025 position statement says
AI can assist screening, extraction and risk-of-bias assessment, but the team remains responsible for its use and outputs. The November 2025 position statement from Cochrane, the Campbell Collaboration, JBI and the Collaboration for Environmental Evidence calls for human oversight. It permits AI within those organizations’ syntheses where authors can demonstrate that rigor and integrity are not compromised.
The statement calls for transparent reporting when AI makes or suggests judgments, including eligibility, appraisal, extraction, synthesis and certainty assessment. AnswerThis supports this with a logged record of every AI suggestion and reviewer decision.
Document the tool, version and dates used, tasks affected, justification, validation or calibration, human verification and limitations. Check confidentiality, permissions and institutional rules before uploads. Retain enough evidence for another reviewer to inspect decisions.
AnswerThis shows screening reasons and draft appraisal explanations, logs changes and permits exports of review steps. These features support an inspectable workflow. They do not remove the need to test suitability, maintain independent human checks or explain error handling.
Common mistakes to avoid
Starting with a vague question: specify the population, intervention or phenomenon, and outcomes before setting search terms.
Summarizing one paper after another: compare methods, findings and limitations to answer the review question.
Changing eligibility after seeing results: date and justify amendments.
Using citation counts as an unplanned eligibility filter: relevance is not popularity.
Treating AI as an independent person: retain the human-review process required by your protocol and applicable standards.
Confusing missing information with a negative finding: inspect reports and companion materials.
Applying the wrong appraisal edition: record the design, tool, version and target.
Pooling incompatible results: justify synthesis choices first.
Calling the review compliant because it has a diagram: reporting and methods need separate checks.
FAQs
Can AI replace an independent reviewer for screening or risk-of-bias assessment?
No. Treat AI eligibility suggestions and risk-of-bias judgments as drafts requiring human verification. Cochrane C39 requires at least two people working independently on study eligibility, and the November 2025 joint position statement calls for oversight and transparent reporting of AI that makes or suggests judgments.
Which risk-of-bias tool matches my included studies?
Match the instrument to the design and assessment target: RoB 2 for randomized intervention results, ROBINS-I for non-randomized intervention effects, QUADAS-3 for diagnostic accuracy, or a suitable Newcastle-Ottawa form or JBI checklist. Check the implemented edition: AnswerThis’s website tool uses QUADAS-2 and ROBINS-I 2016, not QUADAS-3 or ROBINS-I V2, and its RoB 2 implementation covers individually randomized parallel-group trials only.
What should I do when extracted information is missing or ambiguous?
Inspect the source report, relevant tables and companion materials before accepting the extraction status. Missing means the tool did not locate information in the supplied text, not that the study necessarily omitted it. Preserve unresolved questions and reviewer decisions alongside the extraction record.
Does every systematic review need a meta-analysis?
No. Pool results only when the studies and available data support a meaningful combined estimate. When pooling is inappropriate, use a structured narrative synthesis that compares findings, methods, limitations and uncertainty.
How should I choose AI systematic review software?
Choose software that fits your protocol and lets reviewers inspect evidence, revise suggested decisions, record reasons and export records. Check instrument versions and data-handling terms, then test the workflow on your own review material.
Methodological sources
Before starting: agree the protocol, independent-review process and evidence trail. A reason shown by software becomes a research decision only after the team evaluates it.
