A systematic review is a form of analysis that medical researchers carry out to synthesize all the available evidence on a particular question, such as how effective a drug is.
A meta-analysis is a type of systematic review. Instead of basing conclusions on a single study, a meta-analysis looks at numerous studies for the answer.
It pools numerical analyses from studies of similar design. A meta-analysis can also form part of a further systematic review.
A panel of experts usually leads the researchers who carry out a systematic review. There are set ways to search for and analyze the medical literature.
A systematic review is a high form of evidence. The conclusions help medical experts to form an agreement on the best form of treatment.
The findings also inform policies set by state healthcare systems, such as whether they should fund a new drug.
The BMJ define a systematic review as “an overview of primary studies that used explicit and reproducible methods.”
Researchers carry out systematic reviews of all the available medical evidence and specifically of primary research. Primary research is data that researchers have collected from patients or populations.
Experts then base recommendations, or guidelines, on these findings. These guidelines lay out the treatment choices that health care providers and professionals should follow.
Researchers must carry out these reviews in a specific way, because they must ensure the recommendations that follow will result in the best healthcare for patients.
There are step-by-step instructions for conducting systematic reviews.
The Cochrane Library is a collection of systematic reviews that the international medical community respects. It follows a scientifically rigorous protocol to produce robust reviews.
The 2011 Cochrane Handbook for Systematic Reviews of Interventions lays out the guidelines that Cochrane require scientists to follow.
The Cochrane Library asks researchers to follow the steps below when producing a review. They provide a meticulous process through which researchers can synthesize data from a range of studies.
1: Define the research question
Researchers must first decide what research question they need an answer for. The aim could be, for example: “To assess the effects of a new drug for a particular health problem in certain types of people.” The question needs to be very specific.
2: Decide which studies to include in the review
The research question will partly decide this, but further “eligibility criteria” will define in advance which studies the team will include or exclude. The studies must have a rigorous design, for example, a randomized control trial (RCT).
3: Search for the studies
Step 3 outlines which sources the researcher will consult and the search terms they will use to search for them. In a Cochrane review, specially trained search coordinators do this. The researchers should also try to identify unpublished studies.
4: Select the studies and collect the data
Researchers take data from studies that meet the predetermined eligibility criteria. The data may have to come from a variety of formats.
5: Assess the risk of bias in the included studies
This ensures that all the studies reviewed are relevant and reliable.
- Was the randomization in the trial double-blinded?
- Was there a risk of bias, for example, in selecting participants for treatment or comparison?
It is acceptable to include some studies of a lower quality, as long as the researchers take this kind of bias into account.
6: Analyze the data and undertake meta-analyses
This is the core process of a systematic review. It is the main step toward synthesizing conclusions. The previous steps must be complete before carrying out this step.
7: Address any publication bias
Publication bias is when researchers specifically choose, or cherry-pick, a study for inclusion. This can lead to a misrepresentation of the true effects of treatment.
Researchers should avoid cherry-picking and usually sign an agreement that they have no vested interest in the work. For instance, if they work for a pharmaceutical company and are supporting a drug made by that company they must disclose it.
8: Present the final results of the review
The team publishes the work, with a table showing a summary of findings. Decision makers can use this published outcome.
A systematic review is a synthesis or overview of all the available evidence about a particular medical research question. Based on the evidence currently available, it can give a definitive answer on a particular question about therapy, prevention, causes of disease, or harm.
The conclusions of a review are more reliable than those from a single study.
The BMJ list the following as key advantages of a systematic review:
- The methods that scientists use to find and select studies reduce bias and are more likely to produce reliable and accurate conclusions.
- A review summarizes findings from multiple studies. This makes the information easier for the end user to read and understand.
It is helpful for establishing whether a certain technique or drug works and is safe.
A review can also:
- give an idea of how well findings might apply to everyday practice
- identify knowledge gaps that call for more research
- reduce bias when drawing conclusions, as it takes in a range of views and findings
Systematic reviews also offer practical advantages. They are less costly to carry out than a new set of experiments, and they take less time.
A systematic review may have some disadvantages.
It can be hard to combine the findings of different studies, because the researchers have carried out their investigation in different ways.
The number of participants, the length of the original study, and many other factors can make it hard to compare the findings of two or more studies.
Authors of a review must decide whether the quality of a source is “high” or “low,” in other words how reliable each one is. The decision usually depends the design of the study.
For instance, a randomized controlled trial is considered the highest of the primary studies. Other recommendations include transparency and reproducibility of judgments.
The role of unpublished research
If researchers only use published or readily available studies, it could be a threat to the validity of a review. This occurs because researchers tend to publish studies that show a significant effect and may not take the time to write up negative findings.
Unpublished studies can be hard to find, but using published literature alone may lead to misrepresentation because it does not include findings from all the existing research.
The term gray literature refers to articles or books not formally published and may include government reports, conference proceedings, graduate dissertations, unpublished clinical trials and more.
As previously mentioned, results that are negative or inconclusive, for example, may remain unpublished. Publication bias can cause positive results to become exaggerated, because the findings do not incorporate neutral or negative results.
Medical researchers are less likely to submit bad results, so systematic reviews could have a bias towards good results.
The role of editors and peer reviewers
The decisions of journal editors and peer reviewers can also lead to publication bias.
Sometimes, results do not reach the publication stage because there is funding for research, but this does not cover the cost of analyzing and publishing the results.
This can limit the motivation to write up and submit any negative or neutral findings for publication.
In 2011, the Institute of Medicine (IOM) noted that systematic reviews can help clinicians make good decisions in their daily practice and help health organizations to prepare guidelines.
However, they added that systematic reviews can also be “uncertain or poor quality,” due to a lack of universal standards, especially when it comes to bias, conflicts of interest, and how authors evaluate evidence.
In an attempt to counter this, the IOM recommend some standards for authors to follow at each stage.
They provide guidelines for a number of areas, including:
A meta-analysis uses a statistical approach to summarize the results of other studies, all of which must have a similar design. It aims to provide reliable evidence.
Using statistical analysis, researchers combine the numbers from previous studies, and they use this information to calculate an overall result.
The BMJ define a meta-analysis as “a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.”
As with a review, authors must follow certain steps.
A meta-analysis can stand alone, or it can be part of a wider systematic review. A wider review can include results from studies of various scientific designs.
A meta-analysis can provide more reliable evidence than other investigations, but still the results may not always apply directly to the everyday treatment of disease.
Simple numerical answers cannot solve complex clinical problems, however, and they cannot tell a clinician how to treat a person.
A meta-analysis may also conclude, for example, that antibiotics are effective in treating a disease, but they are unlikely to specify the type, dosage, or how a specific antibiotic will affect an individual.
More studies and trials are necessary before healthcare providers can make these kinds of decision.
Medical research is crucial for understanding what works, what does not work, and whether a strategy or a drug is safe.
Systematic reviews and meta-analyses bring together the findings of several investigations. In theory, this makes the findings more reliable.
However, even this type of report has its pitfalls.
Whether they look at the findings of an investigation, a review, or a meta-analysis, healthcare professionals must always interpret the findings with care.
In the case of drugs and new medical techniques, clinical trials are necessary to get a better view of their safety and effectiveness.