A systematic review attempts to identify, select, synthesize, and appraise all evidence on a given research question in a systematic and transparent manner.
Frequently Asked Questions:
Steps for completing a systematic review
Uman (2011) breaks down the process of systematic reviews and meta-analyses into 8 steps:
- Formulate the review question
- Define inclusion/exclusion criteria
- Develop reproducible search strategies and locate studies
- Select studies
- Extract data
- Assess study quality
- Analyze and interpret results
- Disseminate findings
Click here for: Guides on Conducting Systematic Reviews.
Systematic reviews versus narrative reviews
- Searching for relevant studies - Unlike narrative reviews, systematic reviews attempt to locate all relevant published and unpublished studies on a topic to limit the impact of biases.
- Deciding which studies to include or exclude - Narrative reviews usually do not address why studies are included or excluded whereas systematic reviews explicitly describe what types of studies are to be included or excluded to reduce selection bias by the reviewer.
- Assessing study quality - While narrative reviews may not consider the methods or quality of the studies included in the review, systematic reviews systematically examine the methods of original studies and investigate potential biases.
- Synthesizing study results - Whereas narrative reviews may not differentiate between methodologically sound and unsound studies, systematic reviews base their conclusions on the most methodologically sound studies.
Systematic reviews versus meta-analyses
A meta-analysis uses a statistical procedure to combine the results of individual studies. This provides a statistical estimate of net benefit aggregated over all included studies (Crombie & Davies, 2009). For example, if the results of a quantitative systematic review are similar enough in terms of interventions, designs and outcomes, it may be possible to conduct a meta-analysis to combine the results (Bettany-Saltikov, 2012).
The Cochrane Handbook for Systematic Review of Interventions (2011) provides reasons for researchers to consider taking the meta-analysis approach to a quantitative systematic review (Section 9.1.3 Why perform a meta-analysis in a review?):
To increase power. Power is the chance of detecting a real effect as statistically significant if it exists. Many individual studies are too small to detect small effects, but when several are combined there is a higher chance of detecting an effect.
To improve precision. The estimation of an intervention effect can be improved when it is based on more information.
To answer questions not posed by the individual studies. Primary studies often involve a specific type of patient and explicitly defined interventions. A selection of studies in which these characteristics differ can allow investigation of the consistency of effect and, if relevant, allow reasons for differences in effect estimates to be investigated.
To settle controversies arising from apparently conflicting studies or to generate new hypotheses. Statistical analysis of findings allows the degree of conflict to be formally assessed, and reasons for different results to be explored and quantified.
Types of Systematic Reviews
The type(s) of studies included in a systematic review depends on the review question. While systematic reviews of healthcare interventions generally include quantitative studies in the form of randomized controlled trials (RCTs), other types of reviews will find it appropriate to synthesize other types of quantitative and/or qualitative studies. For example, the Cochrane Library describes five types of Cochrane Reviews:
- Intervention reviews assess the benefits and harms of interventions used in healthcare and health policy.
- Diagnostic test accuracy reviews assess how well a diagnostic test performs in diagnosing and detecting a particular disease.
- Methodology reviews address issues relevant to how systematic reviews and clinical trials are conducted and reported.
- Qualitative reviews synthesize qualitative evidence to address questions on aspects other than effectiveness.
- Prognosis reviews address the probable course or future outcome(s) of people with a health problem.
Timeline considerations for systematic reviews
- Estimating how long it will take to complete a systematic review will depend on the review team's availability to work on the review. Systematic review team members may have competing priorities throughout the duration of the systematic review that can delay the process.
- The topic of the review will also impact the amount of time it will take to conduct the review. For example, a systematic review of a relatively new drug therapy may require less time to complete if the search yields a low number of results to screen and if the review itself only includes a small number of studies.
- The amount of time needed to complete this component of the review will depend on how straight-forward or complex the topic is, the number and type of resources that will be searched (refer to Resources to Search), and the searcher's level of expertise. Working with an experienced librarian is highly recommended.
- One research study found that the median time spent searching was just under 8 hours, including 1.5 hours spent searching the grey literature, while the average time spent searching was approximately 24 hours, of which 6.5 hours were spent searching the grey literature (Saleh et al., 2014). The average number of resources searched was 9, including grey literature resources. Grant funding influenced the amount of time spent searching and institution type impacted the number of resources searched.
- Greenhalgh and Peacock (2005) found that: "Electronic searching, including developing and refining search strategies and adapting these to different databases, took about two weeks of specialist librarian time..." (p1065).
How can my librarian help?
Librarians are available throughout the systematic review process.
"Work with a librarian or other information specialist trained in performing systematic reviews to plan the search strategy." Standard 3.1.1 of Finding What Works in Health Care: Standards for Systematic Reviews (Morton, 2011).
"Expert searchers are an important part of the systematic review team, crucial throughout the review process—from the development of the proposal and research question to publication" (McGowan & Sampson, 2005, p. 74).
Librarians can be involved in a consultative role or as part of the systematic review team to assist with the following:
- Converting a clinical question into PICO format or refining the research question
- Conducting a preliminary search to determine if a systematic review on the topic
- Suggesting appropriate resources to search (databases, trial registries etc.)
- Suggesting appropriate search terms (keywords and subject headings)
- Suggesting study methodology filters
- Developing and executing multiple database-specific search strategies
- Setting up search alerts for new publications
- Help tracking search strategies
- Training and assistance with bibliographic management software
- Describing the search process in the methods section of your monograph
Cochrane Library. (n.d.). About Cochrane reviews. Retrieved May 11, 2015, from http://www.cochranelibrary.com/about/about-cochrane-systematic-reviews.html
Crombie, I. K. & Davies, H. T. (2009). What is meta-analysis? (2nd ed.). Hayward Group Ltd.
Hemingway, P. & Brereton, N. (2009). What is a systematic review? (2nd ed.). Hayward Group Ltd.
Higgins, J. P. (Ed.). (2011). Cochrane handbook for systematic reviews of interventions (Version 5.1.0). Chichester: Wiley-Blackwell.
McGowan, J., & Sampson, M. (2005). Systematic reviews need systematic searchers. Journal of the Medical Library Association, 93(1), 74.
Morton, S., Levit, L., Berg, A., & Eden, J. (Eds.). (2011). Finding what works in health care: Standards for systematic reviews. National Academies Press.
Petticrew, M. (2001). Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ: British Medical Journal, 322(7278), 98.
Saleh, A. A., Ratajeski, M. A., & Bertolet, M. (2014). Grey literature searching for health sciences systematic reviews: A prospective study of time spent and resources utilized. Evidence Based Library and Information Practice, 9(3), 28-50.
Did you know?
There are 3 main types of systematic reviews (SRs):
•Quantitative SRs include only quantitative research studies.
•Qualitative SRs include only qualitative research studies.
•Mixed methods SRs include both quantiative and qualitative studies.
The What is...? series published
by HayWard Group Ltd. communicates key issues and concepts in a concise format: