Our guide to carrying out systematic reviews.
What is a Systematic Review?
A systematic review is an attempt to identify and bring together, or synthesise, all literature associated with a specific question. Systematic reviews use clearly defined methodology in both the search strategy and the selection of studies. This rigorous approach minimises bias in the search results, facilitates evidence-based decision-making, and enables the review to be transparent, reproduced and updated.
Systematic reviews are of particular importance in medical and other healthcare research, where they are frequently carried out in order to establish the effectiveness of particular treatment interventions, but they are also used in other disciplines.
A systematic review is much more than a literature review carried out systematically.
The table below highlights the similarities and differences between a systematic review and a literature review.
|Systematic review||Literature review|
|Question||Focused on a single question||Not necessarily focused on a single question but maybe describe an overview|
|Protocol||A peer review protocol or plan is included||No protocol is included|
|Background||Both provide summaries of the available literature on a topic|
|Objectives||Clear objectives are identified||Objectives may or may not be identified|
|Inclusion and exclusion criteria||Criteria stated before review is conducted||Criteria not specified|
|Search strategy||Comprehensive search conducted in a systematic way||Strategy not explicitly stated|
|Process of selecting articles||Usually clear and explicit||Not described in a literature review|
|Process of evaluating articles||Comprehensive evaluation of study quality||Evaluation of study quality may or may not be included|
|Results and data synthesis||Clear summaries based on high quality evidence||Summary based on studies where the quality of articles may not be specified. May also be influenced by the reviewer's theories, needs and beliefs|
|Discussion||Written by an expert or group of experts with a detailed and well-grounded knowledge of the issues|
Used with permission of Curtin University.
In summary, systematic reviews:
- Answer a specific question
- Have an explicit, reproducible methodology
- Have predefined eligibility criteria for studies
- Attempt to identify all studies that would meet the eligibility criteria
- Are carried out by at least two people to ensure validity of results, eg by removing bias
- Include an assessment of the validity of the findings of the included studies
- Result in a systematic presentation, and synthesis, of the characteristics and findings of the included studies
More information can be found online at The Cochrane Handbook section 2.2.1.
Steps to carrying out a systematic review
- Spend time identifying and understanding the research question.
- Check Prospero to see if anyone else has registered a similar review.
- Develop a protocol that states what the review will include and exclude.
- Turn the question into a search strategy using free-text keywords and phrases. Use a thesaurus and database subject headings to identify synonyms. If necessary, contact Library Services for help.
- Consider using a search strategy worksheet to manage the process, and aid the transparency and replicability of the systematic review.
Example search strategy
|Concept 1||Concept 2||Concept 3||Concept 4|
* MeSH = Medical Subject Headings, the controlled vocabulary used in the PubMed database.
For a good example of a search strategy, have a look at the one used in the protocol for this systematic review: Quality of family relationships and outcomes of dementia: a SR (BMJ Open).
- For health-related research, consider using
PICO to identify search concepts. PICO is used to answer
clinical and healthcare questions that look at the effectiveness of
interventions, eg "is drug x more effective than drug y?"
P Person/Population I Intervention C Comparison O Outcome
You do not have to use all four elements. Quite often only P and I are used. Agree with team which criteria are needed.
- Define inclusion and exclusion criteria.
- Project team agrees final search strategy.
- Carry out scoping search first to identify key resources using two or three databases.
- If necessary, modify search to ensure key articles are found.
Running the final search
- Once happy with the search strategy, select the most relevant databases and run the search. If necessary, contact Library Services for help to identify relevant databases.
- Run the search on all appropriate databases, including sources of grey literature, eg Open Grey, NIHR, or clinicaltrials.gov. Adapt the strategy as appropriate for each database.
- Run a citation search using Scopus.
- Identify journals not included in databases for manual searching.
- Check references at the end of located articles for other relevant material.
- Large numbers of results tend to be found, so use reference management software to help manage and screen these results.
- Once all the literature has been found, the abstracts then need to be screened by reviewers to decide whether the full text is required. Fewer than 1% of results usually make it to final review once exclusion/inclusion criteria are applied.
- Include a PRISMA statement in your review clearly explaining your methodology.
There is a range of literature available from UWE Bristol Library to aid your understanding of systematic reviews.
Supporting systematic literature reviews: What UWE Bristol Library Services can do
UWE Bristol Library Service is pleased to support systematic reviews as follows:
- Standard offer - We can meet with you individually to discuss your search strategy.
- Enhanced offer - If you have funding and are able to pay for more extensive support from your grant, we can meet with you to discuss timescales, develop a search strategy in conjunction with the review team, run the agreed searches and deposit results into a reference management package for you to access.