Our guide to carrying out systematic reviews.
What is a Systematic Review?
A systematic review is a type of evidence review, which attempts to identify and bring together, or synthesise, all evidence associated with a specific question. Systematic reviews can be quantitative (with or without meta-analysis), qualitative (sometimes called qualitative evidence syntheses or meta-ethnographies), and (often using mixed methods) in other forms, such as scoping reviews and realist syntheses. Systematic reviews use clearly defined methodology in both the search strategy, the selection of studies, appraisal of evidence and data synthesis. This rigorous approach minimises bias in the search results, facilitates evidence-based decision-making, and enables the review to be transparent, reproduced and updated.
Systematic reviews are of particular importance in medical and other healthcare research, where they are frequently carried out in order to establish the effectiveness of particular treatment interventions, but they are also used in other disciplines.
A systematic review is much more than a literature review carried out systematically.
The table below highlights the similarities and differences between a systematic review and a literature review.
|Systematic review||Literature review|
|Question||Focused on a single question||Not necessarily focused on a single question but may describe an overview|
|Protocol||A peer review protocol or plan is included||No protocol is included|
|Background||Both provide summaries of the available literature on a topic|
|Objectives||Clear objectives are identified||Objectives may or may not be identified|
|Inclusion and exclusion criteria||Criteria stated before review is conducted||Criteria not specified|
|Search strategy||Comprehensive search conducted in a systematic way||Strategy not explicitly stated|
|Process of selecting articles||Usually clear and explicit||Not described in a literature review|
|Process of evaluating articles||Comprehensive evaluation of study quality||Evaluation of study quality may or may not be included|
|Results and data synthesis||Clear summaries based on high quality evidence||Summary based on studies where the quality of articles may not be specified. May also be influenced by the reviewer's theories, needs and beliefs|
|Discussion||Written by an expert or group of experts with a detailed and well-grounded knowledge of the issues|
|Search strategy||Detailed search strategy is included|
|Sources of literature||List of databases, websites and other sources of included studies are listed. Both published and unpublished literature are considered||Not usually stated and non-exhaustive, usually well-known articles. Prone to publication bias.|
|Process of evaluating articles/Critical appraisal||Rigorous appraisal of study quality||Variable evaluation of study quality or method|
Used with permission of Curtin University.
In summary, systematic reviews:
- Answer a specific question
- Have an explicit, reproducible methodology
- Have predefined eligibility criteria for studies
- Attempt to identify all studies that would meet the eligibility criteria
- Are carried out by at least two people to ensure validity of results, eg by removing bias
- Include an assessment of the validity of the findings of the included studies
- Result in a systematic presentation, and synthesis, of the characteristics and findings of the included studies.
Steps to carrying out a systematic review
- Spend time identifying and understanding the research question.
- Check Prospero to see if anyone else has registered a similar review.
- Develop a protocol that states what the review will include and exclude. Example protocol.
- Turn the question into a search strategy using free-text keywords and phrases. Use a thesaurus and database subject headings to identify synonyms. If necessary, contact Library Services for help.
- Consider using a search strategy worksheet to manage the process, and aid the transparency and replicability of the systematic review.
Example search strategy
|Concept 1||Concept 2||Concept 3||Concept 4|
* MeSH = Medical Subject Headings, the controlled vocabulary used in the PubMed database.
For a good example of a search strategy, have a look at the one used in the protocol for this systematic review: Quality of family relationships and outcomes of dementia: a SR (BMJ Open).
- For quantitative research, consider using PICO to identify search concepts. PICO is used to answer clinical and healthcare questions that look at the effectiveness of interventions, eg "is drug x more effective than drug y?"
P Person/Population I Intervention C Comparison O Outcome
You do not have to use all four elements. Quite often only P and I are used. Agree with team which criteria are needed.
- For qualitative reviews, consider using SPIDER:
Sample The group of people being looked at, because qualitative research is not easy to generalise, sample is preferred over patient Phenomenon of Interest Looks at the reasons for behaviour and decisions, rather than an intervention Design The form of research used, such as interview or survey Evaluation The outcome measures Research type Qualitative, quantitative and/or mixed methods
- Define inclusion and exclusion criteria.
- Project team agrees final search strategy.
- Carry out scoping search first to identify key resources using two or three databases.
- If necessary, modify search to ensure key articles are found.
Running the final search
- Once happy with the search strategy, select the most relevant databases and run the search. If necessary, contact Library Services for help to identify relevant databases.
- Run the search on all appropriate databases, including sources of grey literature, eg Open Grey, NIHR, or clinicaltrials.gov. Adapt the strategy as appropriate for each database.
- Run a citation search using Scopus.
- Identify journals not included in databases for manual searching.
- Check references at the end of located articles for other relevant material.
- Large numbers of results tend to be found, so use reference management software to help manage and screen these results.
- Once all the literature has been found, the abstracts then need to be screened by reviewers to decide whether the full text is required. Fewer than 1% of results usually make it to final review once exclusion/inclusion criteria are applied.
- Include a PRISMA statement in your review clearly explaining your methodology.
In these pages we focus on the first steps in carrying out your review - systematically searching for the literature. For further information the following are useful links:
- The Cochrane Handbook
- The Centre for Research and Dissemination’s guidance for carrying out reviews in healthcare.
- Doing a Systematic Review: a Student's Guide.
There is a range of literature available from UWE Bristol Library to aid your understanding of systematic reviews.
Supporting systematic literature reviews: What UWE Bristol Library Services can do
UWE Bristol Library Service is pleased to support systematic reviews as follows:
- Standard offer - We can meet with you individually to discuss your search strategy.
- Enhanced offer - If you have funding and are able to pay for more extensive support from your grant, we can meet with you to discuss timescales, develop a search strategy in conjunction with the review team, run the agreed searches and deposit results into a reference management package for you to access. Information on potential costs, charged at an hourly rate, can be found on the staff intranet.