what is systematic review of research

Systematic Reviews

• What is a Systematic Review?

A systematic review is an evidence synthesis that uses explicit, reproducible methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Key Characteristics of a Systematic Review:

Generally, systematic reviews must have:

• a clearly stated set of objectives with pre-defined eligibility criteria for studies
• an explicit, reproducible methodology
• a systematic search that attempts to identify all studies that would meet the eligibility criteria
• an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias
• a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

A meta-analysis is a systematic review that uses quantitative methods to synthesize and summarize the pooled data from included studies.

Additional Information

• How-to Books
• Beyond Health Sciences

• Cochrane Handbook For Systematic Reviews of Interventions Provides guidance to authors for the preparation of Cochrane Intervention reviews. Chapter 6 covers searching for reviews.
• Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care From The University of York Centre for Reviews and Dissemination: Provides practical guidance for undertaking evidence synthesis based on a thorough understanding of systematic review methodology. It presents the core principles of systematic reviewing, and in complementary chapters, highlights issues that are specific to reviews of clinical tests, public health interventions, adverse effects, and economic evaluations.
• Cornell, Sytematic Reviews and Evidence Synthesis Beyond the Health Sciences Video series geared for librarians but very informative about searching outside medicine.
• << Previous: Getting Started
• Next: Levels of Evidence >>
• Getting Started
• Levels of Evidence
• Locating Systematic Reviews
• Searching Systematically
• Developing Answerable Questions
• Identifying Synonyms & Related Terms
• Using Truncation and Wildcards
• Identifying Search Limits/Exclusion Criteria
• Keyword vs. Subject Searching
• Where to Search
• Search Filters
• Sensitivity vs. Precision
• Core Databases
• Other Databases
• Clinical Trial Registries
• Conference Presentations
• Databases Indexing Grey Literature
• Web Searching
• Handsearching
• Citation Indexes
• Documenting the Search Process
• Managing your Review

Research Support

• Last Updated: Apr 8, 2024 3:33 PM
• URL: https://guides.library.ucdavis.edu/systematic-reviews

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Systematic Review | Definition, Examples & Guide

Systematic Review | Definition, Examples & Guide

Published on 15 June 2022 by Shaun Turney . Revised on 17 October 2022.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question ‘What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?’

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs meta-analysis, systematic review vs literature review, systematic review vs scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce research bias . The methods are repeatable , and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesise the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesising all available evidence and evaluating the quality of the evidence. Synthesising means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism, run a free check.

Systematic reviews often quantitatively synthesise the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesise results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarise and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimise bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimise research b ias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinised by others.
• They’re thorough : they summarise all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fourth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomised control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective(s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesise the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Grey literature: Grey literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of grey literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of grey literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Grey literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarise what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgement of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomised into the control and treatment groups.

Step 6: Synthesise the data

Synthesising the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesising the data:

• Narrative ( qualitative ): Summarise the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarise and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analysed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Turney, S. (2022, October 17). Systematic Review | Definition, Examples & Guide. Scribbr. Retrieved 29 April 2024, from https://www.scribbr.co.uk/research-methods/systematic-reviews/

Is this article helpful?

Shaun Turney

Other students also liked, what is a literature review | guide, template, & examples, exploratory research | definition, guide, & examples, what is peer review | types & examples.

Jump to navigation

• Bahasa Malaysia

What are systematic reviews?

Watch this video from   Cochrane Consumers and Communication to learn what systematic reviews are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone. 

Cochrane evidence, including our systematic reviews, provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence and what Cochrane does. 

• Search our Plain Language Summaries of health evidence
• Learn more about Cochrane and our work

1.2.2  What is a systematic review?

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question.  It  uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993) . The key characteristics of a systematic review are:

a clearly stated set of objectives with pre-defined eligibility criteria for studies;

an explicit, reproducible methodology;

a systematic search that attempts to identify all studies that would meet the eligibility criteria;

an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and

a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarize the results of independent studies (Glass 1976). By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review (see Chapter 9, Section 9.1.3 ). They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 14, Issue 3
• What is a systematic review?
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Jane Clarke
• Correspondence to Jane Clarke 4 Prime Road, Grey Lynn, Auckland, New Zealand; janeclarkehome{at}gmail.com

https://doi.org/10.1136/ebn.2011.0049

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

A high-quality systematic review is described as the most reliable source of evidence to guide clinical practice. The purpose of a systematic review is to deliver a meticulous summary of all the available primary research in response to a research question. A systematic review uses all the existing research and is sometime called ‘secondary research’ (research on research). They are often required by research funders to establish the state of existing knowledge and are frequently used in guideline development. Systematic review findings are often used within the healthcare setting but may be applied elsewhere. For example, the Campbell Collaboration advocates the application of systematic reviews for policy-making in education, justice and social work.

Systematic reviews can be conducted on all types of primary research. Many are reviews of randomised trials (addressing questions of effectiveness), cross-sectional studies (addressing questions about prevalence or diagnostic accuracy, for example) or cohort studies (addressing questions about prognosis). When qualitative research is reviewed systematically, it may be described as a systematic review, but more often other terms such as meta-synthesis are used.

Systematic review methodology is explicit and precise and aims to minimise bias, thus enhancing the reliability of the conclusions drawn. 1 , 2 The features of a systematic review include:

■ clear aims with predetermined eligibility and relevance criteria for studies;

■ transparent, reproducible methods;

■ rigorous search designed to locate all eligible studies;

■ an assessment of the validity of the findings of the included studies and

■ a systematic presentation, and synthesis, of the included studies. 3

The first step in a systematic review is a meticulous search of all sources of evidence for relevant studies. The databases and citation indexes searched are listed in the methodology section of the review. Next, using predetermined reproducible criteria to screen for eligibility and relevance assessment of titles and the abstracts is completed. Each study is then assessed in terms of methodological quality.

Finally, the evidence is synthesised. This process may or may not include a meta-analysis. A meta-analysis is a statistical summary of the findings of independent studies. 4 Meta-analyses can potentially present more precise estimates of the effects of interventions than those derived from the individual studies alone. These strategies are used to limit bias and random error which may arise during this process. Without these safeguards, then, reviews can mislead, such that we gain an unreliable summary of the available knowledge.

The Cochrane Collaboration is a leader in the production of systematic reviews. Cochrane reviews are published on a monthly basis in the Cochrane Database of Systematic Reviews in The Cochrane Library (see: http://www.thecochranelibrary.com ).

• Antman EM ,
• Kupelnick B ,
• Higgins JPT ,

Competing interests None.

Read the full text or download the PDF:

Systematic Review

• Library Help
• What is a Systematic Review (SR)?
• Steps of a Systematic Review
• Framing a Research Question
• Developing a Search Strategy
• Searching the Literature
• Managing the Process
• Meta-analysis
• Publishing your Systematic Review

Introduction to Systematic Review

• Introduction
• Types of literature reviews
• Other Libguides
• Systematic review as part of a dissertation
• Tutorials & Guidelines & Examples from non-Medical Disciplines

Depending on your learning style, please explore the resources in various formats on the tabs above.

For additional tutorials, visit the SR Workshop Videos  from UNC at Chapel Hill outlining each stage of the systematic review process.

Know the difference! Systematic review vs. literature review

Types of literature reviews along with associated methodologies

JBI Manual for Evidence Synthesis .  Find definitions and methodological guidance.

- Systematic Reviews - Chapters 1-7

- Mixed Methods Systematic Reviews -  Chapter 8

- Diagnostic Test Accuracy Systematic Reviews -  Chapter 9

- Umbrella Reviews -  Chapter 10

- Scoping Reviews -  Chapter 11

- Systematic Reviews of Measurement Properties -  Chapter 12

Systematic reviews vs scoping reviews - 

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal , 26 (2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1 (28). htt p s://doi.org/ 10.1186/2046-4053-1-28

Munn, Z., Peters, M., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018).  Systematic review or  scoping review ?  Guidance for authors when choosing between a systematic or scoping review approach.  BMC medical research methodology, 18 (1), 143. https://doi.org/10.1186/s12874-018-0611-x. Also, check out the  Libguide from Weill Cornell Medicine  for the  differences between a systematic review and a scoping review  and when to embark on either one of them.

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: Exploring review types and associated information retrieval requirements . Health Information & Libraries Journal , 36 (3), 202–222. https://doi.org/10.1111/hir.12276

Temple University. Review Types . - This guide provides useful descriptions of some of the types of reviews listed in the above article.

UMD Health Sciences and Human Services Library.  Review Types . - Guide describing Literature Reviews, Scoping Reviews, and Rapid Reviews.

Whittemore, R., Chao, A., Jang, M., Minges, K. E., & Park, C. (2014). Methods for knowledge synthesis: An overview. Heart & Lung: The Journal of Acute and Critical Care, 43 (5), 453–461. https://doi.org/10.1016/j.hrtlng.2014.05.014

Differences between a systematic review and other types of reviews

Armstrong, R., Hall, B. J., Doyle, J., & Waters, E. (2011). ‘ Scoping the scope ’ of a cochrane review. Journal of Public Health , 33 (1), 147–150. https://doi.org/10.1093/pubmed/fdr015

Kowalczyk, N., & Truluck, C. (2013). Literature reviews and systematic reviews: What is the difference? Radiologic Technology , 85 (2), 219–222.

White, H., Albers, B., Gaarder, M., Kornør, H., Littell, J., Marshall, Z., Matthew, C., Pigott, T., Snilstveit, B., Waddington, H., & Welch, V. (2020). Guidance for producing a Campbell evidence and gap map . Campbell Systematic Reviews, 16 (4), e1125. https://doi.org/10.1002/cl2.1125. Check also this comparison between evidence and gaps maps and systematic reviews.

Rapid Reviews Tutorials

Rapid Review Guidebook  by the National Collaborating Centre of Methods and Tools (NCCMT)

Hamel, C., Michaud, A., Thuku, M., Skidmore, B., Stevens, A., Nussbaumer-Streit, B., & Garritty, C. (2021). Defining Rapid Reviews: a systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews.  Journal of clinical epidemiology ,  129 , 74–85. https://doi.org/10.1016/j.jclinepi.2020.09.041

• Müller, C., Lautenschläger, S., Meyer, G., & Stephan, A. (2017). Interventions to support people with dementia and their caregivers during the transition from home care to nursing home care: A systematic review . International Journal of Nursing Studies, 71 , 139–152. https://doi.org/10.1016/j.ijnurstu.2017.03.013
• Bhui, K. S., Aslam, R. W., Palinski, A., McCabe, R., Johnson, M. R. D., Weich, S., … Szczepura, A. (2015). Interventions to improve therapeutic communications between Black and minority ethnic patients and professionals in psychiatric services: Systematic review . The British Journal of Psychiatry, 207 (2), 95–103. https://doi.org/10.1192/bjp.bp.114.158899
• Rosen, L. J., Noach, M. B., Winickoff, J. P., & Hovell, M. F. (2012). Parental smoking cessation to protect young children: A systematic review and meta-analysis . Pediatrics, 129 (1), 141–152. https://doi.org/10.1542/peds.2010-3209

Scoping Review

• Hyshka, E., Karekezi, K., Tan, B., Slater, L. G., Jahrig, J., & Wild, T. C. (2017). The role of consumer perspectives in estimating population need for substance use services: A scoping review . BMC Health Services Research, 171-14.  https://doi.org/10.1186/s12913-017-2153-z
• Olson, K., Hewit, J., Slater, L.G., Chambers, T., Hicks, D., Farmer, A., & ... Kolb, B. (2016). Assessing cognitive function in adults during or following chemotherapy: A scoping review . Supportive Care In Cancer, 24 (7), 3223-3234. https://doi.org/10.1007/s00520-016-3215-1
• Pham, M. T., Rajić, A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., & McEwen, S. A. (2014). A scoping review of scoping reviews: Advancing the approach and enhancing the consistency . Research Synthesis Methods, 5 (4), 371–385. https://doi.org/10.1002/jrsm.1123
• Scoping Review Tutorial from UNC at Chapel Hill

Qualitative Systematic Review/Meta-Synthesis

• Lee, H., Tamminen, K. A., Clark, A. M., Slater, L., Spence, J. C., & Holt, N. L. (2015). A meta-study of qualitative research examining determinants of children's independent active free play . International Journal Of Behavioral Nutrition & Physical Activity, 12 (5), 121-12. https://doi.org/10.1186/s12966-015-0165-9

Videos on systematic reviews

Systematic Reviews: What are they? Are they right for my research? - 47 min. video recording with a closed caption option.

More training videos  on systematic reviews:   

Books on Systematic Reviews

Books on Meta-analysis

• University of Toronto Libraries  - very detailed with good tips on the sensitivity and specificity of searches.
• Monash University  - includes an interactive case study tutorial. 
• Dalhousie University Libraries - a comprehensive How-To Guide on conducting a systematic review.

Guidelines for a systematic review as part of the dissertation

• Guidelines for Systematic Reviews in the Context of Doctoral Education Background  by University of Victoria (PDF)
• Can I conduct a Systematic Review as my Master’s dissertation or PhD thesis? Yes, It Depends!  by Farhad (blog)
• What is a Systematic Review Dissertation Like? by the University of Edinburgh (50 min video) 

Further readings on experiences of PhD students and doctoral programs with systematic reviews

Puljak, L., & Sapunar, D. (2017). Acceptance of a systematic review as a thesis: Survey of biomedical doctoral programs in Europe . Systematic Reviews , 6 (1), 253. https://doi.org/10.1186/s13643-017-0653-x

Perry, A., & Hammond, N. (2002). Systematic reviews: The experiences of a PhD Student . Psychology Learning & Teaching , 2 (1), 32–35. https://doi.org/10.2304/plat.2002.2.1.32

Daigneault, P.-M., Jacob, S., & Ouimet, M. (2014). Using systematic review methods within a Ph.D. dissertation in political science: Challenges and lessons learned from practice . International Journal of Social Research Methodology , 17 (3), 267–283. https://doi.org/10.1080/13645579.2012.730704

UMD Doctor of Philosophy Degree Policies

Before you embark on a systematic review research project, check the UMD PhD Policies to make sure you are on the right path. Systematic reviews require a team of at least two reviewers and an information specialist or a librarian. Discuss with your advisor the authorship roles of the involved team members. Keep in mind that the  UMD Doctor of Philosophy Degree Policies (scroll down to the section, Inclusion of one's own previously published materials in a dissertation ) outline such cases, specifically the following: 

" It is recognized that a graduate student may co-author work with faculty members and colleagues that should be included in a dissertation . In such an event, a letter should be sent to the Dean of the Graduate School certifying that the student's examining committee has determined that the student made a substantial contribution to that work. This letter should also note that the inclusion of the work has the approval of the dissertation advisor and the program chair or Graduate Director. The letter should be included with the dissertation at the time of submission.  The format of such inclusions must conform to the standard dissertation format. A foreword to the dissertation, as approved by the Dissertation Committee, must state that the student made substantial contributions to the relevant aspects of the jointly authored work included in the dissertation."

• Cochrane Handbook for Systematic Reviews of Interventions - See Part 2: General methods for Cochrane reviews
• Systematic Searches - Yale library video tutorial series 
• Using PubMed's Clinical Queries to Find Systematic Reviews  - From the U.S. National Library of Medicine
• Systematic reviews and meta-analyses: A step-by-step guide - From the University of Edinsburgh, Centre for Cognitive Ageing and Cognitive Epidemiology

Bioinformatics

• Mariano, D. C., Leite, C., Santos, L. H., Rocha, R. E., & de Melo-Minardi, R. C. (2017). A guide to performing systematic literature reviews in bioinformatics .  arXiv preprint arXiv:1707.05813.

Environmental Sciences

Collaboration for Environmental Evidence. 2018.  Guidelines and Standards for Evidence synthesis in Environmental Management. Version 5.0 (AS Pullin, GK Frampton, B Livoreil & G Petrokofsky, Eds) www.environmentalevidence.org/information-for-authors .

Pullin, A. S., & Stewart, G. B. (2006). Guidelines for systematic review in conservation and environmental management. Conservation Biology, 20 (6), 1647–1656. https://doi.org/10.1111/j.1523-1739.2006.00485.x

Engineering Education

• Borrego, M., Foster, M. J., & Froyd, J. E. (2014). Systematic literature reviews in engineering education and other developing interdisciplinary fields. Journal of Engineering Education, 103 (1), 45–76. https://doi.org/10.1002/jee.20038

Public Health

• Hannes, K., & Claes, L. (2007). Learn to read and write systematic reviews: The Belgian Campbell Group . Research on Social Work Practice, 17 (6), 748–753. https://doi.org/10.1177/1049731507303106
• McLeroy, K. R., Northridge, M. E., Balcazar, H., Greenberg, M. R., & Landers, S. J. (2012). Reporting guidelines and the American Journal of Public Health’s adoption of preferred reporting items for systematic reviews and meta-analyses . American Journal of Public Health, 102 (5), 780–784. https://doi.org/10.2105/AJPH.2011.300630
• Pollock, A., & Berge, E. (2018). How to do a systematic review.   International Journal of Stroke, 13 (2), 138–156. https://doi.org/10.1177/1747493017743796
• Institute of Medicine. (2011). Finding what works in health care: Standards for systematic reviews . https://doi.org/10.17226/13059
• Wanden-Berghe, C., & Sanz-Valero, J. (2012). Systematic reviews in nutrition: Standardized methodology . The British Journal of Nutrition, 107 Suppl 2, S3-7. https://doi.org/10.1017/S0007114512001432

Social Sciences

• Bronson, D., & Davis, T. (2012).  Finding and evaluating evidence: Systematic reviews and evidence-based practice (Pocket guides to social work research methods). Oxford: Oxford University Press.
• Petticrew, M., & Roberts, H. (2006).  Systematic reviews in the social sciences: A practical guide . Malden, MA: Blackwell Pub.
• Cornell University Library Guide -  Systematic literature reviews in engineering: Example: Software Engineering
• Biolchini, J., Mian, P. G., Natali, A. C. C., & Travassos, G. H. (2005). Systematic review in software engineering .  System Engineering and Computer Science Department COPPE/UFRJ, Technical Report ES, 679 (05), 45.
• Biolchini, J. C., Mian, P. G., Natali, A. C. C., Conte, T. U., & Travassos, G. H. (2007). Scientific research ontology to support systematic review in software engineering . Advanced Engineering Informatics, 21 (2), 133–151.
• Kitchenham, B. (2007). Guidelines for performing systematic literature reviews in software engineering . [Technical Report]. Keele, UK, Keele University, 33(2004), 1-26.
• Weidt, F., & Silva, R. (2016). Systematic literature review in computer science: A practical guide .  Relatórios Técnicos do DCC/UFJF ,  1 .
• Academic Phrasebank - Get some inspiration and find some terms and phrases for writing your research paper
• Oxford English Dictionary  - Use to locate word variants and proper spelling
• << Previous: Library Help
• Next: Steps of a Systematic Review >>
• Last Updated: Apr 19, 2024 12:47 PM
• URL: https://lib.guides.umd.edu/SR

Library Services

UCL LIBRARY SERVICES

• Guides and databases
• Library skills
• Systematic reviews

What are systematic reviews?

• Types of systematic reviews
• Formulating a research question
• Identifying studies
• Searching databases
• Describing and appraising studies
• Synthesis and systematic maps
• Software for systematic reviews
• Online training and support
• Live and face to face training
• Individual support
• Further help

Systematic reviews are a type of literature review of research which require equivalent standards of rigour as primary research. They have a clear, logical rationale that is reported to the reader of the review. They are used in research and policymaking to inform evidence-based decisions and practice. They differ from traditional literature reviews particularly in the following elements of conduct and reporting.

Systematic reviews: 

• use explicit and transparent methods
• are a piece of research following a standard set of stages
• are accountable, replicable and updateable
• involve users to ensure a review is relevant and useful.

For example, systematic reviews (like all research) should have a clear research question, and the perspective of the authors in their approach to addressing the question is described. There are clearly described methods on how each study in a review was identified, how that study was appraised for quality and relevance and how it is combined with other studies in order to address the review question. A systematic review usually involves more than one person in order to increase the objectivity and trustworthiness of the reviews methods and findings.

Research protocols for systematic reviews may be peer-reviewed and published or registered in a suitable repository to help avoid duplication of reviews and for comparisons to be made with the final review and the planned review.

• History of systematic reviews to inform policy (EPPI-Centre)
• Six reasons why it is important to be systematic (EPPI-Centre)
• Evidence Synthesis International (ESI): Position Statement Describes the issues, principles and goals in synthesising research evidence to inform policy, practice and decisions

On this page

Should all literature reviews be 'systematic reviews', different methods for systematic reviews, reporting standards for systematic reviews.

Literature reviews provide a more complete picture of research knowledge than is possible from individual pieces of research. This can be used to: clarify what is known from research, provide new perspectives, build theory, test theory, identify research gaps or inform research agendas.

A systematic review requires a considerable amount of time and resources, and is one type of literature review.

If the purpose of a review is to make justifiable evidence claims, then it should be systematic, as a systematic review uses rigorous explicit methods. The methods used can depend on the purpose of the review, and the time and resources available.

A 'non-systematic review' might use some of the same methods as systematic reviews, such as systematic approaches to identify studies or quality appraise the literature. There may be times when this approach can be useful. In a student dissertation, for example, there may not be the time to be fully systematic in a review of the literature if this is only one small part of the thesis. In other types of research, there may also be a need to obtain a quick and not necessarily thorough overview of a literature to inform some other work (including a systematic review). Another example, is where policymakers, or other people using research findings, want to make quick decisions and there is no systematic review available to help them. They have a choice of gaining a rapid overview of the research literature or not having any research evidence to help their decision-making. 

Just like any other piece of research, the methods used to undertake any literature review should be carefully planned to justify the conclusions made. 

Finding out about different types of systematic reviews and the methods used for systematic reviews, and reading both systematic and other types of review will help to understand some of the differences. 

Typically, a systematic review addresses a focussed, structured research question in order to inform understanding and decisions on an area. (see the  Formulating a research question  section for examples). 

Sometimes systematic reviews ask a broad research question, and one strategy to achieve this is the use of several focussed sub-questions each addressed by sub-components of the review.  

Another strategy is to develop a map to describe the type of research that has been undertaken in relation to a research question. Some maps even describe over 2,000 papers, while others are much smaller. One purpose of a map is to help choose a sub-set of studies to explore more fully in a synthesis. There are also other purposes of maps: see the box on  systematic evidence maps  for further information. 

Reporting standards specify minimum elements that need to go into the reporting of a review. The reporting standards refer mainly to methodological issues but they are not as detailed or specific as critical appraisal for the methodological standards of conduct of a review.

A number of organisations have developed specific guidelines and standards for both the conducting and reporting on systematic reviews in different topic areas.  

• PRISMA PRISMA is a reporting standard and is an acronym for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. The Key Documents section of the PRISMA website links to a checklist, flow diagram and explanatory notes. PRISMA is less useful for certain types of reviews, including those that are iterative.
• eMERGe eMERGe is a reporting standard that has been developed for meta-ethnographies, a qualitative synthesis method.
• ROSES: RepOrting standards for Systematic Evidence Syntheses Reporting standards, including forms and flow diagram, designed specifically for systematic reviews and maps in the field of conservation and environmental management.

Useful books about systematic reviews

Systematic approaches to a successful literature review

An introduction to systematic reviews

Cochrane handbook for systematic reviews of interventions

Systematic reviews: crd's guidance for undertaking reviews in health care.

Finding what works in health care: Standards for systematic reviews

Systematic Reviews in the Social Sciences

Meta-analysis and research synthesis.

Research Synthesis and Meta-Analysis

Doing a Systematic Review

Literature reviews.

• What is a literature review?
• Why are literature reviews important?
• << Previous: Systematic reviews
• Next: Types of systematic reviews >>
• Last Updated: Apr 4, 2024 10:09 AM
• URL: https://library-guides.ucl.ac.uk/systematic-reviews

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Digital Health Device Collection
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives

Systematic Reviews

• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations

What is a Systematic Review?

• Types of Reviews
• Manuals and Reporting Guidelines
• Our Service
• 1. Assemble Your Team
• 2. Develop a Research Question
• 3. Write and Register a Protocol
• 4. Search the Evidence
• 5. Screen Results
• 6. Assess for Quality and Bias
• 7. Extract the Data
• 8. Write the Review
• Additional Resources
• Finding Full-Text Articles

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. The key characteristics of a systematic review are:

• a clearly defined question with inclusion and exclusion criteria;
• a rigorous and systematic search of the literature;
• two phases of screening (blinded, at least two independent screeners);
• data extraction and management;
• analysis and interpretation of results;
• risk of bias assessment of included studies;
• and report for publication.

Medical Center Library & Archives Presentations

The following presentation is a recording of the Getting Started with Systematic Reviews workshop (4/2022), offered by the Duke Medical Center Library & Archives. A NetID/pw is required to access the tutorial via Warpwire. 

• << Previous: Overview
• Next: Types of Reviews >>
• Last Updated: Apr 30, 2024 11:19 AM
• URL: https://guides.mclibrary.duke.edu/sysreview
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

Easy guide to conducting a systematic review

Affiliations.

• 1 Discipline of Child and Adolescent Health, University of Sydney, Sydney, New South Wales, Australia.
• 2 Department of Nephrology, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• 3 Education Department, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• PMID: 32364273
• DOI: 10.1111/jpc.14853

A systematic review is a type of study that synthesises research that has been conducted on a particular topic. Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology. To minimise bias, systematic reviews utilise a predefined search strategy to identify and appraise all available published literature on a specific topic. The meticulous nature of the systematic review research methodology differentiates a systematic review from a narrative review (literature review or authoritative review). This paper provides a brief step by step summary of how to conduct a systematic review, which may be of interest for clinicians and researchers.

Keywords: research; research design; systematic review.

© 2020 Paediatrics and Child Health Division (The Royal Australasian College of Physicians).

Publication types

• Systematic Review
• Research Design*

• Conclusions
• Article Information

Evidence reviews for the USPSTF use an analytic framework to visually display the key questions that the review will address in order to allow the USPSTF to evaluate the effectiveness and safety of a preventative service. The questions are depicted by linkages that relate interventions and outcomes. A dashed line indicates a health outcome that immediately follows an intermediate outcome. For additional details see the US Preventive Services Task Force Procedure Manual. 13

Reasons for exclusion: Design: Study did not use an included design. Outcomes: Study did not have relevant outcomes or had incomplete outcomes. Comparator: Study used an excluded comparator. Intervention: Study used an excluded intervention/screening approach. Population: Study was not conducted in an average-risk population. Timing: Study only reported first (prevalence) round screening follow-up. Publication type: Study was published in non–English-language or only available in an abstract. Quality: Study did not meet criteria for fair or good quality. Setting: Study was not conducted in a setting relevant to US practice. KQ indicates key question.

DBT indicates digital breast tomosynthesis; DM, digital mammography; and RR, relative risk.

a From random-effects restricted maximum likelihood model.

eMethods. Literature Search Strategies for Primary Literature

eTable 1. Inclusion and Exclusion Criteria

eTable 2. Quality Assessment Criteria

eTable 3. Included Studies and Their Ancillary Publications

eTable 4. Screen-Detected DCIS Diagnosed in Studies Comparing Digital Breast Tomosynthesis and Digital Mammography

eFigure 1. Pooled Analysis of Screen-Detected Invasive Cancers Diagnosed in Trials Comparing Digital Breast Tomosynthesis and Digital Mammography

eFigure 2. Pooled Analysis of Interval Cancers Diagnosed in Trials Comparing Digital Breast Tomosynthesis and Digital Mammography

eFigure 3. Cumulative Probability of False-Positive Biopsy in One NSRI Using BCSC Data Comparing Annual vs Biennial Screening with DBT or DM

eFigure 4. Cumulative Probability of False-Positive Recall in One NSRI Using BCSC Data Comparing Annual vs Biennial Screening with DBT or DM

eFigure 5. Cumulative Probability of False-Positive Recall or Biopsy in One NSRI Using BCSC Data Comparing Annual vs Biennial Screening with DBT or DM, among Women with Extremely Dense Breasts

• USPSTF Recommendation: Screening for Breast Cancer JAMA US Preventive Services Task Force April 30, 2024 This 2024 Recommendation Statement from the US Preventive Services Task Force recommends biennial screening mammography for women aged 40 to 74 years (B recommendation) and concludes that evidence is insufficient to assess the balance of benefits and harms of screening mammography in women 75 years or older (I statement) and of screening using ultrasonography or MRI in women with dense breasts on a negative mammogram (I statement). US Preventive Services Task Force; Wanda K. Nicholson, MD, MPH, MBA; Michael Silverstein, MD, MPH; John B. Wong, MD; Michael J. Barry, MD; David Chelmow, MD; Tumaini Rucker Coker, MD, MBA; Esa M. Davis, MD, MPH; Carlos Roberto Jaén, MD, PhD, MS; Marie Krousel-Wood, MD, MSPH; Sei Lee, MD, MAS; Li Li, MD, PhD, MPH; Carol M. Mangione, MD, MSPH; Goutham Rao, MD; John M. Ruiz, PhD; James J. Stevermer, MD, MSPH; Joel Tsevat, MD, MPH; Sandra Millon Underwood, PhD, RN; Sarah Wiehe, MD, MPH
• USPSTF Report: Collaborative Modeling to Compare Breast Cancer Screening Strategies JAMA US Preventive Services Task Force April 30, 2024 This modeling study uses Cancer Intervention and Surveillance Modeling Network models and national data on breast cancer incidence, mammography performance, treatment effects, and other-cause mortality in US women without previous cancer diagnoses to estimate outcomes of various mammography screening strategies. Amy Trentham-Dietz, PhD, MS; Christina Hunter Chapman, MD, MS; Jinani Jayasekera, PhD, MS; Kathryn P. Lowry, MD; Brandy M. Heckman-Stoddard, PhD, MPH; John M. Hampton, MS; Jennifer L. Caswell-Jin, MD; Ronald E. Gangnon, PhD; Ying Lu, PhD, MS; Hui Huang, MS; Sarah Stein, PhD; Liyang Sun, MS; Eugenio J. Gil Quessep, MS; Yuanliang Yang, MS; Yifan Lu, BASc; Juhee Song, PhD; Diego F. Muñoz, PhD; Yisheng Li, PhD, MS; Allison W. Kurian, MD, MSc; Karla Kerlikowske, MD; Ellen S. O’Meara, PhD; Brian L. Sprague, PhD; Anna N. A. Tosteson, ScD; Eric J. Feuer, PhD; Donald Berry, PhD; Sylvia K. Plevritis, PhD; Xuelin Huang, PhD; Harry J. de Koning, MD, PhD; Nicolien T. van Ravesteyn, PhD; Sandra J. Lee, ScD; Oguzhan Alagoz, PhD, MS; Clyde B. Schechter, MD, MA; Natasha K. Stout, PhD; Diana L. Miglioretti, PhD, ScM; Jeanne S. Mandelblatt, MD, MPH
• Toward More Equitable Breast Cancer Outcomes JAMA Editorial April 30, 2024 Joann G. Elmore, MD, MPH; Christoph I. Lee, MD, MS
• Screening for Breast Cancer JAMA JAMA Patient Page April 30, 2024 In this JAMA Patient Page, the US Preventive Services Task Force provides a guide to screening for breast cancer. US Preventive Services Task Force
• New Recommendations for Breast Cancer Screening—In Pursuit of Health Equity JAMA Network Open Editorial April 30, 2024 Lydia E. Pace, MD, MPH; Nancy L. Keating, MD, MPH
• USPSTF Breast Cancer Screening Guidelines Do Not Go Far Enough JAMA Oncology Editorial April 30, 2024 Wendie A. Berg, MD, PhD

See More About

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Others Also Liked

• Download PDF
• X Facebook More LinkedIn
• CME & MOC

Henderson JT , Webber EM , Weyrich MS , Miller M , Melnikow J. Screening for Breast Cancer : Evidence Report and Systematic Review for the US Preventive Services Task Force . JAMA. Published online April 30, 2024. doi:10.1001/jama.2023.25844

Manage citations:

© 2024

• Permissions

Screening for Breast Cancer : Evidence Report and Systematic Review for the US Preventive Services Task Force

• 1 Kaiser Permanente Evidence-based Practice Center, Center for Health Research, Portland, Oregon
• 2 University of California Davis Center for Healthcare Policy and Research, Sacramento
• Editorial Toward More Equitable Breast Cancer Outcomes Joann G. Elmore, MD, MPH; Christoph I. Lee, MD, MS JAMA
• Editorial New Recommendations for Breast Cancer Screening—In Pursuit of Health Equity Lydia E. Pace, MD, MPH; Nancy L. Keating, MD, MPH JAMA Network Open
• Editorial USPSTF Breast Cancer Screening Guidelines Do Not Go Far Enough Wendie A. Berg, MD, PhD JAMA Oncology
• US Preventive Services Task Force USPSTF Recommendation: Screening for Breast Cancer US Preventive Services Task Force; Wanda K. Nicholson, MD, MPH, MBA; Michael Silverstein, MD, MPH; John B. Wong, MD; Michael J. Barry, MD; David Chelmow, MD; Tumaini Rucker Coker, MD, MBA; Esa M. Davis, MD, MPH; Carlos Roberto Jaén, MD, PhD, MS; Marie Krousel-Wood, MD, MSPH; Sei Lee, MD, MAS; Li Li, MD, PhD, MPH; Carol M. Mangione, MD, MSPH; Goutham Rao, MD; John M. Ruiz, PhD; James J. Stevermer, MD, MSPH; Joel Tsevat, MD, MPH; Sandra Millon Underwood, PhD, RN; Sarah Wiehe, MD, MPH JAMA
• US Preventive Services Task Force USPSTF Report: Collaborative Modeling to Compare Breast Cancer Screening Strategies Amy Trentham-Dietz, PhD, MS; Christina Hunter Chapman, MD, MS; Jinani Jayasekera, PhD, MS; Kathryn P. Lowry, MD; Brandy M. Heckman-Stoddard, PhD, MPH; John M. Hampton, MS; Jennifer L. Caswell-Jin, MD; Ronald E. Gangnon, PhD; Ying Lu, PhD, MS; Hui Huang, MS; Sarah Stein, PhD; Liyang Sun, MS; Eugenio J. Gil Quessep, MS; Yuanliang Yang, MS; Yifan Lu, BASc; Juhee Song, PhD; Diego F. Muñoz, PhD; Yisheng Li, PhD, MS; Allison W. Kurian, MD, MSc; Karla Kerlikowske, MD; Ellen S. O’Meara, PhD; Brian L. Sprague, PhD; Anna N. A. Tosteson, ScD; Eric J. Feuer, PhD; Donald Berry, PhD; Sylvia K. Plevritis, PhD; Xuelin Huang, PhD; Harry J. de Koning, MD, PhD; Nicolien T. van Ravesteyn, PhD; Sandra J. Lee, ScD; Oguzhan Alagoz, PhD, MS; Clyde B. Schechter, MD, MA; Natasha K. Stout, PhD; Diana L. Miglioretti, PhD, ScM; Jeanne S. Mandelblatt, MD, MPH JAMA
• JAMA Patient Page Screening for Breast Cancer US Preventive Services Task Force JAMA

Importance   Breast cancer is a leading cause of cancer mortality for US women. Trials have established that screening mammography can reduce mortality risk, but optimal screening ages, intervals, and modalities for population screening guidelines remain unclear.

Objective   To review studies comparing different breast cancer screening strategies for the US Preventive Services Task Force.

Data Sources   MEDLINE, Cochrane Library through August 22, 2022; literature surveillance through March 2024.

Study Selection   English-language publications; randomized clinical trials and nonrandomized studies comparing screening strategies; expanded criteria for screening harms.

Data Extraction and Synthesis   Two reviewers independently assessed study eligibility and quality; data extracted from fair- and good-quality studies.

Main Outcomes and Measures   Mortality, morbidity, progression to advanced cancer, interval cancers, screening harms.

Results   Seven randomized clinical trials and 13 nonrandomized studies were included; 2 nonrandomized studies reported mortality outcomes. A nonrandomized trial emulation study estimated no mortality difference for screening beyond age 74 years (adjusted hazard ratio, 1.00 [95% CI, 0.83 to 1.19]). Advanced cancer detection did not differ following annual or biennial screening intervals in a nonrandomized study. Three trials compared digital breast tomosynthesis (DBT) mammography screening with digital mammography alone. With DBT, more invasive cancers were detected at the first screening round than with digital mammography, but there were no statistically significant differences in interval cancers (pooled relative risk, 0.87 [95% CI, 0.64-1.17]; 3 studies [n = 130 196]; I 2  = 0%). Risk of advanced cancer (stage II or higher) at the subsequent screening round was not statistically significant for DBT vs digital mammography in the individual trials. Limited evidence from trials and nonrandomized studies suggested lower recall rates with DBT. An RCT randomizing individuals with dense breasts to invitations for supplemental screening with magnetic resonance imaging reported reduced interval cancer risk (relative risk, 0.47 [95% CI, 0.29-0.77]) and additional false-positive recalls and biopsy results with the intervention; no longer-term advanced breast cancer incidence or morbidity and mortality outcomes were available. One RCT and 1 nonrandomized study of supplemental ultrasound screening reported additional false-positives and no differences in interval cancers.

Conclusions and Relevance   Evidence comparing the effectiveness of different breast cancer screening strategies is inconclusive because key studies have not yet been completed and few studies have reported the stage shift or mortality outcomes necessary to assess relative benefits.

Breast cancer is the second leading cause of cancer mortality for US women, despite a steady overall decline in breast-cancer mortality rates over the past 20 years. 1 The average age-adjusted rate for the years 2016-2020 was 19.6 per 100 000, with an estimated 43 170 deaths in 2023. 1 , 2 The majority of cases occur between the ages of 55 and 74 years, 1 and incidence is highest among women ages 70 to 74 (468.2 per 100 000). 3 Non-Hispanic White women have the highest breast cancer incidence, 4 but mortality is 40% higher for non-Hispanic Black women (27.6 per 100 000) compared with White women (19.7 per 100 000); non-Hispanic Black women experience lower 5-year survival regardless of the cancer subtype or stage at the time of detection. 1 , 5 - 7

Previous reviews of breast cancer screening effectiveness established the benefits and harms of mammography based primarily on large, long-term trials. 8 , 9 In 2016, the US Preventive Services Task Force (USPSTF) recommended screening for breast cancer in women starting at age 50 years every 2 years continuing through age 74 years (B recommendation) and that screening from ages 40 to 49 years should be based on clinical discussions of patient preferences and individual breast cancer risk (C recommendation). 10 This comparative effectiveness systematic review of breast cancer screening strategies was conducted concurrently with a separate decision modeling study. 11 Both informed the USPSTF updated breast cancer screening recommendations. 12

This review addressed 3 key questions (KQs) on the comparative effectiveness and harms of different screening strategies ( Figure 1 ). Methodological details including study selection, a list of excluded studies, detailed study-level results for all outcomes and for specific subpopulations, and contextual observations are available in the full evidence report. 14

Studies included in the 2016 USPSTF reviews 8 , 9 , 15 , 16 were evaluated for inclusion with eligibility criteria for the current review. In addition, database searches for relevant studies published between January 2014 and August 22, 2022, were conducted in MEDLINE, the Cochrane Central Register of Controlled Clinical Trials, and the Cochrane Database of Systematic Reviews (eMethods in the Supplement ). Reference lists of other systematic reviews were searched to identify additional relevant studies. ClinicalTrials.gov was searched for relevant ongoing trials. Ongoing surveillance to identify newly published studies was conducted through March 2024 to identify major studies published in the interim. Two new nonrandomized studies were identified 17 , 18 and are not further discussed, as they would not change interpretation of the review findings or conclusions.

Two independent reviewers screened titles, abstracts, and relevant full-text articles to ensure consistency with a priori inclusion and exclusion criteria (eTable 1 in the Supplement ). We included English-language studies of asymptomatic screening populations not at high risk for breast cancer. The eligible population for this review is adult females (sex assigned at birth). For consistency with the underlying evidence, the term “women” is used throughout this report; however, cancer registries and studies of breast cancer generally infer gender based on physiology and medical history rather than measuring self-reported gender. Included studies compared mammography screening modalities (mammography with or without digital breast tomosynthesis [DBT]), different screening strategies with respect to interval, age to start, age to stop, or supplemental screening strategies using ultrasound or magnetic resonance imaging (MRI) with mammography.

For KQ1, randomized clinical trials (RCTs) or nonrandomized studies of interventions with contemporaneous comparison groups that reported breast cancer morbidity, mortality, all-cause mortality, or quality of life were included. For KQ2, the primary outcome of interest was progression to advanced breast cancer, defined for this review as stage IIB or higher, which encompasses tumors with local lymph node involvement or distant metastases. 19 Study-defined advanced breast cancer outcomes were used when this outcome was not reported (eg, stage II or higher). Invasive breast cancer detection outcomes from multiple screening rounds can indicate whether a screening modality or strategy reduces the risk of advanced cancer by detecting early cancers that would otherwise have progressed (stage shift), thereby potentially reducing breast cancer morbidity and mortality. 20 - 23

For KQ3, RCTs and nonrandomized studies of interventions reporting adverse events, including psychological harms, radiation exposure, and interval invasive cancers (incident or missed due to false-negative screening) were included, regardless of the number of screening rounds reported. False-positive recall, false-positive biopsy recommendation, and false-positive biopsy rates (individuals who underwent a biopsy for a benign lesion) were obtained from included RCTs and from nonrandomized studies reporting cumulative rates of these potential harms of screening.

Two reviewers evaluated all articles that met inclusion criteria using prespecified quality criteria (eTable 2 in the Supplement ). Discordant quality ratings were resolved through discussion and input from a third reviewer. Risk-of-bias assessment was conducted using the USPSTF-specific criteria for randomized trials 13 and an adapted tool from the Risk of Bias in Non-Randomized Studies of Interventions (ROBINS-I). 24 Studies determined to be at high risk of bias were excluded. One reviewer extracted key elements of included studies into standardized evidence tables in DistillerSR (Evidence Partners) and a second reviewer checked the data for accuracy. Limited evidence on sub-KQs is available in the full report. 14 When available, reported relative risks were provided in the tables, but we calculated and reported crude effect estimates and confidence intervals when studies did not provide them. For KQ2 intermediate detection outcomes, the definition of advanced cancer reported in the studies was used for synthesis; commonly this was stage II or later. Comparisons of prognostic characteristics or markers (eg, grade, tumor size, nodal involvement, receptor status) were included for comparisons as data allowed.

All quantitative analyses were conducted in Stata version 16 (StataCorp). The presence of statistical heterogeneity was assessed among pooled studies using the I 2 statistic. Where effects were sufficiently consistent and clinical and statistical heterogeneity low, random-effects meta-analyses were conducted using the restricted maximum likelihood; all tests were 2-sided, with P  < .05 indicating statistical significance.

Aggregate strength of evidence (ie, high, moderate, or low) was assessed for each KQ and comparison using the approach described in the Methods Guide for the Effectiveness and Comparative Effectiveness Reviews, 25 based on consistency, precision, publication bias, and study quality.

Investigators reviewed 10 378 unique citations and 419 full-text articles for all KQs ( Figure 2 ). Twenty studies reported in 45 publications were included. 26 - 45 A full list of included studies by KQ is located in eTable 3 in the Supplement .

Key Question 1. What is the comparative effectiveness of different mammography-based breast cancer screening strategies (eg, by modality, interval, initiation and stopping age, use of supplemental imaging, or personalization based on risk factors) on breast cancer morbidity and mortality?

Two nonrandomized studies reported on the association of different screening programs with breast cancer morbidity and mortality. One study was designed to compare different ages to stop screening 30 and another compared annual and triennial screening intervals. 41

A fair-quality observational study (n = 1 058 013) on age to stop screening used an emulated trial methodology to analyze a random sample of US Medicare A and B claims data for enrollees aged 70 to 84 years (1999 to 2008), eligible for breast cancer screening, and with at least a 10-year estimated life expectancy. The study estimated the effect of stopping screening at ages 70, 75, and 80 years compared with continued annual screening. 30 , 46 Continuation of screening between the ages of 70 and 74 years was associated with reduced mortality risk based on survival analysis (hazard ratio, 0.78 [95% CI, 0.63 to 0.95]), but the absolute difference in the risk of death for the age group was small and the confidence interval included null (1.0 fewer deaths per 1000 screened [95% CI, −2.3 to 0.1]). These results indicate a difference in the cumulative incidence curves that approached a difference in the mortality risk for the age group. Conversely, continued screening vs no screening from ages 75 to 84 years did not result in statistically significant differences in the absolute risk of breast cancer mortality (0.07 fewer deaths per 1000 [95% CI, –0.93 to 1.3]) or the cumulative mortality incidence (hazard ratio, 1.00 [95% CI, 0.83 to 1.19]).

A fair-quality nonrandomized clinical study (n = 14 765) conducted in Finland during the years 1985 to 1995 assigned participants aged 40 to 49 years to annual or triennial screening invitations by alternating birth year. 41 The study reported no difference in breast cancer mortality: 20.3 deaths per 100 000 person-years with annual screening invitations and 17.9 deaths per 100 000 person-years with triennial screening invitations (relative risk [RR], 1.14 [95% CI, 0.59-1.27]).

Key Question 2. What is the comparative effectiveness of different mammography-based breast cancer screening strategies (eg, by modality, interval, initiation and stopping age, use of supplemental imaging, or personalization based on risk factors) on the incidence of and progression to advanced breast cancer?

No eligible studies of age to start or stop screening, supplemental screening, or personalized screening were included, because no RCTs or nonrandomized studies reported more than a single round of screening comparing screening strategies. For screening interval, 1 RCT 26 and 1 nonrandomized study, 41 and for comparisons of different screening modalities (DBT vs digital mammography) 3 RCTs 27 , 33 , 42 and 2 nonrandomized studies, 34 , 44 met eligibility criteria.

Two fair-quality studies addressed the effect of screening interval on the characteristics of detected cancers. A fair-quality United Kingdom Co-ordinating Committee on Cancer Research (UKCCCR) RCT comparing screening intervals was conducted as part of the UK National Breast Screening Program. The study randomized participants aged 50 to 62 years to annual (n = 37 530) or triennial (n = 38 492) breast cancer screening during the years 1989 to 1996. 26 After 3 years of screening (1 incidence screen in the triennial screening group), a similar number of cancers (screen-detected and interval) had been diagnosed in the annual and triennial screening groups (6.26 and 5.40 per 1000 screened, respectively; RR, 1.16 [95% CI, 0.96 to 1.40]). No statistically significant differences were found in the cancer characteristics (tumor size, nodal status, histological grade) between groups over the course of the study.

A fair-quality nonrandomized study using Breast Cancer Surveillance Consortium (BCSC) registry data (1996 to 2012) 39 found the relative risk of being diagnosed with a breast cancer with less favorable prognostic characteristics (stage IIB or higher, tumor size >15 mm, or node-positive) was not statistically different for women screened biennially compared with those screened annually for any age category (40-49, 50-59, 60-69, 70-85 years).

Three fair-quality RCTs 27 , 33 , 42 reported cancer detection over 2 rounds of screening, comparing the effects of screening with DBT and digital mammography on the presence of advanced cancer at subsequent screening rounds ( Table 1 ). Participants were randomized to the DBT intervention group or the digital mammography control group at a first round of screening, followed in 2 trials by a second round of screening with digital mammography for all second-round participants (Proteus Donna, 27 RETomo 42 ) and in 1 trial with DBT for all second-round participants (To-Be 33 ). The trials used an identical screening modality for both study groups at the second round because using the same instrument is a stronger design for detection of stage shift.

The RCTs reported increased detection of invasive cancer with DBT at the first round of screening (pooled RR, 1.41 [95% CI, 1.20 to 1.64]; 3 RCTs [n = 129 492]; I 2  = 7.6%) and no statistical difference in invasive cancer at the subsequent screening (pooled RR, 0.87 [95% CI, 0.73 to 1.05]; 3 RCTs [n = 105 064]; I 2  = 0%) (eFigure 1 in the Supplement ). 27 , 33 , 42 There was no statistically significant difference in the incidence of advanced cancers at the subsequent screening round (progression of cancers not found at prior screening that would indicate stage shift) in the individual trials ( Figure 3 ). Results were inconsistent and thus not pooled for the advanced cancer, larger tumor (>20 mm), and node-positive cancer outcomes. The results for histologic grade 3 cancer at the second screening were consistent (pooled RR, 0.97 [95% CI, 0.61-1.55]; 3 RCTs [n = 105 244]; I 2  = 0%) ( Figure 3 ). Due to the small number of cases, it was not possible to assess differences in the detection of cancers lacking hormone or growth factor receptors (ie, triple-negative cancers) that have the worst prognosis among breast cancer subtypes.

Two fair-quality nonrandomized studies of interventions (NRSIs), including a US study using BCSC data, compared breast cancer detection outcomes from screening over multiple rounds (≥2) with either DBT-based mammography or digital mammography alone. 34 , 44 The findings were generally consistent with the trial results for cancer detection and stage shift.

Key Question 3. What are the comparative harms of different mammography-based breast cancer screening strategies (modality, interval, initiation age, use of supplemental imaging, or personalization based on risk factors)?

No eligible studies of age to start screening or personalized screening were identified. For age to stop screening, 1 fair-quality nonrandomized study met eligibility criteria. 30 For comparisons of potential harms associated with different screening intervals, a fair-quality RCT 26 and 2 fair-quality nonrandomized studies 39 , 41 were included. For comparisons of different screening modalities (DBT vs digital mammography), 4 RCTs (3 good- and 1 fair-quality) 27 , 31 , 33 , 42 and 7 fair-quality nonrandomized studies were included. 28 , 32 , 34 - 36 , 43 , 44

In the NRSI using an emulated trial methodology to evaluate the age to stop screening, 30 the 8-year cumulative proportion of participants with a breast cancer diagnosis was higher among those who continued annual screening from ages 70 to 84 years (5.5%) compared with those who discontinued screening (3.9%) at age 70 years. Because fewer cancers were diagnosed among those who discontinued screening, there was a lower risk of undergoing cancer treatment and experiencing related morbidity. Notably, for participants aged 75 to 84 years, screening (and treatment) were not associated with lower breast cancer mortality (see KQ1 results).

The UKCCCR trial included for KQ2 26 reported fewer interval cancers (false-negative and incident cancers) diagnosed in the annual invitation group compared with triennial screening (1.84 vs 2.70 per 1000 women screened, respectively; RR, 0.68 [95% CI, 0.50 to 0.92]). The nonrandomized clinical trial conducted in Finland included for KQ1 41 also reported interval cancers diagnosed with annual vs triennial screening and found no statistical difference in incidence ( P  = .22, data not reported). Data from 2 studies from the BCSC registry reported higher probabilities of false-positive recalls and biopsy recommendations with annual screening compared with biennial screening and no statistical difference in interval cancers in adjusted analyses. 32 , 39 , 44

Four RCTs (3 good-quality, 1 fair-quality) 27 , 31 , 33 , 42 and 7 fair-quality nonrandomized studies 28 , 32 , 34 - 36 , 43 , 44 reported outcomes related to potential screening harms associated with DBT-based screening compared with digital mammography–only screening, including interval cancer rates, round-specific and cumulative false-positive recalls and biopsies, and radiation exposure. Meta-analysis of 3 large trials did not show a statistically significant difference in rates of interval cancer after screening with DBT compared with digital mammography (pooled RR, 0.87 [95% CI, 0.64 to 1.17]; 3 RCTs [n = 130 196]; I 2  = 0%) (eFigure 2 in the Supplement ). 27 , 33 , 42

Data on interval cancers were also obtained from 7 nonrandomized studies. 28 , 32 , 34 - 36 , 43 , 44 The most recent BCSC analysis, reporting interval cancer rates across multiple screening rounds with either DBT or digital mammography, did not identify statistically significant differences in invasive or advanced interval cancers. 44

The effects of DBT screening on false-positive recall and false-positive biopsy rates varied across studies 27 , 33 , 42 and by screening round, with small or no statistical differences between study groups, not consistently favoring DBT-based mammography or digital mammography.

Evidence from 2 nonrandomized BCSC studies provided false-positive results across several screening rounds. 32 , 44 In 1 study, rates of false-positive recall and false-positive biopsy rates were lower with DBT in initial screening rounds, but differences were attenuated and not statistically significant compared with digital mammography only after additional rounds of screening ( Table 2 ). 44 The other study reported no statistical difference in 10-year cumulative false-positive biopsy recommendation rates between biennial DBT and digital mammography screening, but false-positive recall was slightly lower with DBT (eFigures 3 and 4 in the Supplement ); no differences by modality were identified for individuals with extremely dense breasts in stratified analyses (eFigure 5 in the Supplement ). 32

Four RCTs 27 , 31 , 33 , 42 and 1 NRSI 35 reported the mean, median, or relative radiation dose received in each study group at a single screening round. The 3 studies using DBT/digital mammography screening reported radiation exposure approximately 2 times higher in the intervention group compared with the digital mammography–only group. 27 , 35 , 42 Differences between study groups in radiation exposure were smaller in studies using DBT with synthetic digital mammography. 33 , 47

The Dense Tissue and Early Breast Neoplasm Screening (DENSE) trial, a good-quality RCT conducted in the Netherlands, randomized (1:4) participants aged 50 to 75 years with extremely dense breasts and negative mammography findings (2011-2015) (n = 40 373) to an invitation or no invitation for supplemental MRI screening. 45 (The RCT was not included for KQ2 because second round results in the control group were unavailable). Fifty-nine percent of those randomized to the invitation underwent an MRI examination (n = 4783). In intention-to-treat analysis, 2.2 per 1000 experienced interval breast cancer diagnoses in the supplemental screening invitation group, compared with 4.7 per 1000 screened in the digital mammography control group (RR, 0.47 [95% CI, 0.29 to 0.77]). Adverse events related to the supplemental MRI screening reported in the trial included 5 classified as serious adverse events (2 vasovagal reactions and 3 allergic reactions to the contrast agent) and 2 reports of extravasation (leaking) of the contrast agents and 1 shoulder subluxation. Twenty-seven participants (0.6% of the MRI group) reported a serious adverse event within 30 days of the MRI. Those who underwent supplemental MRI screening also experienced additional recalls (94.9 per 1000 screened), false-positive recalls (80.0 per 1000 screened), and false-positive biopsies (62.7 per 1000 screened).

A fair-quality nonrandomized study used claims data from commercially insured women (MarketScan database) aged 40 to 64 years who had received at least 1 bilateral screening breast MRI (n = 9208) or mammogram (n = 9208) between January 2017 and June 2018. 29 Following propensity score matching, those undergoing screening with MRI were more likely to have additional health care cascade events such as office visits and follow-up tests unrelated to breast conditions (adjusted difference between groups, 19.6 per 100 screened [95% CI, 8.6 to 30.7]) in the subsequent 6 months.

A fair-quality RCT, the Japan Strategic Anti-cancer Randomized Trial, randomly assigned asymptomatic women aged 40 to 49 years (2007-2011) to breast cancer screening with mammography plus handheld ultrasound (digital mammography/ultrasound) (n = 36 859) or mammography only (digital mammography) (n = 36 139). 40 The relative risk of invasive interval cancer was not statistically significantly different for digital mammography/ultrasound vs digital mammography only (RR, 0.58 [95% CI, 0.31 to 1.08]). This result differs from the statistically significant population-average effect reported in the study ( P  = .03), which included interval ductal carcinoma in situ (proportion difference, −0.05% [95% CI, −0.09 to 0]). Those undergoing ultrasound in addition to digital mammography experienced 48.0 per 1000 additional false-positive recall results compared with those assigned to digital mammography screening only.

A fair-quality nonrandomized study using data from 2 BCSC registry sites compared screening outcomes for participants receiving ultrasonography on the same day as a screening mammogram (digital mammography/ultrasound) (n = 3386, contributing 6081 screens) compared with those that received only a mammogram (digital mammography) (n = 15 176, contributing 30 062 screens). 37 However, 31% of participants had a first-degree family history of breast cancer or previous breast biopsy. There was no statistical difference in interval cancer risk (adjusted RR, 0.67 [95% CI, 0.33 to 1.37]), and rates of false-positive biopsy were twice as high for the mammography/ultrasound group (adjusted RR, 2.23 [95% CI, 1.03 to 2.58]).

Prior screening effectiveness reviews based on large trials initiated in previous decades established a statistically significant mortality benefit for mammography screening of women aged 50 to 69 years. 8 , 9 , 15 The current review considered comparative effectiveness questions on the relative benefits and harms of different screening start and stop ages, intervals, and modalities for women at average breast cancer risk. Findings are summarized in Table 3 .

The evidence was insufficient for addressing the age to start or end screening. No eligible studies comparing different ages to start screening were identified. Limited evidence from 1 nonrandomized study, using an emulated trial study design, suggested that screening beyond age 74 years may not reduce breast cancer mortality. 30

Evidence was also insufficient for evaluating the effect of screening intervals on breast cancer morbidity and mortality. Two nonrandomized studies found no difference in breast cancer outcomes. 26 , 39 Moderate evidence supported longer screening intervals (eg, biennial) to reduce the cumulative risk of false-positive recall and biopsy. The observational studies of different screening intervals compared individuals who self-selected or were referred for different screening intervals, contributing to risk of bias in the results.

Results from 3 RCTs 27 , 33 , 42 and 2 nonrandomized studies 34 , 44 provided moderate evidence that DBT-based mammography does not reduce the risk of invasive interval cancer or advanced cancer at subsequent screening rounds. Additional rounds of screening and longer follow-up are needed to fully evaluate whether DBT reduces breast cancer morbidity and mortality. Consistent with trial findings, a nonrandomized BCSC study did not find reduced risks of advanced or interval cancers with DBT. 44 Limited evidence from trials on harms of screening with DBT 27 , 33 , 42 indicated similar false-positive recall and biopsy rates. An observational BCSC study did not show differences in the 10-year cumulative false-positive biopsy rates 32 ; lower false-positive recall and biopsy with DBT screening were attenuated after several screening rounds. 44 Additional research is needed to ascertain whether DBT-based screening would reduce false-positives over a lifetime of screening.

The evidence was not adequate to evaluate the benefits and harms of supplemental MRI screening for people with dense breasts. No eligible studies were identified that provide evidence on breast cancer morbidity or mortality outcomes with supplemental MRI screening compared with mammography alone among individuals with dense breasts. The DENSE trial 45 reported fewer interval cancers with 1 round of supplemental MRI screening, but results from a second screening round are not yet published. Evidence of higher advanced cancer incidence in the mammography-only group relative to the MRI group would be needed to anticipate effects on morbidity or mortality. Supplemental MRI led to additional false-positive recalls and biopsies, and uncommon but serious adverse events were observed. 45 Two recent systematic reviews of the test performance literature reported higher cancer detection with supplemental MRI screening along with substantially increased recall and biopsy rates among individuals without cancer. 48 , 49

Lack of a standardized and reliable assessment tool for measuring breast density and density variation across the lifespan pose challenges for research into the optimal screening strategy for persons with dense breasts. 16 Research is also needed to evaluate personalized risk-based screening, based on breast cancer risk factors and personal screening preferences. The ongoing WISDOM trial and My Personalized Breast Screening study (expected completion in 2025) may help to address these research gaps. 50 , 51

Breast cancer is an active area of research, yet few longitudinal RCTs comparing different screening strategies have been conducted following completion of the major trials that established the effectiveness of mammography for reducing breast cancer mortality for women aged 50 to 69 years. This review included 6 new randomized trials, 27 , 31 , 33 , 40 , 42 , 45 4 comparing DBT with digital mammography screening 27 , 31 , 33 , 42 and 2 on supplemental screening compared with mammography alone. 40 , 45 Three of these trials are ongoing 31 , 40 , 45 and have reported preliminary results only. Observational studies were also included, but few studies were available that followed up a screening population over time to compare the health outcomes associated with different screening approaches. These studies, while potentially more representative of a screening population, have higher risk of biased results due to confounding and selection.

Changes in population health, imaging technologies, and available treatments may limit the applicability of previous studies. Recent trials included in this review were conducted outside of the US and enrolled mostly White European populations. No studies evaluated screening outcomes for racial or ethnic groups in the US that experience health inequities and higher rates of breast cancer mortality. Black women are at highest risk of breast cancer mortality, 52 with lower 5-year survival than all other race and ethnicity groups. 7 Breast cancer mortality risk also increases at younger ages for Black women compared with White women. 53 This review did not address additional factors beyond screening that contribute to breast cancer mortality inequities. 54 Rigorous research is essential to understand and identify improvements needed along the pathway from screening to treatment 55 and to address inequities in follow-up time after a positive screening result, time to diagnosis, 56 - 60 and receipt of high-quality treatment and support services. 59 , 61 , 62

Evidence comparing outcomes for different screening intervals and ages to start and stop screening was limited or absent. Trials of personalized screening based on risk and patient preferences are in progress and may address evidence gaps related to optimal screening start ages and intervals. Research is needed to better characterize potential harms of screening, including patient perspectives on experiencing false-positive screening results. Women with false-positive screening results may be less likely to return for their next scheduled mammogram, as reported in a large US health system study. 55 , 63 Rigorous studies that enroll screening populations and report advanced cancer detection, morbidity, and mortality outcomes from multiple rounds of screening are needed to overcome persistent limitations in the evidence on breast cancer screening. Multiple screening rounds are essential to determine whether a screening modality or strategy reduces the risk of advanced cancer by detecting early cancers that would otherwise have progressed (stage shift), potentially reducing breast cancer morbidity and mortality. 20 - 23 , 64

The potential benefits of risk-stratified screening strategies, including the use of supplemental screening with ultrasound or MRI, have not been fully evaluated, although some harms are evident. Longer term follow-up on existing comparative effectiveness trials, complete results from ongoing RCTs of personalized screening programs, 65 , 66 and rigorous new studies are needed to further strengthen the evidence and optimize breast cancer screening strategies.

Evidence comparing the effectiveness of different breast cancer screening strategies is inconclusive because key studies have not yet been completed and few studies have reported the stage shift or mortality outcomes necessary to assess relative benefits.

Accepted for Publication: November 23, 2023.

Published Online: April 30, 2024. doi:10.1001/jama.2023.25844

Corresponding Author: Jillian T. Henderson, PhD, MPH, Kaiser Permanente Evidence-based Practice Center, Center for Health Research, Kaiser Permanente Northwest, 3800 N Interstate Ave, Portland, OR 97227 ( [email protected] ).

Author Contributions: Dr Henderson had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical review of the manuscript for important intellectual content: Henderson, Weyrich, Miller.

Statistical analysis: Henderson.

Administrative, technical, or material support: Webber, Melnikow.

Supervision: Henderson.

Conflict of Interest Disclosures: None reported.

Funding/Support: This research was funded under contract number 75Q80120D00004, Task Order 75Q80121F32004, from the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services.

Role of the Funder/Sponsor: Investigators worked with USPSTF members and AHRQ staff to develop the scope, analytic framework, and key questions for this review. AHRQ had no role in study selection, quality assessment, or synthesis. AHRQ staff provided project oversight, reviewed the report to ensure that the analysis met methodological standards, and distributed the draft for peer review. Otherwise, AHRQ had no role in the conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript findings.

Disclaimer: The opinions expressed in this document are those of the authors and do not reflect the official position of AHRQ or the US Department of Health and Human Services.

Additional Contributions: The authors gratefully acknowledge the following individuals for their contributions to this project: Howard Tracer, MD (AHRQ); Heidi D. Nelson, MD, MPH, MACP (Kaiser Permanente Bernard J. Tyson School of Medicine); current and former members of the USPSTF who contributed to topic deliberations; and Evidence-based Practice Center staff members Melinda Davies, MA, Jill Pope, and Leslie A. Purdue, MPH, for technical and editorial assistance at the Kaiser Permanente Center for Health Research. USPSTF members, peer reviewers, and federal partner reviewers did not receive financial compensation for their contributions.

Additional Information: A draft version of this evidence report underwent external peer review from 5 content and methods experts (Nehmat Houssami, MBBS, MPH, Med, PhD [University of Sydney-Australia]; Patricia Ganz, MD [UCLA]; Gerald Gartlehner, MD, MPH [Cochrane Austria]; Karla Kerlikowske, MD [UC San Francisco]; Lisa Newman, MD, MPH [New York Presbyterian/Weill Cornell Medical Center]) and 4 scientific representatives from 3 federal partner organizations (Centers for Disease Control and Prevention; Office of Research on Women’s Health; National Institute on Minority Health and Health Disparities). Comments were presented to the USPSTF during its deliberation of the evidence and were considered in preparing the final evidence review.

Editorial Disclaimer: This evidence report is presented as a document in support of the accompanying USPSTF Recommendation Statement. It did not undergo additional peer review after submission to JAMA .

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 14, Issue 4
• Carbon footprint of healthcare systems: a systematic review of evidence and methods
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-0180-2358 Mattis Keil 1 , 2 ,
• Leonie Frehse 3 ,
• Marco Hagemeister 3 ,
• Mona Knieß 3 ,
• Oliver Lange 1 , 4 ,
• Tobias Kronenberg 5 ,
• Wolf Rogowski 1
• 1 Department of Health Care Management, Institute of Public Health and Nursing Research, Health Sciences , University of Bremen , Bremen , Germany
• 2 Joint research cluster “Healthy City Bremen” of the University of Bremen, Bremen University of Applied Sciences and Apollon University of Applied Sciences Bremen , Bremen , Germany
• 3 Professional Public Decision Making, Faculty of Cultural Studies , University of Bremen , Bremen , Germany
• 4 Leibniz ScienceCampus Digital Public Health , Bremen , Germany
• 5 Department of Economics , Bochum University of Applied Sciences , Bochum , Germany
• Correspondence to Mattis Keil; m.keil{at}uni-bremen.de

Objective Given the demand for net-zero healthcare, the carbon footprint (CF) of healthcare systems has attracted increasing interest in research in recent years. This systematic review investigates the results and methodological transparency of CF calculations of healthcare systems. The methodological emphasis lies specifically on input–output based calculations.

Design Systematic review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guideline.

Data sources PubMed, Web of Science, EconBiz, Scopus and Google Scholar were initially searched on 25 November 2019. Search updates in PubMed and Web of Science were considered until December 2023. The search was complemented by reference tracking within all the included studies.

Eligibility criteria We included original studies that calculated and reported the CF of one or more healthcare systems. Studies were excluded if the specific systems were not named or no information on the calculation method was provided.

Data extraction and synthesis Within the initial search, two independent reviewers searched, screened and extracted information from the included studies. A checklist was developed to extract information on results and methodology and assess the included studies’ transparency.

Results 15 studies were included. The mean ratio of healthcare system emissions to total national emissions was 4.9% (minimum 1.5%; maximum 9.8%), and CFs were growing in most countries. Hospital care led to the largest relative share of the total CF. At least 71% of the methodological items were reported by each study.

Conclusions The results of this review show that healthcare systems contribute substantially to national carbon emissions, and hospitals are one of the main contributors in this regard. They also show that mitigation measures can help reduce emissions over time. The checklist developed here can serve as a reference point to help make methodological decisions in future research reports as well as report homogeneous results.

• decision making
• health services administration & management
• change management
• international health services
• organisation of health services

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information. Data are available upon reasonable request. The data that support the findings of this study are available in online supplemental file 4 'System description and results' and online supplemental file 5 'Methods and transparency'. Further data are available from the corresponding author (MKeil), upon reasonable request.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2023-078464

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

The assessment of methodological choices and the transparency of methods when assessing the greenhouse gas (GHG) emissions of entire sectors in systematic reviews can help deepen our understanding of the results.

The systematic review of all available evidence on GHG emissions of and within healthcare can help to understand its impact and to identify reduction potentials.

This review was limited to articles in English and German, and excluded assessments, grey literature from public reports, and reports from statistical offices published in other languages.

Introduction

Climate change is one of the most pressing issues of our time. 1 Considering the correlation between the gross domestic product and carbon emissions, 2 the healthcare industry is likely an essential contributor to greenhouse gas (GHG) emissions. Demographic shifts and income effects have likely spurred greater demand for healthcare services, a trend projected to persist and further elevate the economic significance of the healthcare industry. 3 Evidence on healthcare’s GHG emissions is needed to understand its role better.

Methods for calculating a carbon footprint (CF) can be broadly categorised into bottom-up and top-down approaches. Bottom-up methods, such as process-based lifecycle assessments, require extensive data, which currently limits their application at a sectoral level. However, the CF of various sectors can be estimated using a more uncertain top-down methodology, providing a trade-off for broader coverage. In this case, emissions are divided according to the final demand or economic sectors of emission occurrence.

Input–output (I–O) analysis, which follows this approach, can be used to estimate sectoral CF. 4 Calculations of the CF use the static open-quantity I–O model in combination with an environmental extension. They rely on two fundamental building blocks: an I–O table and a demand vector. The I–O table describes the interactions between the sectors of production, often in monetary terms, and are usually constructed by national statistics offices. With additional information on their environmental impact, the emission intensity of a sector and its upstream production processes can be calculated. The demand vector represents the expenditures of the relevant sectors. For example, the demand vector of the healthcare sector includes expenditure on diesel fuel to power ambulances, electricity consumed by hospitals, and all other forms of energy. It may be necessary to synchronise the structures of the I–O table and the demand vector by balancing the definitions of different sectors and adjusting the level of sectoral aggregation.

I–O models can be grouped into single-region I–O (SRIO) and multi-region I–O (MRIO) models. SRIO models use I–O data from a single country, thus restricting their scope to domestic production and emissions only. MRIO models connect multiple I–O tables from multiple countries, and can thus account for different levels of production and ‘trade’ in emissions (ie, emissions occurring in one country related to the final demand of another country). The need for synchronised data from multiple countries complicates the development and update of the data of MRIO models.

The results of CF calculations for a specific sector can be influenced by methodological choices, including the selection between SRIO or MRIO models and the GHGs taken into account. Therefore, comprehensive reporting is needed to ensure the transparency of methodological choices, the data and the results. However, our search of the literature yielded neither a standardised procedure nor standardised reporting.

The aim of this study is to conduct a systematic review of research using I–O analysis to quantify the CF of systems, encompassing total CF, CF per capita, and its proportion relative to the national CF. Furthermore, data on emission trends over time, can deepen the understanding of the trajectory of the CF of healthcare systems. Finally, an assessment of the methodological choices and their transparency within the reviewed studies can help to discuss the state of the methodology and provides a foundation to discuss methodological differences between the studies.

Search strategy and selection criteria

This systematic review was performed by following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines 5 (the checklist is provided in online supplemental file 1 ). The databases PubMed, Web of Science, EconBiz, Scopus and Google Scholar were searched for studies on 25 November 2019. The full search strategy is provided in online supplemental file 2 . The search was complemented by reference tracking within all the included studies. The updated search considered hits in PubMed and Web of Science up to December 2023.

Supplemental material

Following the screening of the titles and abstracts, studies were included for further investigation if they had (1) addressed the method of CF calculation, (2) addressed one or more healthcare systems or subsystems and (3) been written in English or German. A healthcare system was defined as the national healthcare system, federal system and/or state system. Single entities, such as individual hospitals, and specialised branches, such as dentistry, were excluded. In addition to the criteria used for screening the titles and abstracts of articles, full-text articles were excluded if they (1) did not name the specific healthcare (sub)system, (2) did not calculate the CF or (3) did not provide any information on the method of calculation used. In the initial search, two of the authors separately screened titles and abstracts, read the full text, extracted data and assessed the transparency. In the case of disagreement, decisions were made through discussion until a consensus was reached. During the search update these steps were conducted by one person.

Data extraction and analysis

The CF per capita, the contribution of healthcare to the country’s total CF emissions, and the origins of emissions were used as main results of the studies. The breakdown of the emission sources could be in scopes, demand categories or places of origin. The Greenhouse Gas Protocol Corporate Accounting and Reporting Standard 6 proposes three standardised scopes. Scope 1 represents direct emissions from owned or controlled sources, scope 2 represents indirect emissions generated by the purchased energy and scope 3 represents all indirect emissions that occur in the value chain. The categories of demand included the classes of expenditures of the demand vector, and the places of the origin of emissions were divided into hospitals, ambulatory services and so on.

In addition to evaluating their general characteristics and results, we developed and applied a checklist to assess the methodological transparency of the studies under consideration. We opted to use the term ‘transparency’ rather than ‘quality’ to address the issue that even a flawless study could receive a low score if the authors failed to adequately report their methodology. The checklist served as both a qualitative extraction tool and a quantitative transparency tool. The qualitative extraction tool facilitated the assessment of information from each included study, with responses to each criterion collected accordingly. As a quantitative transparency tool, it was evaluated whether the criteria were adequately addressed. When information was provided, the criterion was considered fulfilled, resulting in an increase in the transparency score. All criteria were weighted equally, therefore for each ‘fulfilled’ criterion one point was added to the transparency score, with a maximum of 17 points per study.

The utilisation of I–O data can introduce uncertainties into the assessment, given that the top-down approach relies on aggregated information from industrial sectors. When heterogeneous products with varying emission intensities are grouped into one industry, aggregation errors might occur: the average emission intensity of the aggregated industry would not appropriately reflect the emissions caused by the specific product within the industry. 7 Therefore, information on the extend of usage of I–O method (criterion 5), and the number of industry sectors (criterion 12) could help to understand the scope of this uncertainty.

The choice between MRIO and SRIO (criterion 11) can also help to understand the level of uncertainty. While MRIOs can account for differences between countries and trade between these countries, SRIO might provide a more detailed framework of the domestic economy. Finally, the specific source of the I–O tables (criterion 9) and emission data (criterion 13) can help the reader to assess the quality of the used data.

Similar to the I–O data, the level of aggregation within the demand data can impact the accuracy of the results. The number of demand or expenditure categories (criterion 8) can indicate on the level of aggregation and the source of demand data (criterion 6) could help to assess the quality of the data source. The quality of the outcomes is also influenced by the alignment between the temporal representativeness of the demand data (criterion 7) and the I–O data (criterion 10). Changes over time (eg, in technology, import and exports) can impact the results and in the best case both data sources refer to the same year. Finally, information on the matching process of demand categories and industry sectors, the publication of the concordance matrix (criterion 15), increases transparency for the reader.

The quantitative (criterion 16) and qualitative (criterion 17) assessment of uncertainty helps the readers to contextualise the results. A list of the included GHGs can indicate the scope of the study, in this case 0.5 were given, when the unit (typically CO 2 equivalents (CO 2 eq)) was mentioned and another 0.5 points if all included GHGs were listed. For the final transparency checklist, the criteria on outcomes ( table 1A ) and on methodology ( table 1B ) were combined. A more detailed description of the transparency criteria are provided in online supplemental file 3 .

• View inline

(A) Extracted outcomes. (B) Extracted methodological items

Emissions over time

To assess trends in GHG emissions of healthcare, data from all studies that reported total emissions for more than 1 year were taken. The data were normalised to the respective starting point of the report as a base year. Therefore, GHG emissions of time period t were divided by the GHG emissions of the base year t 0 and used in a descriptive analysis.

Patient and public involvement

A total of 4285 records were identified in the three searches ( figure 1 ). After removing duplicates and searching for eligible title, abstracts and full texts, 15 reports were included in this review ( figure 1 ). A summary of included studies is provided in table 2 . The detailed results of the data collection are listed in online supplemental files 4 and 5 .

• Download figure
• Open in new tab
• Download powerpoint

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram, based on Page et al . 5

Characteristics and main results of the studies considered in this review

Characteristics of the studies considered

Eleven studies focused on a single national healthcare system, including England, 8 9 Japan, 10 USA, 11 12 Canada, 13 Scotland, 14 China, 15 Australia, 16 Austria 17 and the Netherlands. 18 The series of CFs from the Sustainable Development Unit of the English NHS was aggregated, and only the newest available report was cited. One study examined the healthcare system of the largest Australian state, New South Wales, 19 while three studies reported on healthcare systems in multiple countries. Pichler et al 20 reported results for 36 countries, Healthcare without Harm for 43 countries, 21 and the investigation by Lenzen et al 22 considered 189 countries.

Excluding the one that assessed the Scottish NHS, all studies were published after 2016. However, it is worth noting that the year of the analysis could be older. For instance, the study by Nansai et al 10 was published in 2020 but used demand data from 2011.

Differences in methodology and data

Eleven studies considered top-down data on emissions, while three studies employed bottom-up data on energy usage. 8 9 14 Steenmeijer et al 18 incorporated bottom-up data regarding the quantities of anaesthetic gases, inhalers and travel.

Most single-country studies used SRIO data from the respective governmental offices. In contrast, the studies on British and Dutch healthcare, and those that considered more than one country, used MRIO data. Additionally, Malik et al 23 used MRIO data, however, the database only included data from Australian regions. The EORA database emerged as the most frequently used MRIO database (three times), with one study each employing the WIOD database, the EXIOBASE database and the MRIO database provided by the British Department for the Environment, Food, and Rural Affairs.

The number of production sectors varied among the SRIO studies, ranging from 46 to 405 sectors. The MRIO studies typically used more extensive databases comprising approximately 15 000 sectors, although the MRIO study focusing on the UK considered 424 sectors.

All studies considered CO 2 emissions. However, only five studies considered the six GHGs covered in the Kyoto Protocol; three studies considered CO 2 , methane and nitrous oxide; two reported only that they had used CO 2 eq as unit; and two studies did not report any included GHG or the unit in which the outcomes were reported. The data on emissions were drawn mostly from national accounts in the case of SRIO databases and integrated accounts in the case of MRIO databases. One study did not report the source of its emission account data.

The demand data was taken either from official health expenditure accounts or from international organisations such as the WHO and the World Bank (which uses data provided by national offices and accounts). Lenzen et al 22 identified and directly used data on healthcare-related sectors from the MRIO database EORA. The number of reported expenditure accounts varied, mostly ranging from 13 to 19, although three studies reported fewer accounts. Weisz et al 17 used nine accounts, Wu 15 used eight accounts, and the study on the NHS in England employed five accounts. 9 Due to the distinct methodologies employed by Lenzen et al 22 and the structure of the EORA database, which reports country-specific sectors, they used 163 sectors from the EORA as demand data.

The time periods covered by the demand data were largely consistent with those covered by the respective I–O data. Some studies reporting outcomes for more than 1 year only used one reference year for the I–O database and adjusted the demand data for inflation. 11–13 The lag between the time at which the data were collected and the time of publication of the corresponding study ranged from 3 to 6 years, with deviations in the studies by Nansai et al , 10 Eckelman et al 12 (2 years) and in the report by the SDU. 9 The latter reported the CF periodically; the lag between the latest publication and the latest data was 1 year. 9 Further information on this is provided in online supplemental file 5 .

Five studies provided their concordance matrices, which link the categories of demand with the industrial sectors. The authors of one study had made their matrix available on request, and two articles had referred to a matrix previously used in another study. Five studies did not report their concordance matrices.

Reporting of the results

The origins of emissions were documented six times in the three scopes defined by the GHG protocol. Emission sources were reported eight times in the (sub)categories of final demand, such as hospitals or pharmaceuticals. Two studies reported the economic sector in which the emissions occurred, for example, the textile sector or the manufacture of fuels. Furthermore, three studies reported a breakdown of emissions by employing more than one reporting structure. Several differences were observed in the scopes of the reported results. Some studies directly referenced the GHG protocol while others reported emissions in divisions, such as travel, energy, procurement, etc. 47% of the articles did not normalise the results by reporting the CF per capita.

Overall transparency

Except for the three criteria ‘reporting of the concordance matrix’, ‘uncertainty analysis’ and ‘CF per capita’, all criteria were fulfilled by at least 75% of the studies ( figure 2 ). The studies fulfilled between 70.5% and 94% of all criteria with a mean of 85% ( figure 3 ). The full transparency assessment is provided in online supplemental file 6 .

Fulfilment rate of the transparency and reporting criteria.

Transparency score in percentage per article.

The results of the time series revealed successful efforts to mitigate the CF by the NHS in England and Scotland ( figure 4 ). In the nearly three decades from 1990 to 2019, the English NHS reduced its CF by roughly 25%. The four remaining countries (Japan, Canada, USA and Australia) examined in the studies considered here and the global trend showed increased CF due to healthcare ( figure 4 ). The annual increase in the CF ranged from 0.7% (USA, 2010–2018) to 3.8% (Japan, 2011–2015) over the observed period, with the CFs of Canada (1.9%, 2009–2015), USA (2.8%, 2011–2015) and Australia (2.9%, 2013–2015) in between these extremes. The global trend showed an increase in the CF of 2.7% per year from 2000 to 2015.

Emission trends over time. CF, carbon footprint.

The emission sources were mainly reported using the scope system from the GHG protocol or the categories of expenditure, that is, the categories of final demand. The largest dataset that used the categories of final demand was provided by Pichler et al , 20 who applied this to 36 countries and reported the average values. Medical retail (ie, provider of healthcare products without medical services, eg, pharmacies), hospitals and ambulatory healthcare services constituted 80% of the CF of healthcare, with medical retail contributing 33.1%, hospitals 28.6% and ambulatory healthcare services 18%. They also made a major contribution to the CF in Japan (hospitals, 25.1%; ambulatory services, 22.7%), USA in 2013 (hospital care, 36%; physician and clinical services, 12%) 11 and in 2018 (hospital care, 34.9%; physician and clinical services, 12.6%; ambulatory medical services, 4.8%), 13 Australia (public hospitals, 34.4%; private hospitals, 10.2%; ambulatory medical services, 15%), 16 China (public hospitals, 47%; private hospitals, 4%) 15 and Austria (hospitals, 32%; ambulatory services, 18%). 17 Other important categories of emissions were construction and pharmaceutical products, at around 10%, 11 16 20 with a higher share in China (pharmaceuticals, 18%; construction, 15%). 15

An alternative approach involved categorising emissions into direct emissions, indirect emissions through electricity production, and other indirect emissions. This division along these lines could also align with the three GHG protocol scopes.

By averaging data from 43 countries, HCWH reported a distribution of 17% for scope 1 emissions, 12% for scope 2 emissions and 71% for scope 3 emissions. 21 These findings, particularly the significance of scope 3 emissions, are corroborated by evidence from single-country studies. 8 11 12 14 24 The scope 3 emissions were further divided into those due to travel (patient and visitor travel, and staff commutes), production of pharmaceuticals, and medical instruments and equipment, which accounted for the largest share of scope 3 emissions.

Scotland’s scope 3 travel emissions in 2004 were 18% while those of England accounted for 13% in 2015 and 9.6% in 2018. 9 The share of emissions owing to pharmaceutical production ranged from 11% to 18%, and that owing to medical instruments and equipment accounted for 7%–10% of the total CF. 13 14 24

The ratio of emissions by the healthcare sector to the total CF in studies focused on a single country ranged from 2.7% in China in 2012 15 to 9.8% in the USA in 2013. 11 The three cross-national studies considered here estimated that healthcare had contributed 5.5% 20 on average to the national CF in 2014 and 4.4% in 2015. 22

Interpretation of results

The results indicate that healthcare significantly contributes to the CF, both in absolute numbers and in relation to a country’s overall emissions and its per capita emissions. However, the results varied among the studies, and their calculation methods were heterogeneous and frequently not fully transparent. The breakdown of the sources of emissions revealed the major contribution made by hospitals.

The time series results showed that the trend of emissions due to healthcare was positive in all the countries considered, that is, they were increasing, except in Scotland and England. These results align with the graphical results provided by Lenzen et al . 22 Furthermore, they indicated that the efforts of the British NHS systems to reduce their CF based on the Greener NHS programme was effective in reducing GHG emissions. The breakdown of the sources of emissions verified the important contribution of hospitals. However, hospitals provide the majority of medical care in many countries. Therefore, their large CF is not surprising but might motivate the relevant decision-makers to allocate scarce resources more efficiently. The breakdown further showed that a large portion of the CF of healthcare stemmed from scope 3 emissions. Decision-makers may conclude that the most considerable reduction in emissions can be obtained by considering staff and patient travel. Therefore, ‘greening’ the healthcare sector requires a sustainable transportation system and green healthcare goods.

Most data were from the Organisation for Economic Co-operation and Development (OECD) countries, China and India. The only exception was the work by Lenzen et al , 22 who considered 189 countries in their analysis. 22 However, even if the distribution of countries limits the representativeness of the results, the findings are consistent with the fact that OECD countries are the main emitters of GHGs.

While heterogeneity in methodology, in general, can lead to more robust results and a more informative perspective on the issue at hand, the differences in I–O methodologies to calculate the CF of healthcare may reduce the comparability of the results. However, the choice of method depends on the corresponding research question, for example, while SRIO may be more up-to date and include a more detailed description of the domestic production sectors, MRIO can account for international trade and differences in production emissions between countries.

Limitations

This review has several limitations. First, the review process used here was limited due to restrictions on the language used in the study and those related to access. Second, it is possible that further CF assessments exist which were published in the official languages of many countries in the grey literature, such as publications by national statistics offices or governmental agencies. Because this review included only publications in English and German, many such studies have likely been neglected. Third, the reporting scheme and transparency score used in this study may have limitations. Both were based only on a consensus among the authors. The instruments used to assess the quality of the published studies are typically chosen based on a broad consensus among experts, such as in the case of the Consolidated Health Economic Evaluation Reporting Standards. 25 However, we did not find similar guidance for I–O analyses. Finally, the review is limited as the studies only report averages instead of CIs or data ranges. Only Malik et al 16 report the 68% CI with a range of 20 748 kt CO 2 eq in the results (68% CI 25 398 kt CO 2 eq to 46 146 kt CO 2 eq). Therefore, the results presented in both the individual studies and in this review should not be regarded as precise measurements, but rather as indicative trends or directions.

Implications for further research

This review identified research gaps that should be investigated by future research. First, there is a need to assess the potential effects of efforts to reduce emissions on the system and pathways to a low-carbon healthcare system. Second, it should be examined errors of aggregation when using the I–O methodology in the healthcare context. Third, the differences in the outcomes when making different methodological choices (SRIO or MRIO, systemic boundaries, etc) should be analysed to guide future research.

The transparency checklist used in this study can serve as an initial reference point for future developments. For example, in the checklist’s current state, all criteria are weighted equally. However, some might be less crucial to delivering harmonised study findings. An extended consensus process with further experts is proposed to validate the checklist further and increase its value for research and practice.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Acknowledgments.

We would like to thank Frauke Waßmuth for her help in the screening and extracting phase of the search update.

• Kronenberg T
• Wiedmann T ,
• Wood R , et al
• McKenzie JE ,
• Bossuyt PM , et al
• WBCSD and WRI
• Heijungs R ,
• Tennison I ,
• Roschnik S ,
• Ashby B , et al
• Malik A , et al
• Eckelman MJ ,
• Lagasse R , et al
• Sherman JD ,
• MacNeill AJ
• Health Facilities Scotland
• McAlister S , et al
• Pichler P-P ,
• Jaccard IS , et al
• Steenmeijer MA ,
• Rodrigues JFD ,
• Zijp MC , et al
• Carter S , et al
• Jaccard IS ,
• Weisz U , et al
• Karliner J ,
• Slotterback S ,
• Boyd R , et al
• Li M , et al
• du Plessis M , et al
• Husereau D ,
• Drummond M ,
• Augustovski F , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2
• Data supplement 3
• Data supplement 4
• Data supplement 5
• Data supplement 6

X @MattisKeil

Contributors MKeil (guarantor): methodology, screening, formal analysis, writing – original draft, writing – review and editing, visualisation; LF, MH, MKnieß: methodology, screening, formal analysis, writing – original draft; OL: conceptualisation, methodology, writing – review and editing; TK: methodology, writing – review and editing. WR: conceptualising, methodology, writing – review and editing, supervision, project administration. All authors have read and approved the final manuscript for publication.

Funding This work was supported by the Leibniz ScienceCampus Bremen Digital Public Health (lsc-diph.de), which is jointly funded by the Leibniz Association (W4/2018), the Federal State of Bremen, and the Leibniz Institute for Prevention Research and Epidemiology (BIPS) for OL’s inputs to the research project. The funder had no role in the study design, the collection, analysis, interpretation or submission of the data. MKeil’s inputs were supported by the research cluster 'Health City Bremen' which is funded by the Federal State of Bremen. The funder had no role in the study design, the collection, analysis, interpretation or submission of the data.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

• Search Menu
• Advance Articles
• Themed Collections
• Supplements
• Author Guidelines
• Submission Site
• Open Access Options
• Self-Archiving Policy
• Call for Papers
• About International Journal of Pharmacy Practice
• About the Royal Pharmaceutical Society
• Editorial Board
• Advertising & Corporate Services
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

• < Previous

Reflexivity in pharmacy practice qualitative research: systematic review of twelve peer-reviewed journals

• Article contents
• Figures & tables
• Supplementary Data

J Tomlinson, K Medlinskiene, Reflexivity in pharmacy practice qualitative research: systematic review of twelve peer-reviewed journals, International Journal of Pharmacy Practice , Volume 32, Issue Supplement_1, April 2024, Pages i5–i6, https://doi.org/10.1093/ijpp/riae013.007

• Permissions Icon Permissions

Reflexivity is named as an important component for establishing trustworthiness and authenticity within qualitative research. Depending on the philosophical stance of the researcher, subjectivity is often intertwined with qualitative research, thus engagement in reflexivity accounts for how this subjectivity shapes the study and aids to increase its rigor. Walsh’s typology [1] proposed personal, interpersonal, methodological, and contextual dimensions of reflexivity, which are overlapping and interacting, and was endorsed by the International Association for Medical Education (AMEE). [2] As pharmacy professionals conducting research, we bring many aspects (personal and professional experiences, beliefs, values) that can help us develop deeper and richer insights into our work; if not considered through reflexivity, this may have unrealised impacts which can bring the findings under scrutiny.

This systematic review aimed to explore reflexivity reporting, type, and strategies in qualitative studies published in peer-reviewed pharmacy journals.

Two reviewers screened twelve pharmacy related peer-reviewed journals from inception to 1 st August 2023 for qualitative studies; these were identified as the ‘top’ pharmacy practice journals in JANE (Journal/Author Name Estimator). Then, studies were independently reviewed against inclusion criteria. Any disagreements were discussed to reach a consensus. Qualitative studies discussing reflexivity, having dedicated reflexivity sections in the manuscript, or providing quality checklists with reflexivity were included. Quantitative, mixed methods, and educational research studies were excluded. Two reviewers completed data extraction: reflexivity methods, where discussed in the paper and to what extent, and type of reflexivity as per Walsh’s typology to determine the dimension of reflexivity. [1] Data were synthesised narratively. Quality appraisal of study conduct was not completed and all eligible papers were included.

452 qualitative studies were reviewed and 102 were included from seven journals. Thirty-four studies provided quality checklists and 18 had personal ( n= 10), interpersonal ( n= 2), methodological ( n= 1), or multi-type ( n= 5) reflexivity. In the remaining 68 articles, reflexivity was personal ( n= 20), interpersonal ( n= 3), methodological ( n= 16), multi-type ( n= 28), or unclear ( n= 1). Where reflexivity was presented, it was mostly reported in the manuscript’s methods section or within limitations sections of discussions; some had reflexivity named sections. However, reviewed studies lacked true reflexive comments, and often descriptions of research team backgrounds were provided without consideration of the impact on the study, or the focus was on minimising study bias. Twelve studies alluded to the use of reflexive diaries, journals, or note sheets but lacked detail in how they were implemented.

This is the first systematic review, to our best knowledge, focused on reflexivity reporting in pharmacy practice qualitative studies. It highlighted that reflexivity reporting was poor, often misunderstood, and use of the quality checklist did not increase appropriate reporting. Reflexivity was often described as something researchers ‘did’ to minimise bias, rather than something to embrace and explore as part of qualitative research process. We posit that this could be due to a lack of understanding of what true reflexivity is, unclear quality checklist instructions and limited potential to include it meaningfully in the final manuscript. The review was limited to twelve pharmacy practice journals and articles published in English.

1. Walsh R. The methods of reflexivity. Humanist Psychol. 2003; 31(4): 51–66.

2. Olmos-Vega FM, Stalmeijer RE, Varpio L et al . A practical guide to reflexivity in qualitative research: AMEE Guide No. 149. Medical Teacher 2023; 45(3): 241-251.

Email alerts

Citing articles via.

• Recommend to Your Librarian
• Advertising and Corporate Service
• Journals Career Network

Affiliations

• Online ISSN 2042-7174
• Print ISSN 0961-7671
• Copyright © 2024 Royal Pharmaceutical Society
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Open access
• Published: 26 April 2024

A Systematic review of the factors that affect soccer players’ short-passing ability—based on the Loughborough Soccer Passing Test

• Bihan Wang 1   na1 ,
• Bin Wan 1   na1 ,
• Shu Chen 1 ,
• Yu Zhang 1 ,
• Xiaorong Bai 2 ,
• Wensheng Xiao 2 ,
• Changfa Tang 1 &
• Bo Long 1  

BMC Sports Science, Medicine and Rehabilitation volume  16 , Article number:  96 ( 2024 ) Cite this article

66 Accesses

Metrics details

This study synthesizes evidence from the Loughborough Passing Test to evaluate the short-passing ability of soccer players and summarizes the reported variables that affect this ability to provide support for the development and improvement of short-passing abilities in soccer players.

In this systematic review using the PRISMA guidelines, a comprehensive search was conducted in Web of Science, PubMed, and EBSCOhost from inception to July 2023 to identify relevant articles from the accessible literature. Only studies that used the Loughborough test to assess athletes' short-passing ability were included. The quality of the included studies was independently assessed by two reviewers using the PEDro scale, and two authors independently completed the data extraction.

Based on the type of intervention or influencing factor, ten studies investigated training, nine studies investigated fatigue, nine studies investigated supplement intake, and five studies investigated other factors.

Evidence indicates that fitness training, small-sided games training, and warm-up training have positive effects on athletes' short-passing ability, high-intensity special-position training and water intake have no discernible impact, mental and muscular exhaustion have a significantly negative effect, and the effect of nutritional ergogenic aid intake is not yet clear. Future research should examine more elements that can affect soccer players' short-passing ability.

Trial registration

https://inplasy.com/ ., identifier: INPLASY20237.

Peer Review reports

Introduction

Soccer is a game of skills and strategy, and one of the most crucial techniques is short passing [ 1 , 2 , 3 ]. A player's ability to make short passes is important for the team to initiate offense and control the pace of the game. Soccer players can more effectively control the game by strategically use their short passing ability. Making multiple quick, short passes in succession can speed up the game, complete the attacking strategy, and increase pressure on the defence of the opposition, which can provide scoring opportunities [ 4 ]. According to a study, most goals are preceded by short passes [ 5 ].

Players who use short-passing techniques in the game must decide on the pass's timing, strength, and direction under time and space constraints based on the placement of teammates and opponents on the field. However, the conventional short-passing ability assessment employs a single short-passing ability test. The most striking feature of this type of test is that it is performed in a relatively static environment with a short pass to a target or teammate at a known distance and direction; therefore, only motion patterns are shown throughout the test, and it has limited ecological validity [ 6 , 7 , 8 , 9 ]. This type of test cannot be used to effectively evaluate the short-pass technique of athletes with different levels of competitive ability [ 10 , 11 , 12 ]. In contrast to conventional short-passing ability tests, the Loughborough Soccer Passing Test (LSPT), as shown in Fig.  1 , as a multitask test, has advantages in the evaluation of athletes' short passes: it requires participants to remember the relative orientation of the target, process oral information for quick decision-making, squelch potential errors, and make flexible cognitive transitions while using their short-passing ability. As a result, the LSPT is consistent with the shifting circumstances of soccer matches [ 12 , 13 ]. The LSPT procedure is manageable and requires subjects to pass the ball 16 times while surrounded by a rectangular bench. Each bench has a colourful metal strip or coloured cardboard (0.6 × 0.3 m) that can be utilized as a target area to make an effective pass in one of four randomly selected colour sequences. Subjects must complete 16 brief passes of the test as quickly and accurately as they can. Time-related metrics are used to define LSPT scores, including execution time (the amount of time needed to complete 16 passes), penalty time (the amount of time added for mistakes, including incorrect passes and sluggish performance), and total time (execution time plus penalty time). All LSPT time values are inversely correlated with a player's short-passing ability in soccer (a player with a lower LSPT time value has greater short-passing ability). The LSPT is currently used in research on athlete selection in Australia [ 14 ], the Netherlands [ 15 ], and France [ 16 ]. Several studies have shown that the LSPT has good retest reliability and good discriminant validity for players of different sport levels, ages, and genders [ 12 , 17 ].

Layout of the LSPT [ 12 , 18 ]

Regrettably, despite its importance, there is no systematic review of short-passing abilities or the factors that influence them. Systematic reviews of soccer skills have been conducted on most or all skills or overall athletic performance [ 19 , 20 , 21 ], but there is a lack of systematic reviews of specific soccer skills. Due to the importance of short-passing abilities, there is a great need for a more comprehensive analysis of the research on soccer players' short-passing abilities to statistically synthesize the various findings and to examine the factors that affect soccer players' short-passing abilities. The purpose of this paper is to review and analyse research on the factors that affect the short-passing ability of soccer players to contribute to improvements in soccer players’ short-passing ability.

This systematic review used the PRISMA guidelines [ 22 ] and was registered in the International Platform for Registered Programs for Systematic Reviews and Meta-Analyses (INPLASY); https://inplasy.com , INPLASY202370041.

Search strategy

A comprehensive, electronic search of the literature was conducted without data restrictions in Web of Science, PubMed, and EBSCOhost on July 10, 2023, using a search strategy developed by two authors (WBH and XWS). The keyword combinations used were: (("Pass" OR "Skill" OR "Technology" OR "Technique" OR "Art" OR "Performance" OR "Ability" OR "Capacity") AND ("Soccer" OR "Football") AND ("LSPT" OR "Loughborough Soccer Passing Test")). Additionally, the researchers explored Google Scholar and the reference lists of the included studies for potential papers that could meet the inclusion criteria for additional related citations.

Eligibility criteria

The overall, intervention, comparison, outcome, and study design (PICOS) criteria were the inclusion criteria for this study, as detailed in Table 1 . Studies were included if they met the following requirements: 1. football players were the subjects of the study; 2. the paper must include at least one study that aimed to assess the effect of a factor or an intervention on the short-passing ability of soccer players; 3. the method used to assess the short-passing ability of the subjects of the study must have been LSPT. Regardless of the factor that influences a soccer player's short-passing ability, any study that met the above three requirements was included in this systematic review.

Studies that met the following criteria were excluded: 1. conferences, overviews, newsletters, book reviews, and studies that were not supported by data and were not analysed statistically; 2. studies that did not quantitatively evaluate the short-passing abilities of the subjects or evaluated them without using the LSPT; 3. studies that did not apply to the vast majority of soccer players, such as the effect of a particular religious practice on the short-passing ability of a soccer player of that religion or the effect of a certain factor on the short-passing ability of a soccer player with a disability.

Study selection

The following procedure was used to choose the papers. First, prior to importing the studies into EndNote X9 to check for duplication, an experienced librarian assisted with the search strategy by putting key phrases into the three major databases to search for articles. Second, to find pertinent research, two independent reviewers (WBH and XWS) examined the titles and abstracts of all identified papers in accordance with the inclusion and exclusion criteria of the study.

Data extraction

Two independent reviewers (WBH and XWS) completed the data extraction. Any disputes were explored further. When necessary, a third reviewer (BXR) participated until consensus was reached. The records included (1) the author and year of publication; (2) the study design; (3) participant characteristics, namely, age, sex, and athletic level; (4) the characteristics of the intervention; and (5) the final research outcomes.

Quality assessment

Two authors (WBH and XWS) independently utilized the PEDro scale, with disagreements resolved by a third rater (BXR). The eligibility criteria in the scale were not included in the total score, as they were related to external validity. The total PEDro score ranges from 0 to 10. The higher the score, the better the methodological quality. A score of 8 to 10 indicates a study of excellent methodological quality, 5 to 7 is considered to indicate good quality, 3 to 4 is considered a study of average quality, and values less than 3 points are considered to indicate fair quality. A score lower than 3 is considered a poor-quality study [ 23 ].

As shown in Fig. 2 , the electronic search of the relevant databases yielded 147 potentially relevant articles (54 from Web of Science, 30 from PubMed, and 63 from EBSCOhost), while an additional five studies were found through Google Scholar and references. The titles and abstracts of 65 publications were evaluated for conformity after duplicates were eliminated ( n = 87). After 17 items were deleted at the title and abstract levels, the remaining 48 articles were read. Following this reading, an additional 15 publications were excluded, and 33 studies that met all the inclusion criteria for the systematic review were retained. The characteristics of the included studies are detailed in Table 2 .

PRISMA flow chart of the study selection process

Demographic characteristics

The pertinent details of the studies are presented in Table 2 . The age of the players ranged from 8 to 25.5 ± 5.2 years. With regard to the players’ gender, most of the studies reported male players, two studies examined female players [ 41 , 42 ], and only one study reported on male and female players [ 52 ].

Intervention characteristics

For ease of generalization and induction, other factors included motivation, soccer field, verbal interaction, visual observation, and salbutamol intake; these are presented in Fig. 3 . Of the 33 included studies, 24 were one-time intervention studies and 9 were long-term intervention studies. Interventions/influencing factors included training ( n = 10), fatigue ( n = 9), supplement intake ( n = 9), and other factors ( n = 5). The ten papers on the influence of training on football players' short-passing ability included fitness training ( n = 4), small-sided games training ( n = 2), warm-up training ( n = 3) and high-intensity position-specific training ( n = 1). The 9 papers on the effect of fatigue on short-passing ability in soccer players included mental fatigue ( n = 5) and muscle fatigue ( n = 4). The 9 papers on the effect of supplement intake on short-passing ability in soccer players included water intake ( n = 2) and nutritional ergogenic aid intake ( n = 7). The five papers on other factors that influence football players' short-passing ability included motivation ( n = 1), verbal interaction ( n = 1), football field ( n = 1), visual observation ( n = 1) and salbutamol intake ( n = 1).

Influence Factor Chart of Short-passing Skill of Soccer Players

Among the studies ( n = 10) on the effects of training on short-passing ability in soccer players, with the exception of two one-time intervention studies [ 30 , 32 ], 1) all studies explicitly reported the total duration of the intervention, with the shortest being 5 days [ 31 ] and the longest being 22 weeks [ 25 ]; 2) most of the studies explicitly reported the duration of each intervention, with the shortest being 16 minutes [ 33 ] and the longest being 98 minutes [ 25 ] and only two studies failing to explicitly report the duration of each intervention [ 27 , 31 ]; 3) all studies explicitly reported the frequency of intervention, which was once a day in one study [ 31 ], twice a week in three studies [ 28 , 29 , 33 ], three times a week in two studies [ 26 , 27 ], between 2 times a week and 4 times a week in one study [ 25 ], and 2 times in the first week and 3 times in weeks 2 to 4 in one study [ 24 ]. In the studies in which fatigue affected the short-passing ability of soccer players ( n = 9), all reported in detail the intervention protocols used. In terms of mental fatigue, four studies used the Stroop task [ 18 , 34 , 35 , 37 ], one study used Brain It On software [ 36 ], one study used the LSPT random order and clockwise order tasks in addition to the Stroop task [ 18 ], and four studies performed muscle training [ 38 ], soccer matches [ 39 ], high-intensity interval training [ 40 ], and resistance training [ 41 ]. In studies on the effects of supplement intake on short-passing ability in soccer players ( n = 9), 1) all studies reported the intake dose of supplements; 2) all studies explicitly reported the type of supplement ingested, including water, carbohydrate solution, caffeine solution, and carbohydrate caffeine solution (i.e., carbohydrate solution mixed with caffeine solution). In two studies only water was ingested [ 42 , 43 ], in three studies only carbohydrate solutions were ingested [ 44 , 45 , 48 ], two studies used only caffeine solution [ 46 , 50 ], carbohydrate solutions and carbohydrate caffeine solutions were ingested in one study [ 47 ], and carbohydrate solution, caffeine solution, and carbohydrate caffeine solution were ingested in one study [ 49 ]. All other studies of factors that affect short-passing ability in soccer players ( n = 5) provided a clear description or explicit definition of the substance or method of intervention.

Study quality assessment

The quality of the studies is presented in Table 3 . The PEDro checklist was used to assess the quality of the included studies. The results showed that eight studies received a score of 3 or 4, indicating average quality, and 18 studies scored 5 to 7 points, which was considered good quality. Moreover, seven studies had scores ranging from 8 to 10 points and were considered to have excellent methodological quality.

Outcome and measures

The results of the current study were divided into groups based on the various interventions and influencing factors that were found to have an impact on soccer players' short-passing ability.

The effect of training on the short-passing ability of soccer players

Fitness training.

Four studies examined the impact of fitness training on soccer players' short-passing abilities [ 24 , 25 , 26 , 27 ]. The fitness training methods included aerobic interval training [ 24 ], skill combined with agility training [ 25 ], balance training [ 26 ], and strength combined with endurance training [ 27 ]. The subjects included amateur players [ 26 ], youth players [ 24 ], players with five years of experience [ 27 ], and regional sub-elite players [ 25 ]. The results of these studies demonstrate that fitness training improves soccer players' short-passing abilities and is more effective than the training methods used in the control groups of the respective studies.

Small-sided games training

This review comprises two studies that examined the impact of small-sided games training on soccer players' short-passing abilities [ 28 , 29 ]. The participants included amateur players [ 29 ] and professional players [ 28 ]. Both studies found that small-field match training improved short-passing ability in soccer players and demonstrated that small-field match training was more effective than repetitive sprint training and conventional aerobic interval training, respectively, which were used by their control groups.

Warm-up training

The influence of warm-up training on soccer players' short-passing abilities was examined in three studies [ 30 , 31 , 32 ]. One of these studies examined pre-match warm-up training, while the other two explored halftime rewarm-up training. These studies used four warm-up training methods, including passing warm-up training [ 30 ], foam axle rolling training [ 32 ], leg press training, and small-sided games training [ 30 ]. The participants included non-elite players [ 31 ] and professional players [ 30 , 32 ]. Of the four training methods, foam axle rolling training [ 32 ] and leg press training [ 30 ] performed during halftime did not significantly affect players’ short-passing ability, while the remaining two warm-up training methods positively affected players’ short-passing ability [ 30 , 31 ].

High-intensity special-position training

Only one study included in this systematic review presented inferences about the effect of high-intensity special position training on soccer players' short-passing abilities [ 33 ]. The participants in this study were national youth events and professional soccer training services. This study revealed no improvement in short-passing ability after high-intensity special-position training [ 33 ].

The effect of fatigue on the short-passing ability of soccer players

Mental fatigue.

This review included five studies that examined the impact of mental fatigue on soccer players' short-passing abilities [ 18 , 34 , 35 , 36 , 37 ]. The participants included youth players [ 36 ], trained players [ 18 ], players competing at the national level [ 37 ], and professional players [ 34 , 35 ]. These five studies revealed a significant negative impact of mental weariness on soccer players' short-passing abilities.

Muscle fatigue

This systematic review comprised four studies that examined how soccer players' short-passing abilities were affected by muscular exhaustion [ 38 , 39 , 40 , 41 ]. Importantly, two of the studies provided indirect confirmation rather than directly investigating how muscular exhaustion affects soccer players' short-passing abilities [ 40 , 41 ]. The participants included college soccer players [ 38 ], high-level competition players [ 40 ], professional elites, sub-elite players [ 41 ], and professional footballers [ 39 ]. These four studies demonstrated that muscle exhaustion can significantly impair soccer players' short-passing abilities.

The effect of supplement intake on short-passing ability in soccer players

Water intake.

This review included two trials that examined the impact of water intake on soccer players' short-passing abilities [ 42 , 43 ]. The participants included semi-professional players [ 43 ] and professional players [ 42 ]. The intake of water had no discernible impact on players' ability to produce short passes in both experiments.

Nutrition ergogenic aid intake

This review comprised seven trials to confirm the impact of nutrition ergogenic aid intake use on football players' short passing ability [ 44 , 45 , 46 , 47 , 48 , 49 , 50 ]. The subjects included semi-professional players, ex-professional players or players who had reached at least college 1st/2nd team standards [ 44 ], semi-professional or non-professional players from college teams [ 45 ], regional top league players [ 46 ], class players [ 47 ], college players [ 48 , 49 ], and casual players [ 50 ]. Only two studies reported a significant positive effect on players' short-passing ability when they ingested a carbohydrate solution [ 48 ] or a caffeine solution [ 46 ]. The results of the remaining five studies indicated that the ingestion of a carbohydrate solution, a caffeine solution, or a carbohydrate caffeine solution did not have a significant effect on players' short-passing ability [ 44 , 45 , 47 , 49 , 50 ].

This review included five studies that examined additional variables that influenced soccer players' short-passing abilities [ 51 , 52 , 53 , 54 , 55 ]. The participants included amateur players [ 51 ], amateur student players [ 52 ], top players in school soccer games [ 53 ], soccer academy high-level players, soccer academy low-level players [ 54 ] and professional junior soccer players [ 55 ]. In five studies, motivation [ 51 ] and verbal interaction [ 52 ] were reported to positively influence players' short-passing ability. O’Meagher et al. (2022) reported no significant difference in players' short-passing ability between grass and artificial turf. One study reported the important effect of visual observation on players' short-passing ability [ 54 ]. Another study showed that salbutamol intake did not have a significant effect on players' short-passing ability [ 55 ].

The growth and performance of soccer players’ technical and tactical skills depend on their level of fitness. The four studies that examined how short-passing abilities in soccer players were affected by fitness training all concluded that players could benefit from the training techniques used in their studies, which included aerobic interval training [ 24 ], strength and endurance training [ 27 ], skill and agility training [ 25 ], and balance training [ 26 ]. This means that a player's short-passing ability benefits not only from technical training but also from fitness training. The training techniques employed in these studies involve only a portion of the fitness training approach, including endurance training, balance training, and strength training. Some studies support the findings of earlier research that showed that fitness training can enhance athletes' abilities [ 56 , 57 , 58 ]. In fact, the same rationale that supports the positive effects of fitness training on athletes' specialized skills in other sports likely applies to the short-passing abilities of soccer players. In addition to athletes’ mastery of the technique itself, athletes’ physical attributes are crucial to the use of the skill. For instance, an athlete's balance directly influences the mass of the short passing, which is a dynamic unilateral technical movement [ 59 , 60 ], especially when a game-time physical altercation with the opponent occurs. Future studies should examine the effects of various fitness training programmes on soccer players' short-passing abilities.

Soccer training for small-sided games is referred to as skill-based match training [ 61 ] or match-based training [ 62 ] and is typically played on a smaller pitch. According to the two included studies on the impact of small-sided games training on soccer players' short-passing ability [ 28 , 29 ], small-sided games training considerably enhances players' short-passing ability. Small-sided games training simulates the athletic demands, physiological intensity, and technical requirements of a soccer game. Compared with traditional short-passing practice (e.g., one-on-one passing, multiple passes to each other), small-sided games training forces players to use short passes more frequently under increased defensive pressure and reduced field size due to the limitations of the rules. In other words, small-sided games training allows players more opportunities to use and practice short passes under time and space pressure [ 62 , 63 ]. This may also explain why small-sided games training improves soccer players' short-passing ability more significantly than traditional short-passing training or other training methods. This means that coaches and players can use small-sided games training drills to improve short passes in real scenarios that are more similar to games.

Soccer training before a game is essential. In recent years, researchers have examined various warm-up training strategies, such as rewarming up during the game's halftime break and conventional pregame warm-up training, as the methods and means of warm-up training have become more varied. In comparison to a ball size of five, Burcak's (2015) study found that pre-match warm-up training with a ball size of four had a positive effect on players' short-passing ability. In a randomized crossover experiment, Zois et al. (2013) discovered that practising for a small-sided game during halftime increased players' short-passing ability. In contrast, the halftime leg press drill had little impact on players' short-passing ability. Similarly, randomized crossover research by Kaya et al. (2021) revealed that halftime foam-axis rolling drills had no positive impact on players' short-passing ability [ 64 , 65 ]. Nevertheless, due to the lower intensity, foam-axis rolling training and leg press training during halftime tend to reduce muscle temperature in athletes who have recently concluded a game's first half. Based on these findings, athletes may decide to maintain their muscular temperature by engaging in rewarming exercises during halftime. However, it is crucial to remember that each player must be evaluated individually. If a player is extremely exhausted at halftime, rewarming up for training may worsen his or her short passing ability and athletic performance.

Soccer players who engage in high-intensity position-specific training practise the skill most pertinent to their position at a high level (90% HRmax) [ 33 ]. Compared to the impact of small-sided games training on soccer players' short-passing abilities, this produces the opposite outcome. Due to the limitations of the field size, small-sided games training may offer more possibilities for practising short-passing techniques. High-intensity position-specific training, in contrast, includes many additional elements and requires less time to improve short-passing ability. This indicates that high-intensity special-position training is used by coaches and players to enhance short passing, which is an unwise choice.

According to one definition, mental tiredness is a psychobiological condition marked by feelings of exhaustion that can occur during or after prolonged periods of perceived exertion [ 66 , 67 ]. The five studies in this paper on the effect of psychological exhaustion on soccer players' short-passing abilities all concluded that psychological exhaustion may be detrimental to these abilities [ 18 , 34 , 35 , 36 , 37 ]. This suggests that coaches and players should pay increased attention to this easily overlooked factor that affects short-passing ability. According to Filipas et al. (2021), mental fatigue has a significant negative impact on U18 players' short-passing abilities as well as a negative, albeit nonsignificant, impact on U14 and U16 players' short-passing abilities; total LSPT times are 7.4% (U14) and 4.2% (U16) greater than the control group. Smith et al. (2017) also performed more thorough statistical analyses using the LSPT penalty time rule. In contrast to players who are not mentally weary, mentally fatigued players targeted errors substantially more and completed passes significantly less frequently. These findings confirm earlier studies suggesting that mental weariness impairs athletes' performance [ 66 , 68 , 69 ]. According to some research, mental weariness can impair a player's ability to concentrate, lengthen reaction times for cognitive activities, and increase a player's risk of making mistakes when using short-passing techniques [ 70 ]. When using short-passing abilities in soccer, players must maintain a high degree of focus and accurate perception to allow them to make the right choices in a highly dynamic environment and under time and space pressures. As a result, athletes should try to prevent developing premature mental tiredness. Cognitive tasks that require considerable energy typically lead to mental weariness [ 71 ]. Therefore, to avoid premature mental fatigue, players should be wary of high levels of pregame cognitive demands (e.g., excessive use of cell phones, tablets, and video games, as well as prolonged cognitive skill training).

Short inter-match recovery times (halftime) and high neuromuscular demands during soccer matches may result in muscle fatigue during the game, decreasing players' abilities and fitness, which may have an impact on match performance [ 72 ]. Researchers of soccer have paid close attention to the impact of muscular fatigue on short-passing ability, one of the skills most often employed by players in games. Two studies directly reported significant negative effects of muscle fatigue on soccer players' short-passing ability [ 38 , 39 ], and two other studies provided indirect support that short-passing ability can have significant negative effects on soccer players. After high-intensity interval training, Draganidis et al. (2013) reported that professional sub-elite players' short-passing abilities deteriorated, and Lyons et al. (2021) found that high-level players' short-passing abilities deteriorated. These findings provide circumstantial evidence that players with short-passing abilities can suffer from muscle exhaustion resulting from persistent dynamic exercise and resistance training [ 73 , 74 ].

These results are consistent with those from earlier investigations. In fact, numerous studies have documented losses in athletic ability and performance that occur as players approach a state of muscular tiredness [ 75 , 76 ], and one study reported that after fatigue training, a considerable drop occurred in shooting scores [ 77 ]. Soccer players’ short-passing abilities might suffer from muscle exhaustion, perhaps as a result of a reduction in muscle functioning capacity [ 78 , 79 ], which decreases the stability and accuracy of a player's passes. The decrease in players' short-passing ability caused by completing short bursts of high-intensity activity at the same absolute workload is also related to players' physical quality [ 39 ]. Therefore, in actual daily training, to prevent premature muscle fatigue from impairing short-passing ability in play, players should enhance their physical training and practice.

The effect of supplement intake on the short-passing ability of soccer players

During a game, soccer players exert both mental and physical effort. Under extreme physical and mental strain, the body is susceptible to water loss and mental exhaustion. Reduced endurance and cognitive function can result from dehydration in athletes with up to 2% body weight loss during exercise [ 80 , 81 ]. Two variables may cause a player's ability to pass short passes to gradually deteriorate throughout sports: dehydration and inadequate water intake. To keep players' short-passing ability or slow down its decline, several studies have tried feeding them a specific volume of water. However, neither of the experiments presented in this study revealed a significant impact of water intake on players' short-passing abilities [ 42 , 43 ]. These findings suggest that soccer players cannot rely on drinking water during a game to prevent a decrease in short-passing ability.

Nutritional ergogenic aid intake

The nutritional ergogenic aid intake is anything that enhances athletic performance. It can be a nutrient, a nutrient metabolite, a food extract (from a plant), or something that is typically present in other items (i.e., caffeine or carbohydrates) [ 82 ]. Carbohydrate solution, caffeine solution, and carbohydrate and caffeine solution were utilized in the seven studies that examined the impact of the nutritional ergogenic aid intake on soccer players' short-passing abilities. O'Reilly et al. (2013) and Foskett et al. (2009) reported that ingesting a carbohydrate solution or a caffeine solution significantly improved players' short-passing ability. The other five studies found no evidence that ingesting a carbohydrate solution, a caffeine solution, or a carbohydrate and caffeine solution significantly improved players' short-passing abilities [ 44 , 45 , 47 , 49 , 50 ]. Three studies indicate that players' short pass ability is positively impacted by nutritional ergogenic supplement intake, but these findings also indicate that this relationship is not statistically significant [ 44 , 45 , 50 ]. Therefore, the impact of nutritional ergogenic aid intake on soccer players' short-passing abilities is unclear and requires additional explanation. In fact, the effects of nutritional ergogenic aid intake on athletes' skills have been similarly ambiguous in other investigations. For instance, Stuart et al. (2005) [ 83 ] reported that rugby players who swallowed a caffeine solution had a 10% increase in passing accuracy on the exam. However, rugby players took the same dose of caffeine solution in the study by Assi and Bottoms (2014), and the findings revealed no appreciable impact on test-passing accuracy [ 84 ]. Belenky et al. (2005) [ 85 ] claimed that ingesting a caffeinated solution enhanced shooting ability, although other studies have demonstrated that doing so did not significantly enhance this ability [ 86 , 87 ]. Therefore, football players are advised to not employ nutritional ergogenic aid intake to maintain their short-passing abilities or to halt their decline. Future research should confirm these findings with additional randomized, double-blind crossover experiments. Future research is necessary to determine the potential impact of other commonly used nutritional ergogenic aids, such as creatine, L-carnitine, protein, and amino acid supplements, on football players' short pass ability.

Other factors affecting short-passing ability in soccer players

The five studies that were evaluated in this section of the paper examined five underappreciated or overlooked factors that may affect soccer players' short-passing abilities [ 51 , 52 , 53 , 54 , 55 ]. Barte et al. (2019) reported that various methods of motivating worn-out players while at rest improve athletes' short-passing abilities. This suggests that motivating players makes practical sense for improving short passes, which provides support for coaches who are accustomed to motivating players. Khalifa et al. (2020) suggested that talking to teammates during halftime can improve short-passing abilities (10.2% reduction in overall LSPT time) and can outperform passive rest (4.2% reduction in total LSPT time).

Players' short-passing abilities did not differ significantly between grass and artificial turf, according to research by Meagher et al. (2022) Players should therefore not worry about the effect that being on two different types of turf may have on their short-passing ability. However, indoor springy wood flooring considerably improved players' short-passing abilities over grass and synthetic turf. Nevertheless, indoor 5-a-side soccer games are usually the only tournaments played on indoor resilient wood floors. Visual observation, as shown by Vansteenkiste et al. (2022), has a significant impact on players' short-passing ability. When players utilize short-passing abilities, spending too much time focusing on the ball might prevent them from seeing teammates' and defenders' locations, which can cause them to miss the ideal opportunity to pass the ball or lose possession. This suggests that players should be more observant of the ever-changing conditions on the field rather than just staring at the ball in the ratios. Coaches must be aware of this key point, which can be easily overlooked, and remind players of it during training, and players must recognize it themselves. Additionally, a study revealed that taking salbutamol had no discernible impact on players' short-passing abilities [ 55 ].

Limitations

This study systematically evaluated the factors that affect soccer players' short-passing ability. The results showed that these factors can be divided into positive and negative categories. This study provides a reference and support for soccer coaches and players to improve their short-passing abilities. However, there are a few limitations to this review. 1) Papers in languages other than English were excluded from the study, which influenced the selection of papers. 2) Because it is unknown whether the participants' sex, age, and level of sport affected the intervention effects of some research, the pertinent conclusions should not be extended without due care. 3) Despite the advantages of the LSPT over the conventional short-passing ability test, the LSPT cannot accurately imitate the intricacies of soccer players' use of the short-passing technique in games. 4) The present results should be applied with caution due to the lack of research on some of the influencing elements included in this study, which could affect the accuracy of some of the conclusions. Nevertheless, we believe that the current study can aid in the development and improvement of short-passing abilities in soccer because it examines some relevant strategies and elements.

Conclusions

This study's findings indicate that a variety of factors can influence soccer players' short-passing abilities. For example, in terms of the effect of training on football players' short-passing abilities, fitness training, small-sided games training, and some warm-up training positively impact these abilities, while high-intensity special-position training has no discernible impact. Mental and muscular exhaustion have a significantly negative effect. In terms of the effect of supplemental intake on football players’ short-passing ability, water intake has no significant effect, and the effect of nutritional ergogenic aid intake is not yet clear. Based on these findings, additional research is encouraged to investigate techniques or variables that affect short-passing ability in soccer players, such as additional training methods (e.g., Specialized short-passing ability training and functional training) and players' own factors (e.g., sleep and mood). However, whether the results of this study apply to all soccer players of all ages, sexes, and athletic levels is unknown. Future research should focus on determining whether a specific subset of the findings is appropriate for a particular group of soccer players. In addition, this study offers only a general directional reference for the sustainable development and improvement of soccer players' short-passing ability.

Availability of data and materials

Data are available on request to the corresponding author by e-mail ([email protected] OR [email protected] OR [email protected]), and registered in the International Platform for Registered Programs for Systematic Reviews and Meta-Analyses (INPLASY); https://inplasy.com , INPLASY202370041.

Liu H, Gomez M-Á, Lago-Peñas C, Sampaio J. Match statistics related to winning in the group stage of 2014 Brazil FIFA World Cup. J Sports Sci. 2015;33(12):1205-13.

Bloomfield J, Polman R, O’Donoghue P. Physical demands of different positions in FA Premier League soccer. J Sports Sci Med. 2007;6(1):63.

PubMed   PubMed Central   Google Scholar  

Rampinini E, Impellizzeri FM, Castagna C, Coutts AJ, Wisløff U. Technical performance during soccer matches of the Italian Serie A league: Effect of fatigue and competitive level. J Sci Med Sport. 2009;12(1):227–33.

Article   PubMed   Google Scholar  

Konefal M, Chmura P, Zacharko M, Chmura J, Rokita A, Andrzejewski M. Match outcome vs match status and frequency of selected technical activities of soccer players during UEFA Euro 2016. Int J Perform Anal Sport. 2018;18(4):568–81.

Article   Google Scholar  

Wright C, Atkins S, Polman R, Jones B, Sargeson L. Factors associated with goals and goal scoring opportunities in professional soccer. Int J Perform Anal Sport. 2011;11(3):438–49.

Haaland E, Hoff J. Non-dominant leg training improves the bilateral motor performance of soccer players. Scand J Med Sci Sports. 2003;13(3):179–84.

Article   CAS   PubMed   Google Scholar  

Hoare DG, Warr CR. Talent identification and women’s soccer: an Australian experience. J Sports Sci. 2000;18(9):751–8.

McMorris T, Gibbs C, Palmer J, Payne A, Torpey N. Exercise and performance of a motor skill. Res Supp. 1994;15:23–7.

Google Scholar  

Ali A. Measuring soccer skill performance: a review. Scand J Med Sci Sports. 2011;21(2):170-83.

Cheng JP. Research on the implementation effect of the secondary standard test item in the campus football skill rating standard. Master's thesis, Beijing Sport University, 2019.

Li J. New definition and theoretical exploration of the concept of football technology. J Chengdu Sport Univ. 2008;34(06):56-9.

Bian C, Yi Q, Li Y. Loughborough football passing test: methods, interpretation and prospects. Sports scientific research. 2023;44(01):77–84.

Ali A, Williams C, Hulse M, Strudwick A, Reddin J, Howart L, Eldred J, Hirst M, Mcgregor S. Reliability and validity of two tests of soccer skill. J Sports. 2007;25(13):1461–70.

Keller BS, Raynor AJ, Bruce L, Iredale F. Technical attributes of Australian youth soccer players: Implications for talent identification. Int J Sports Sci Coaching. 2016;11(6):819–24.

Huijgen BCH, Elferink-Gemser MT, Ali A, Visscher C. Soccer skill development in talented players. Int J Sports Med. 2013;34(8):720–6.

Le Moal E, Rue O, Ajmol A, Abderrahman AB, Hammami MA, Ounis OB, Kebsi W, Zouhal H. Validation of the Loughborough Soccer Passing Test in Young Soccer Players. J Strength Cond Res. 2014;28(5):1418–26.

Wen D, Robertson S, Hu G, Song B, Chen H. Measurement properties and feasibility of the Loughborough soccer passing test: A systematic review. J Sports Sci. 2018;36(15):1682–94.

Bian C, Ali A, Nassis GP, Li Y. Repeated interval loughborough soccer passing tests: An ecologically valid motor task to induce mental fatigue in soccer. Front Physiol. 2022;12: 803528.

Article   PubMed   PubMed Central   Google Scholar  

Dambroz F, Clemente FM, Teoldo I. The effect of physical fatigue on the performance of soccer players: a systematic review. PloS one. 2022;17(7):e0270099.

Pacheco MM, de Oliveira LMM, dos Santos CCA, Godoi Filho JRM, Drews R. Challenging traditions: Systematic review of practice, instruction, and motor skill acquisition in soccer. Int J Sports Sci Coaching. 2023;18(5):1702–25.

Zhang Y, Li D, Gómez-Ruano M-Á, Memmert D, Li C, Fu M. Effects of plyometric training on kicking performance in soccer players: A systematic review and meta-analysis. Frontiers in Physiology. 2023;14:1072798.

Moher D, Liberati A, Tetzlaff J, Altman DG, Grp P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8(5):336–41.

Foley NC, Teasell RW, Bhogal SK, Speechley MR. Stroke rehabilitation evidence-based review: methodology. Top Stroke Rehabil. 2003;10(1):1–7.

Impellizzeri FM, Rampinini E, Maffiuletti NA, Castagna C, Bizzini M, Wisløff U. Effects of aerobic training on the exercise-induced decline in short-passing ability in junior soccer players. Appl Physiol Nutr Metab. 2008;33(6):1192–8.

Zago M, Giuriola M, Sforza C. Effects of a combined technique and agility program on youth soccer players’ skills. Int J Sports Sci Coaching. 2016;11(5):710-20.

Cè E, Longo S, Paleari E, Riboli A, Limonta E, Rampichini S, Coratella G, Esposito F. Evidence of balance training-induced improvement in soccer-specific skills in U11 soccer players. Scand J Med Sci Sports. 2018;28(11):2443–56.

Tarakci S, Pinar S. Investigation of The Effect of Concurrent Training Strength and Endurance Training on Physical, Physiological and Psychological Parameters in Young Football Players. Pakistan J Med Health Sci. 2021;15(10):2945–50.

Eniseler N, Şahan Ç, Özcan I, Dinler K. High-intensity small-sided games versus repeated sprint training in junior soccer players. J Hum Kinet. 2017;60(1):101–11.

Özcan İ, Eniseler N, Şahan Ç. Effects of small-sided games and conventional aerobic interval training on various physiological characteristics and defensive and offensive skills used in soccer. Kinesiology. 2018;50(1):104–11.

Zois J, Bishop D, Fairweather I, Ball K, Aughey R: High-intensity re-warm-ups enhance soccer performance. International journal of sports medicine 2013:800-805.

Burcak K. The Effects on Soccer Passing Skills When Warming Up with Two Different Sized Soccer Balls. Educ Res Rev. 2015;10(22):2860–8.

Kaya S, Cug M, Behm DG. Foam rolling during a simulated half-time attenuates subsequent soccer-specific performance decrements. J Bodyw Mov Ther. 2021;26:193–200.

Le Cuong C, Ma’Ayah F, Nosaka K, Hiscock D, Latella C. Effects of high-intensity position-specific drills on physical and technical skill performance in elite youth soccer players. J Strength Cond Res. 2023;37(5):e332–40.

Smith MR, Coutts AJ, Merlini M, Deprez D, Lenoir M, Marcora SM. Mental fatigue impairs soccer-specific physical and technical performance. Med Sci Sports Exerc. 2016;48(2):267–76.

Smith MR, Fransen J, Deprez D, Lenoir M, Coutts AJ. Impact of mental fatigue on speed and accuracy components of soccer-specific skills. Sci Med Footb. 2017;1(1):48–52.

Greco G, Roberto T, Ambruosi P, Fischetti F. Negative effects of smartphone use on physical and technical performance of young footballers. J Phys Educ Sport. 2017;17(4):2495–501.

Filipas L, Borghi S, La Torre A, Smith MR. Effects of mental fatigue on soccer-specific performance in young players. Sci Med Footb. 2021;5(2):150–7.

Lyons M, Al-Nakeeb Y, Nevill A. Performance of soccer passing skills under moderate and high-intensity localized muscle fatigue. J Strength Cond Res. 2006;20(1):197–202.

PubMed   Google Scholar  

Rampinini E, Impellizzeri FM, Castagna C, Azzalin A, Wisløff U. Effect of match-related fatigue on short-passing ability in young soccer players. Med Sci Sports Exerc. 2008;40(5):934–42.

Draganidis D, Chatzinikolaou A, Jamurtas AZ, Carlos Barbero J, Tsoukas D, Theodorou AS, Margonis K, Michailidis Y, Avloniti A, Theodorou A. The time-frame of acute resistance exercise effects on football skill performance: the impact of exercise intensity. J Sports Sci. 2013;31(7):714–22.

Lyons MJ, Conlon J, Perejmibida A, Chivers P, Joyce C. Sustained passing performance of elite and subelite female soccer players following a female match-specific exercise protocol. Int J Sports Physiol Perform. 2021;16(4):504–10.

Ali A, Gardiner R, Foskett A, Gant N. Fluid balance, thermoregulation and sprint and passing skill performance in female soccer players. Scand J Med Sci Sports. 2011;21(3):437–45.

Owen JA, Kehoe SJ, Oliver SJ. Influence of fluid intake on soccer performance in a temperate environment. J Sports Sci. 2013;31(1):1–10.

Ali A, Williams C, Nicholas CW, Foskett A. The influence of carbohydrate-electrolyte ingestion on soccer skill performance. Med Sci Sports Exerc. 2007;39(11):1969.

Ali A, Williams C. Carbohydrate ingestion and soccer skill performance during prolonged intermittent exercise. J Sports Sci. 2009;27(14):1499–508.

Foskett A, Ali A, Gant N. Caffeine enhances cognitive function and skill performance during simulated soccer activity. Int J Sport Nutr Exerc Metab. 2009;19(4):410–23.

Gant N, Ali A, Foskett A. The influence of caffeine and carbohydrate coingestion on simulated soccer performance. Int J Sport Nutr Exerc Metab. 2010;20(3):191–7.

O’Reilly J, Wong SH. Effect of a carbohydrate drink on soccer skill performance following a sport-specific training program. J Exerc Sci Fitness. 2013;11(2):95–101.

Andrade-Souza VA, Bertuzzi R, de Araujo GG, Bishop D, Lima-Silva AE. Effects of isolated or combined carbohydrate and caffeine supplementation between 2 daily training sessions on soccer performance. Appl Physiol Nutr Metab. 2015;40(5):457–63.

Shabir A, Hooton A, Spencer G, Storey M, Ensor O, Sandford L, Tallis J, Saunders B, Higgins MF. The influence of caffeine expectancies on simulated soccer performance in recreational individuals. Nutrients. 2019;11(10):2289.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Barte JC, Nieuwenhuys A, Geurts SA, Kompier MA. Motivation counteracts fatigue-induced performance decrements in soccer passing performance. J Sports Sci. 2019;37(10):1189–96.

Ben Khalifa W, Zouaoui M, Zghibi M, Azaiez F. Effects of verbal interactions between students on skill development, game performance and game involvement in soccer learning. Sustainability. 2020;13(1):160.

O’Meagher R, O’Reilly J, Ali A. The effect of different playing surfaces on soccer skill performance. Int J Sports Sci Coaching. 2022;17(6):1378–84.

Vansteenkiste P, Lenoir M, Krejtz I, Krejtz K. Visual strategies of young soccer players during a passing test–A pilot study. J Eye Mov Res. 2022;15(1):3.

Halabchi F, Abarashi M, Mansournia MA, Seifbarghi T. Effects of inhaled salbutamol on sport-specific fitness of non-asthmatic football players. Acta Medica Iranica. 2017;55(5):324–32.

Chelly MS, Hermassi S, Shephard RJ. Relationships between power and strength of the upper and lower limb muscles and throwing velocity in male handball players. J Strength Cond Res. 2010;24(6):1480–7.

Jing Z. Influences of balance training on shooting quality in basketball players. Revista Brasileira de Medicina do Esporte. 2023;29:e2023_0010.

Luo S, Soh KG, Nasiruddin NJ, Sun H, Du C, Soh KL. Effect of core training on skill performance among athletes: A systematic review. Front Physiol. 2022;13:915259.

Tracey S-Y, Anderson DI, Hamel KA, Gorelick ML, Wallace SA, Sidaway B. Kicking performance in relation to balance ability over the support leg. Hum Mov Sci. 2012;31(6):1615–23.

Paillard T, Noe F, Riviere T, Marion V, Montoya R, Dupui P. Postural performance and strategy in the unipedal stance of soccer players at different levels of competition. J Athl Train. 2006;41(2):172.

Gabbett TJ. Skill-based conditioning games as an alternative to traditional conditioning for rugby league players. J Strength Cond Res. 2006;20(2):306–15.

Gabbett TJ, Mulvey MJ. Time-motion analysis of small-sided training games and competition in elite women soccer players. J Strength Cond Res. 2008;22(2):543–52.

Gamble P. A skill-based conditioning games approach to metabolic conditioning for elite rugby football players. J Strength Cond Res. 2004;18(3):491–7.

Bergh U, Ekblom B. Influence of muscle temperature on maximal muscle strength and power output in human skeletal muscles. Acta Physiologica Scandinavica. 2010;107(1):33–7.

Stewart D, Macaluso A, De Vito G. The effect of an active warm-up on surface EMG and muscle performance in healthy humans. Eur J Appl Physiol. 2003;89:509–13.

Boksem MA, Meijman TF, Lorist MM. Effects of mental fatigue on attention: an ERP study. Brain Res Cogn Brain Res. 2005;25(1):107–16.

Lorist MM, Boksem MA, Ridderinkhof KR. Impaired cognitive control and reduced cingulate activity during mental fatigue. Cogn Brain Res. 2005;24(2):199–205.

Cao S, Geok SK, Roslan S, Sun H, Lam SK, Qian S. Mental fatigue and basketball performance: a systematic review. Front Psychol. 2022;12:819081.

Weerakkody N, Taylor C, Bulmer C, Hamilton D, Gloury J, O’Brien N, Saunders J, Harvey S, Patterson T. The effect of mental fatigue on the performance of Australian football specific skills amongst amateur athletes. J Sci Med Sport. 2021;24(6):592–6.

Csathó A, van der Linden D, Hernádi I, Buzás P, Kalmar G. Effects of mental fatigue on the capacity limits of visual attention. J Cogn Psychol. 2012;24(5):511–24.

Kaplan S. Meditation, restoration, and the management of mental fatigue. Environ Behav. 2001;33(4):480–506.

Silva JR, Rumpf M, Hertzog M, Castagna C, Farooq A, Girard O, Hader K. Acute and residual soccer match-related fatigue: a systematic review and meta-analysis. Sports Med. 2018;48:539–83.

Doma K, Deakin GB, Bentley DJ. Implications of impaired endurance performance following single bouts of resistance training: an alternate concurrent training perspective. Sports Med. 2017;47:2187–200.

Place N, Yamada T, Bruton JD, Westerblad H. Muscle fatigue: from observations in humans to underlying mechanisms studied in intact single muscle fibres. Eur J Appl Physiol. 2010;110:1–15.

Berger RA, Smith-Hale LA. Effects of fatigue on performance and learning of a gross motor task. J Strength Cond Res. 1991;5(3):155–61.

Davey PR, Thorpe RD, Williams C. Fatigue decreases skilled tennis performance. J Sports Sci. 2002;20(4):311–8.

Evans RK, Scoville CR, Ito MA, Mello RP. Upper body fatiguing exercise and shooting performance. Mil Med. 2003;168(6):451–6.

McKenna MJ, Hargreaves M: Resolving fatigue mechanisms determining exercise performance: integrative physiology at its finest! In . : American Physiological Society; 2008: 104: 286-287.

Sejersted OM, Sjøgaard G. Dynamics and consequences of potassium shifts in skeletal muscle and heart during exercise. Physiol Rev. 2000;80(4):1411–81.

Lieberman HR. Hydration and cognition: a critical review and recommendations for future research. J Am Coll Nutr. 2007;26(sup5):555S–561S.

Sawka MN, Burke LM, Eichner ER, Maughan RJ, Montain SJ, Stachenfeld NS. American College of Sports Medicine position stand. Exercise and fluid replacement. Med Sci Sports Exerc. 2007;39(2):377–90.

Benardot D. Advanced sports nutrition. Champaign: Human Kinetics Publishers; 2021.

Stuart GR, Hopkins WG, Cook C, Cairns SP. Multiple effects of caffeine on simulated high-intensity team-sport performance. Med Sci Sports Exerc. 2005;37(11):1998–2005.

Assi H, Bottoms L. The effects of caffeine on rugby passing accuracy while performing the Reactive Agility Test. Sci Sports. 2014;29(5):275–81.

McLellan TM, Kamimori GH, Bell DG, Smith IF, Johnson D, Belenky G. Caffeine maintains vigilance and marksmanship in simulated urban operations with sleep deprivation. Aviat Space Environ Med. 2005;76(1):39–45.

CAS   PubMed   Google Scholar  

McLellan TM, Kamimori GH, Voss DM, Bell DG, Cole KG, Johnson D. Caffeine maintains vigilance and improves run times during night operations for Special Forces. Aviat Space Environ Med. 2005;76(7):647–54.

Tikuisis P, Keefe AA, McLellan TM, Kamimori G. Caffeine restores engagement speed but not shooting precision following 22 h of active wakefulness. Aviat Space Environ Med. 2004;75(9):771–6.

Download references

Acknowledgements

Not applicable.

A grant from the Ministry of Education of China, Grant No. 20YJA890002, supported this study.

Author information

Bihan Wang and Bin Wan contributed equally to this study.

Authors and Affiliations

College of Physical Education, Hunan Normal University, Changsha, 410006, China

Bihan Wang, Bin Wan, Shu Chen, Yu Zhang, Changfa Tang & Bo Long

School of Physical Education, Huzhou University, Huzhou, 313000, China

Xiaorong Bai & Wensheng Xiao

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, WBH and XWS; methodology, XWS and BXR; writing—original draft preparation, WBH, BXR and XWS; writing—review and editing, WBH and XWS. All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Wensheng Xiao , Changfa Tang or Bo Long .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wang, B., Wan, B., Chen, S. et al. A Systematic review of the factors that affect soccer players’ short-passing ability—based on the Loughborough Soccer Passing Test. BMC Sports Sci Med Rehabil 16 , 96 (2024). https://doi.org/10.1186/s13102-024-00880-y

Download citation

Received : 27 October 2023

Accepted : 11 April 2024

Published : 26 April 2024

DOI : https://doi.org/10.1186/s13102-024-00880-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Short-passing ability
• Influencing factors
• Loughborough Soccer Passing Test

BMC Sports Science, Medicine and Rehabilitation

ISSN: 2052-1847

• General enquiries: [email protected]