types of research materials for literature review

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters and Hedges
• Grey Literature
• Managing and Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Apr 17, 2024 2:02 PM
• URL: https://guides.library.ucla.edu/systematicreviews

• University of Wisconsin–Madison
• University of Wisconsin-Madison
• Research Guides
• Evidence Synthesis, Systematic Review Services
• Literature Review Types, Taxonomies

Evidence Synthesis, Systematic Review Services : Literature Review Types, Taxonomies

• Develop a Protocol
• Develop Your Research Question
• Select Databases
• Select Gray Literature Sources
• Write a Search Strategy
• Manage Your Search Process
• Register Your Protocol
• Citation Management
• Article Screening
• Risk of Bias Assessment
• Synthesize, Map, or Describe the Results
• Find Guidance by Discipline
• Manage Your Research Data
• Browse Evidence Portals by Discipline
• Automate the Process, Tools & Technologies
• Additional Resources

Choosing a Literature Review Methodology

Growing interest in evidence-based practice has driven an increase in review methodologies. Your choice of review methodology (or literature review type) will be informed by the intent (purpose, function) of your research project and the time and resources of your team. 

• Decision Tree (What Type of Review is Right for You?) Developed by Cornell University Library staff, this "decision-tree" guides the user to a handful of review guides given time and intent.

Types of Evidence Synthesis*

Critical Review - Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or model.

Mapping Review (Systematic Map) - Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature.

Meta-Analysis - Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results.

Mixed Studies Review (Mixed Methods Review) - Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies.

Narrative (Literature) Review - Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness.

Overview - Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics.

Qualitative Systematic Review or Qualitative Evidence Synthesis - Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies.

Rapid Review - Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research.

Scoping Review or Evidence Map - Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research.

State-of-the-art Review - Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives on issue or point out area for further research.

Systematic Review - Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review. (An emerging subset includes Living Reviews or Living Systematic Reviews - A [review or] systematic review which is continually updated, incorporating relevant new evidence as it becomes available.)

Systematic Search and Review - Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis.’

Umbrella Review - Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results.

*These definitions are in Grant & Booth's "A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies."

Literature Review Types/Typologies, Taxonomies

Grant, M. J., and A. Booth. "A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies."  Health Information and Libraries Journal  26.2 (2009): 91-108.  DOI: 10.1111/j.1471-1842.2009.00848.x  Link

Munn, Zachary, et al. “Systematic Review or Scoping Review? Guidance for Authors When Choosing between a Systematic or Scoping Review Approach.” BMC Medical Research Methodology , vol. 18, no. 1, Nov. 2018, p. 143. DOI: 10.1186/s12874-018-0611-x. Link

Sutton, A., et al. "Meeting the Review Family: Exploring Review Types and Associated Information Retrieval Requirements."  Health Information and Libraries Journal  36.3 (2019): 202-22.  DOI: 10.1111/hir.12276  Link

• << Previous: Home
• Next: The Systematic Review Process >>
• Last Updated: May 8, 2024 5:33 PM
• URL: https://researchguides.library.wisc.edu/literature_review

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Methodological Approaches to Literature Review

• Living reference work entry
• First Online: 09 May 2023
• Cite this living reference work entry

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  

503 Accesses

1 Citations

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 9, 2024 11:05 AM
• URL: https://libguides.usc.edu/writingguide

ACAP LEARNING RESOURCES

ACAP Pathfinder: Literature Review

• What is a Literature Review?
• Examples of Literature Reviews
• The Research Question

Types of Literature

• How to Search
• Recording the Search
• Reference Management
• Evaluating the Literature

Your research question or thesis statement will inform the types of literature that will best suit the review. There are many different types literature and they come from a variety of sources. The resources described below provide you with a general outline of the types of literature available via the library. Go to the Choosing Resources page in the Information Skills guide to learn more about where and how to find these resources in the library and on the internet.

• THEORY-BASED
• PHILOSOPHICAL
• STATISTICAL

Research Literature

Some edited books, journal articles, theses and government publications will employ research paradigms and methodologies to support a hypothesis. Methodologies can be broadly categorised as longitudinal, qualitative or quantitative. Many of these publications are peer-reviewed, meaning they have been checked by a panel of experts before publication. Most library databases offer a peer-reviewed checkbox which will filter search results in this way. They will also allow you to filter results according to research methodology, focus group, geographical location and much more. Research literature is an essential component of your literature review.

Theory-based Literature

Literature that is informed and tested by research, these books, articles and reference sources will attempt to explain, describe, define and provide a background or theoretical framework for a field of inquiry. These sources may include the original works of primary theorists as well as works which build upon, critique and discuss these primary sources while connecting it to the latest research. This type of literature is also an important component of any literature review.

Philosophical Literature

Information sources such as books and articles, which deal with the underlying beliefs, attitudes and concepts that form the basic assumptions or building blocks within a profession or field of study.  This kind of literature formulates critical inquiry from either an  ethical, epistemological, metaphysical or logical standpoint.  The extent to which you use philosophical literature will depend on the focus and subject matter of your review but it may be useful when constructing a background or theoretical base in your writing.

Empirical or Practice-based Literature

Statistical Reports

Grey Literature

Types of Resources

• FILM & VIDEO
• NEWS & MAGAZINES
• REFERENCE SOURCES
• CONFERENCES & THESES
• GOVERNMENT & POLICIES

Description

Provide an overview of a subject area or of a number of related topics.  They may also include detailed information about a specific topic or topics. Search for print books using  MultiSearch .  These items may be collected from library shelves or requested from other campuses. eBooks are searchable from  MultiSearch  or  A-Z Databases  and may be read online or downloaded to any PC or device.

Use books to gather comprehensive information on a topic. In the library, you will find mostly academic, non-fiction items which may be used in your assessment tasks to sketch out an overview on a subject or to illustrate an in-depth understanding.

• Essentials of Psychology Concepts and Applications
• Addiction: Psychology and Treatment

Relevant Links

• http://libguides.navitas.com/borrow
• http://libguides.navitas.com/eresources/books

Journal publications, sometimes called periodicals or serials, contain articles which offer research, reports, reviews, letters and other papers on specific topics. They are usually published weekly, monthly, quarterly or yearly. Academic publications are often peer-reviewed and as such provide up-to-date information from authorised sources. Search for journal articles via  MultiSearch  and  A-Z  Databases . Search for journal publications using  A-Z Journals . You can also search for articles on the internet via  Google Scholar  or  Researchgate . 

Journal articles can provide you with more up-to-date information on specific aspects of a topic, and in smaller more digestible packages than books. Use journal publications to access scholarly research on a topic, often from a unique or new perspective.

• International Journal of Clinical & Health Psychology
• Khan, F., Chong, J., Theisen, J., Fraley, R., Young, J., & Hankin, B. (2020). Development and Change in Attachment: A Multiwave Assessment of Attachment and Its Correlates Across Childhood and Adolescence.  Journal of Personality and Social Psychology ,  118 (6), 1188–1206. https://doi.org/10.1037/pspi0000211
• http://libguides.navitas.com/eresources/articles

Audiovisual material in the form of DVDs, podcasts, online streaming and so on, is searchable from  MultiSearch  or  A-Z Databases . Videos in the library include movies, conferences and seminars, tutorials, documentaries and much more.

AV items may be used as instructional material or gain an understanding of a topic by way of visual or concrete examples. 

• Narrative Family Therapy
• Waltz with Bashir
• http://libguides.navitas.com/eresources/videos

News items are published at regular intervals and provide new information about various topics of interest to the general public. Magazines are also produced regularly and may focus on a particular subject area or cover a range of topics, again for consumption by the general public. The information within news publications and magazines are not in themselves scholarly works but may refer to academic sources. These sources can be found by searching  MultiSearch  or  A-Z Databases

To access recent or new information about current affairs, social, economic or political issues which provide an overview or introduction in digestible and readable packages.  While not scholarly information, these resources may be required for use in particular assessment tasks or point towards recent research in a particular field. 

• Australia targeted in cyber crime increase
• New Scientist
• Search for news on Google

Reference items such as dictionaries, encyclopaedias, handbooks and industry standards provide an overview of a subject area, or include specific definitions, technical or practical information. They are searchable via  MultiSearch  and  A-Z Databases . Reference works at ACAP cover a range of topics including but not limited to sociology, social work and philosophy, psychology, counselling and mental illness, legal materials and policies, diagnoses, drug overviews, care planning and best practices for healthcare workers.

Useful for a broad understanding of a topic, theory or theorist or for assessment tasks which require technical or practical information about a particular industry or field of inquiry. 

• Evidence-Based Policy
• Diagnostic and statistical manual of mental disorders : DSM-5

A website is a set of related and interlinked pages which are hosted on the World Wide Web (WWW). A webpage refers to the individual pages contained within these sites. To search the WWW, download an internet browser (we recommend Google Chrome) and use a search engine such as Google to find content.

The websites of particular institutions and government departments may be useful for your assessment tasks. You might also want to look at scholarly sites, such as Google Scholar, Researchgate, State Libraries, academic publishers and journal indexing services, to find the most scholarly information on the Web. Links recommended by the library can be found on the  Useful Websites  page. 

• Google Scholar
• Dulwich Centre Resources

Conference papers are presented at conferences which are usually themed around a specific subject area or set of related topics and presented as a collection of proceedings. Some papers may be peer-reviewed and are searchable from  MultiSearch , within  journal databases ,  Dissertation Express  or via  Libraries Australia Trove database . A thesis or dissertation involves personal research, written by a candidate for an advanced university degree. Theses are also searchable from  MultiSearch ,  Trove , or within individual academic institutional repositories and indexing databases such as  PQDT Open ,  NDLTD ,  CORE  and  DART .

Conference proceedings and theses can provide you with an in-depth look at some of the latest research on specific aspects of a topic.

• Counselor education: A personal growth & personal development experience
• Awareness of memory deficits in Parkinson’s disease
• http://libguides.navitas.com/ill

Legal resources include documents such as, but not limited to Bills, Acts, regulations, statutory laws, by-laws, proceedings of Parliament, legal cases and tribunal decisions. Some of these resources may be found by searching  MultiSearch , in databases such as  AustLII  or  JADE  or within legislative sites for individual states and territories. You will also find cases and tribunal decisions on the websites of regulatory authorities such as  AHPRA .

Use these information sources when you need to refer to current laws, records, cases and decisions in your assessment tasks.   

• AustLII Cases & Legislation
• Library Resources for Criminology & Justice: Law & Legislation

Government policies, reports, gazettes, media releases and parliamentary publications such as Hansards, are available from various websites here in Australia.  You can also search Google to find individual publications.  MultiSearch  and  journal  databases  will include some government papers and reports in search results.  However, you should also directly consult  federal  and  state  departments and agencies, Libraries Australia  GovPubs,  the  Analysis and Policy Observatory  and State and  Federal  Parliamentary libraries. 

Use these sources in your assessment tasks to access up-to-date and authoritative information on subject areas which may be affected by Federal or State government.

• Domestic and family violence and parenting: mixed method insights into impact and support needs - final report
• National Statement on Ethical Conduct in Human Research

Statistical reports and data from sites such as the  Australian Bureau of Statistics  gather information for a particular field of research or to report on the views and habits of the population. Data collection may be performed via interviews, questionnaires, surveys, censuses and so on. You'll also be able to search for and access statistical information using  MultiSearch  and  journal databases . Other important sites for statistics include the  Australian Institute of Family Studies ,  Australian Institute of Health & Welfare  and  HILDA .

An important part of the research in any field of study, statistical reporting and datasets are useful for description, analysis and comparison in your assessment tasks.

• 4329.0.00.003 - Patterns of Use of Mental Health Services and Prescription Medications, 2011
• Parent-child contact after separation
• << Previous: The Research Question
• Next: How to Search >>
• Last Updated: Sep 13, 2023 10:53 AM
• URL: https://libguides.navitas.com/literature-review

Types of Literature Review

There are many types of literature review. The choice of a specific type depends on your research approach and design. The following types of literature review are the most popular in business studies:

Narrative literature review , also referred to as traditional literature review, critiques literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps or inconsistencies in a body of knowledge. You need to have a sufficiently focused research question to conduct a narrative literature review

Systematic literature review requires more rigorous and well-defined approach compared to most other types of literature review. Systematic literature review is comprehensive and details the timeframe within which the literature was selected. Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis.

When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn. Meta-analysis is associated with deductive research approach.

Meta-synthesis, on the other hand, is based on non-statistical techniques. This technique integrates, evaluates and interprets findings of multiple qualitative research studies. Meta-synthesis literature review is conducted usually when following inductive research approach.

Scoping literature review , as implied by its name is used to identify the scope or coverage of a body of literature on a given topic. It has been noted that “scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review.” [1] The main difference between systematic and scoping types of literature review is that, systematic literature review is conducted to find answer to more specific research questions, whereas scoping literature review is conducted to explore more general research question.

Argumentative literature review , as the name implies, examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. It should be noted that a potential for bias is a major shortcoming associated with argumentative literature review.

Integrative literature review reviews , critiques, and synthesizes secondary data about research topic in an integrated way such that new frameworks and perspectives on the topic are generated. If your research does not involve primary data collection and data analysis, then using integrative literature review will be your only option.

Theoretical literature review focuses on a pool of theory that has accumulated in regard to an issue, concept, theory, phenomena. Theoretical literature reviews play an instrumental role in establishing what theories already exist, the relationships between them, to what degree existing theories have been investigated, and to develop new hypotheses to be tested.

At the earlier parts of the literature review chapter, you need to specify the type of your literature review your chose and justify your choice. Your choice of a specific type of literature review should be based upon your research area, research problem and research methods.  Also, you can briefly discuss other most popular types of literature review mentioned above, to illustrate your awareness of them.

[1] Munn, A. et. al. (2018) “Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach” BMC Medical Research Methodology

  John Dudovskiy

• University of Texas Libraries
• UT Libraries

Systematic Reviews & Evidence Synthesis Methods

Types of reviews.

• Formulate Question
• Find Existing Reviews & Protocols
• Register a Protocol
• Searching Systematically
• Supplementary Searching
• Managing Results
• Deduplication
• Critical Appraisal
• Glossary of terms
• Librarian Support
• Video tutorials This link opens in a new window
• Systematic Review & Evidence Synthesis Boot Camp

Not sure what type of review you want to conduct?

There are many types of reviews ---  narrative reviews ,  scoping reviews , systematic reviews, integrative reviews, umbrella reviews, rapid reviews and others --- and it's not always straightforward to choose which type of review to conduct. These Review Navigator tools (see below) ask a series of questions to guide you through the various kinds of reviews and to help you determine the best choice for your research needs.

• Which review is right for you? (Univ. of Manitoba)
• What type of review is right for you? (Cornell)
• Review Ready Reckoner - Assessment Tool (RRRsAT)
• A typology of reviews: an analysis of 14 review types and associated methodologies. by Grant & Booth
• Meeting the review family: exploring review types and associated information retrieval requirements | Health Info Libr J, 2019

Reproduced from Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies . Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x

• Last Updated: Apr 9, 2024 8:57 PM
• URL: https://guides.lib.utexas.edu/systematicreviews

Materials Science and Engineering

• Getting Started
• Dictionaries, Encyclopedias, and Handbooks
• Material Properties
• Technical Reports
• Writing & Citing

Introduction

Gather Your Tools

Determine the Project's Scope

Create the Search Strategy

Determine What Resources to Use

Search, read, refine, repeat.

Saved Searches, Alerts and Feeds

This page focuses on how to do an in-depth literature review for a dissertation, thesis, grant application or lengthy term paper in electrical engineering.  

• For a more general description of what an in-depth literature review is and how it looks, see our guide on " Literature Reviews and Annotated Bibliographies " created by Ed Oetting, history and political science librarian.
• For lower-level engineering undergraduate students who are doing a short term paper, the " How to Research a Topic " page on the " Engineerng Basics" guide may be more applicable.

Library Account Is your library account clear of fines?   If not, you may not be allowed to check out more books nor renew books you already have.  All library notices are sent via email to your "asu.edu" address; if you prefer to receive email at a different address make sure you have forwarded your asu.edu correctly.  Also, make sure that your spam filter allows the library email to come through. 

Illiad (Interlibrary Loan) Account   If you don't already have an ILLiad account, please register for one.  Interlibrary loan services will get you material not available at the ASU Library and also scan or deliver materials from the libraries on the other ASU campuses.

Determine the Project's Scope.

Do you know what you are looking for?  Can you describe your project using one simple sentence or can you phrase the project as a question?  Without a clear idea of the project, you may not be able to determine which are the best resources to search, what terminology should be used in those resources, and if the results are appropriate and sufficient.    

If you're having difficulty getting your project described succinctly, try using a PICO chart to identify the concepts involved:

• P is the popluation, problem, predicament or process
• I is the intervention or improvement
• C is what you'll compare your intervention/improvment to, and
• O is the outcome (or results of the comparison of I and C ) 

For example: 

Your client, the owner of a nuclear power generating facility, has had several less than optimal safety inspections recently.  The inspectors have singled out operator error as a major concern and have required changes in employee training.  But is more training the solution?  The employees complain that the plant's poorly designed control room hampers their ability to respond to non-standard situations.  Could a redesign improve performance and decrease the occurance of unsafe events?   Your client wants more than just your opinion, he wants to see the data to back it up.   So, what can you find in the literature?

Here's one way that the PICO chart could be filled out:     

• P =   nuclear power safety  
• I  = human factors engineering
• C  =  additional training; little or no human factors engineering used  
• O = accident rate or safety inspection comparison

And here are examples of possible search statements:  

• I am looking for ways that human factors engineering can improve safety in the nuclear power industry.
• Is additional training or employing human factors engineering the better method for reducing safety violations in a nuclear power plant? 

Your research will always start with a " P AND I " search; those are the most important pieces of the puzzle.  However, once you have the results from that search, you'll need to know where you want to go with those results; that's when the C and O concepts need to be considered.  

 Also, don't forget --- determine if your project has limits.  For example:

• Are you reviewing the literature only within a specific time frame?
• Are you looking at English-language material only?
• Are you considering research from just the United States or worldwide?
• Are there types of material you won't be covering (trade magazines, patents, technical reports, etc.)?  

Take the simple sentence or question that describes what you are looking for.  What are the concepts in the sentence? Are there synonyms that describe the same concept?   If you filled out a PICO chart, concentrate on the  P (problem) and the  I (intervention) for the concept chart.  

Concept Chart:

Concept 1:   _______  OR _______  OR _______  AND Concept 2:   _______   OR   _______  OR   _______  AND Concept 3:  _______  OR  _______  OR   _______ 

  Example:  

I am looking for ways that human factors engineering can improve safety in the nuclear power industry. 

Concept 1:   nuclear power    OR _ nuclear industry _____   AND Concept 2:   _safety___  OR   _accident prevention____   AND Concept 3: _ human factors engineering ___   

What resources you'll use for your literature review depends on what types of materials you want to find.  

• Background Information The more you know about a topic, the better you'll be able to research it.  You'll be familiar with the terminology, understand the underlining science/technology and be aware of the issues in the field. Most importantly, you'll be able to understand what you've retrieved from your search.  But no matter how much you know before hand you'll likely run across terms and concepts with which you're unfamiliar.    Materials such as encyclopedias, dictionaries and handbooks will not only help you learn about the basics of your topic before you begin your search but they'll also help you understand the terminology used in the documents you found from your literature review.    You'll find these types of resources listed on the Dictionaries and Handbooks pages on this guide.
• Books The large size of books (usually 100-500 pages) allows a topic to be studied braodly, covering many different issues.  Conversely, the large size also allows for a specific aspect of the topic to be covered in great detail.  Because of the time it takes to publish, sci-tech books generally do not contain the most current information. To find print and online books from both the ASU Library as well as in other libraries, see the Books page on this guide.
• Conference Papers Scientists and engineers frequently present new findings at conferences before these findings are written up in journal articles or books.  Not every conference, however, publishes it proceedings.  In some cases, conferences publish only a few of the papers presented but not all.   Many resources that help you find journal articles, may also be used to find conference papers, see the Articles page on this guide.
• Journal and Trade Magazine Articles Articles in journals (also called magazines) are short, usually 5-20 pages in length and cover a specific finding, experiment or project.  Articles in scholary journals are usually written by academics or professional scientists/engineers and are aimed at others at the same level.   Articles in trade journals/magazines are written by the journal staff and report on industry news suchs as sales, mergers, prices, etc.   To find journal and trade magazine articles, use the resources listed on the Articles page on this guide. 
• Patents Patents are grants from governments that gives the inventor certain rights of manufacture.  Patents provide a wealth of information for how a technology is being advanced and by which companies.  It is frequently stated that 80% of the information in patents never appears elsewhere in the literature. 
• To identify patents granted in the U.S. and internationally see the " Searching for Patents " guide.
• To see statistical information for U.S. patents by technology class see the US Patent and Trademark's website.
• Technical Reports Technical reports are part of the "gray literature";  gray literature refers to documents that are not published commercially, hence they are difficult to both identify and find.  Technical reports focus on a specific experiment or research project and are meant to convey the results of the experiment or project back to the funding organization.  In the United States, common sources of technical reports are the government agencies that sponsor research projects.  Reports generated within a private corporation and funded soley by that corporation are seldom ever available to anyone outside of the company.      To find technical reports, use the resources listed on the Technical Reports page on this guide.     

Search, Read, Refine and Repeat

Now it's time to apply your search strategy in the resources you've decided to use.

• Use the Advanced Search feature (or whatever search is set up with the 3 lines of boxes) and enter your search strategy just as you recorded in your search strategy chart.  Don't forget to set your limits.   If the resource only provides a single search box, rearrange your chart from vertical into horizontal so that the search statement looks like this:   (Concept#1 OR synonym) AND (Concept#2 OR synonym) AND (Concept#3 OR synonym) Example: (nuclear power OR nuclear industry) AND (safety OR accident prevention) AND (human factors engineering)
• Examine the results to find the most appropriate items.  Keep your one-sentence project description (and/or your PICO chart) in mind to help you stay on track.
• Export the records/citations you want to keep into a citation manager.
• If there are subjects (may also be called subject headings, index terms, descriptors or controlled vocabulary) assigned to each item, make sure that those also transferred into a citation manager.  If not, add them manually.
• Get the full text of the items 
• Read the full text of the items and look at the subjects assigned to the item and consider:
• Do I have to change (narrow) my topic to something more specific because I'm finding way too much? 
• Do I have to change (broaden) my topic because I can't find enough about it? 
• Is there additional terminology for my topic/concepts that I hadn't included in my original search?
• Redo your search strategy according to what you found in step #6 and rerun the search in the resources again.
• You may need to repeat this cycle several times before you are able to identify the best terminology to use in each resource. 

If there will be several months in between when you search the literature and when you turn in the paper, consider setting up alerts and feeds so that you are notified should new items about your topic appear.   How you set up an alert or feed will vary.  In most cases you'll be required to set up a personal account or profile with the journal or database --- there is no charge for this but you will have to identify yourself and provide an email address.  

For instructions on setting up alerts and feeds, see the " Keep Current " page.

• << Previous: Writing & Citing
• Last updated: Jan 2, 2024 8:27 AM
• URL: https://libguides.asu.edu/mse

The ASU Library acknowledges the twenty-three Native Nations that have inhabited this land for centuries. Arizona State University's four campuses are located in the Salt River Valley on ancestral territories of Indigenous peoples, including the Akimel O’odham (Pima) and Pee Posh (Maricopa) Indian Communities, whose care and keeping of these lands allows us to be here today. ASU Library acknowledges the sovereignty of these nations and seeks to foster an environment of success and possibility for Native American students and patrons. We are advocates for the incorporation of Indigenous knowledge systems and research methodologies within contemporary library practice. ASU Library welcomes members of the Akimel O’odham and Pee Posh, and all Native nations to the Library.

• Open access
• Published: 01 May 2024

Hospital performance evaluation indicators: a scoping review

• Shirin Alsadat Hadian   ORCID: orcid.org/0000-0002-1443-1990 1 ,
• Reza Rezayatmand   ORCID: orcid.org/0000-0002-9907-3597 2 ,
• Nasrin Shaarbafchizadeh   ORCID: orcid.org/0000-0001-7104-2214 3 ,
• Saeedeh Ketabi   ORCID: orcid.org/0000-0002-6778-5645 4 &
• Ahmad Reza Pourghaderi   ORCID: orcid.org/0000-0003-2682-2160 5  

BMC Health Services Research volume  24 , Article number:  561 ( 2024 ) Cite this article

238 Accesses

Metrics details

Hospitals are the biggest consumers of health system budgets and hence measuring hospital performance by quantitative or qualitative accessible and reliable indicators is crucial. This review aimed to categorize and present a set of indicators for evaluating overall hospital performance.

We conducted a literature search across three databases, i.e., PubMed, Scopus, and Web of Science, using possible keyword combinations. We included studies that explored hospital performance evaluation indicators from different dimensions.

We included 91 English language studies published in the past 10 years. In total, 1161 indicators were extracted from the included studies. We classified the extracted indicators into 3 categories, 14 subcategories, 21 performance dimensions, and 110 main indicators. Finally, we presented a comprehensive set of indicators with regard to different performance dimensions and classified them based on what they indicate in the production process, i.e., input, process, output, outcome and impact.

The findings provide a comprehensive set of indicators at different levels that can be used for hospital performance evaluation. Future studies can be conducted to validate and apply these indicators in different contexts. It seems that, depending on the specific conditions of each country, an appropriate set of indicators can be selected from this comprehensive list of indicators for use in the performance evaluation of hospitals in different settings.

Peer Review reports

Healthcare is complex [ 1 ] and a key sector [ 2 ] that is now globally faced with problems of rising costs, lack of service efficiency, competition, and equity as well as responsiveness to users [ 3 ]. One estimate by the WHO has shown a yearly waste of approximately 20–40% of total healthcare resources because of inefficiency [ 4 ]. European countries have spent on average 9.6% of their gross domestic product (GDP) on healthcare in 2017 and 9.92% in 2019. Germany, France, and Sweden reported the highest healthcare expenditures in Europe in 2018 (between 10.9% and 11.5% of GDP) [ 5 ]. In the U.S., healthcare spending consumes 18% of the GDP, which is likely to eclipse $6 trillion by 2027 [ 6 ].

Hospitals, as the biggest consumers of health system budgets [ 7 ], are the major part of the health system [ 8 ]. In many countries 50–80% of the health sector budget is dedicated to hospitals [ 8 , 9 ]. As a result, hospital performance analysis is becoming a routine task for every hospital manager. On the one hand, hospital managers worldwide are faced with difficult decisions regarding cost reduction, increasing service efficiency, and equity [ 10 ]. On the other hand, measuring hospital efficiency is an issue of interest among researchers because patients demand high-quality care at lower expenses [ 11 ].

To address the above mentioned need to measure hospital performance, implementing an appropriate hospital performance evaluation system is crucial in any hospital. In doing so, hospital administrators use various tools to analyse and monitor hospital activities [ 1 ], which need well-defined objectives, standards and quantitative indicators [ 12 ]. The latter are used to evaluate care provided to patients both quantitatively and qualitatively and are often related to input, output, processes, and outcomes. These indicators can be used for continuous quality improvement by monitoring, benchmarking, and prioritizing activities [ 13 ]. These parameters are developed to improve health outcomes and to provide comparative information for monitoring and managing and formulating policy objectives within and across health services [ 12 ]. Studies thus far have used their own set of indicators while evaluating hospital performance, which could be context dependent. In addition, those studies have mostly used a limited set of indicators that focus on few dimensions (2–6 dimensions) of hospital performance [ 14 , 15 , 16 , 17 , 18 ].

Therefore, comprehensive knowledge of potential indicators that can be used for hospital performance evaluation is necessary. It would help choose appropriate indicators when evaluating hospital performance in different contexts. It would also help researchers extend the range of analysis to evaluate performance from a wider perspective by considering more dimensions of performance. Although performance is a very commonly used term, it has several definitions [ 19 , 20 ], yet, it is often misunderstood [ 21 ]. Therefore, some researchers have expressed confusion about the related terms and considered them interchangeable. These terms are effectiveness, efficiency, productivity, quality, flexibility, creativity, sustainability, evaluation, and piloting [ 21 , 22 , 23 ]. Thus, this scoping review aimed to categorize and present a comprehensive set of indicators that can be used as a suitable set for hospital performance evaluation at any needed level of analysis, i.e., clinical, para-clinical, logistical, or departmental, and relate those indicators to the appropriate performance dimensions. The uniqueness of this paper is that it provides its readers with a comprehensive collection of indicators that have been used in different performance analysis studies.

Materials and methods

We conducted a scoping review of a body of literature. The scoping review can be of particular use when the topic has not yet been extensively reviewed or has a complex or heterogeneous nature. This type of review is commonly undertaken to examine the extent, range, and nature of research activity in a topic area; determine the value and potential scope and cost of undertaking a full systematic review; summarize and disseminate research findings; and identify research gaps in the existing literature. As a scoping review provides a rigorous and transparent method for mapping areas of research, it can be used as a standalone project or as a preliminary step to a systematic review [ 24 ]. While a systematic review (qualitative or quantitative) usually addresses a narrow topic/scope and is a method for integrating or comparing findings from previous studies [ 25 ].

In our study, we used the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist following the methods outlined by Arksey and O’Malley [ 26 ] and Tricco [ 27 ]. A systematic search for published and English-language literature on hospital performance evaluation models was conducted, using three databases, i.e., PubMed, Scopus, and Web of Science, from 2013 to January 2023. Initially, the identified keywords were refined and validated by a team of experts. Then, a combination of vocabularies was identified by the authors through a brainstorming process. The search strategy was formulated using Boolean operators. The title and abstract of the formulas were searched in the online databases. The search query for each database is presented in Table  1 .

In the screening process, relevant references related to hospital performance evaluation were screened and abstracted into researcher-developed Microsoft® Excel forms by dual independent reviewers and conflicting information was provided by other reviewers.

The inclusion criteria were as follows: focused only on the hospital setting, available full text and written in English. We excluded studies that focused on health organization indicators, not specifically on hospital indicators; articles without appropriate data (only focused on models and not indicators; or qualitative checklist questionnaires); and articles that focused only on clinical or disease-related indicators, not hospital performance dimensions, and provided very general items as indicators, not the domains of the indicators themselves. Then, a PRISMA-ScR Checklist was used to improve transparency in our review [ 28 ].

To extract the data, researcher-developed Microsoft® Excel forms (data tables) were designed. The following data were subsequently extracted into Microsoft®Excel for synthesis and evaluation: title, author, article year, country, indicator category, study environment (number of hospitals studied), study time frame, indicator name, number of indicators, indicator level (hospital level, department level), evaluation perspective (performance, productivity, efficiency, effectiveness, quality, cost, safety, satisfaction, etc. ) , study type (quantitative or qualitative), indicator subtype (input (structure), process, output (result), outcome and impact), and other explanations. To create a descriptive summary of the results that address the objectives of this scoping review, numerical summarization was also used.

The purpose of creating the main category and the evaluation perspective section was to develop them and create new categories, which focused on the type of indicators related to the performance term. For example, in the “Category” section, the names of the departments or wards of the hospital (such as hospital laboratories, pharmacies, clinical departments, and warehouses) and in the “Evaluation perspective” section, various terms related to the evaluation of hospital performance were extracted. These two types were used after extracting their information under the title “performance dimension”.

The indicators’ levels were collected to determine the level of performance evaluation with the relevant index. Some indicators were used to evaluate the performance of the entire hospital, some were used to evaluate the performance of hospital departments, and some were used to evaluate the performance at the level of a specific project. For example, several indicators (such as bed occupancy ratio, length of stay, and waiting time) were used to evaluate the performance of the entire hospital, and other indicators (such as laboratory department indicators, energy consumption indicators, and neonatal department indicators) were used only to measure the performance of specific departments. This sections were used under the title “category”. The “category” and “indicator’s name” sections were defined according to the results of the “subcategory” section.

The subtypes of indicators (input (structure), process, output(result), outcome and impact) were defined based on the chain model, and each of the selected indicators was linked to it (Appendix 1 ). As a result of the chain model, inputs were used to carry out activities, activities led to the delivery of services or products (outputs). The outputs started to bring about change (outcomes), and eventually, this (hopefully) contributed to the impact [ 29 ]. The classification of the set of input, process, output, outcome and impact indicators was such that readers could access these categories if necessary according to their chosen evaluation models. The term was used under the title “Indicators by types”.

The type of study was considered quantitative or qualitative for determining whether an indicator was able to perform calculations. In this way, readers can choose articles that use quantitative or qualitative indicators to evaluate hospital performance.

We included 91 full-text studies (out of 7475) in English published between 2013 and January 2023 (Fig.  1 ), approximately 40% of which were published between 2020 and 2023. More than 20% of the retrieved studies were conducted in Iran and USA.

Study selection and data abstraction

Study characteristic

As shown in Table  2 , in 85% of the reviewed studies, a number of hospitals (1 to 3828 hospitals, 13,221 hospitals in total) were evaluated. More than 90% of the studies used a quantitative approach. In more than 70% of the studies, hospital evaluation occurred at the department level, which can also be divided into three levels: administrative, clinical ward, and paramedical department. In addition, the administrative departments consist of 13 departments, including financial management [ 48 , 55 , 61 , 67 , 68 , 80 , 83 , 109 , 113 ], supply chain management and warehouse [ 15 , 43 , 84 ], value-based purchasing [ 33 , 85 ], human resource management [ 97 , 101 ], medical equipment [ 32 , 87 ], health information management department [ 90 ], information systems [ 106 ], nutritional assessment [ 93 ], energy management [ 30 , 45 , 92 ], facility management [ 52 , 53 ], building sustainability and resilience [ 35 ], research activities [ 44 ], and education [ 107 ].

The clinical wards consisted of 8 wards, namely, emergency departments (EDs) [ 16 , 39 , 56 , 57 , 69 , 70 , 89 ], surgery departments [ 58 , 62 , 63 , 91 , 102 ], intensive care units (ICUs) [ 47 , 64 , 65 ], operating rooms (ORs) [ 38 , 88 , 108 ], surgical intensive care units (SICUs) [ 111 ], obstetrics and gynecology department [ 59 ], neonatal intensive care units (NICUs) [ 74 , 103 ] and quality of care [ 18 , 31 , 40 , 50 , 72 , 92 , 95 , 112 ] indicators. The paramedical departments consisted of 3 departments, pharmacy [ 60 , 76 , 98 ], laboratory and blood bank [ 37 , 42 , 43 , 49 ], and outpatient assessment [ 86 ] indicators.

With regard to data categorization, firstly, a total of 1204 indicators in 91 studies were extracted and after detailed examination, 43 indices (such as hospital ownership, level of care, admission process, and personal discipline) were removed due to their generality and impossibility of calculation in the hospital environment. Then, 1161 performance indicators were entered in this research and were categorized based on the performance criteria (more details about the indicators can be found in Appendix 1 ). Secondly, 145 functional dimensions, including divisions based on different departments and units of the hospital, were defined according to several focus group discussions with 5 health experts. Then, re-categorization and functional summarization were performed, after which 21 performance dimensions were finalized.

As shown in Table  4 , the 21 performance dimensions were divided into three parts: category, subcategory, and related indicators. Additionally, according to the hospital levels, there were three categories: ‘organizational management’, ‘clinical management’, and ‘administrative management’. Then, according to the type of indicators, fifteen subcategories were defined for the 110 selected main indicators.

Performance dimensions

The ‘productivity’ dimension focuses on indicators reflecting the macro-performance of the hospital, considering that this index is more effective and efficient. The ‘efficiency’ dimension focuses on general performance indicators for the optimal use of resources to create optimal output in the hospital. The ‘effectiveness’ dimension is a general performance indicator with an outcome view. The ‘speed’ dimension focuses on the indicators that show attention to the service delivery time and the speed of the procedures. The ‘development’ dimension focuses on matters related to employees’ and students’ training and related training courses. In terms of ‘safety’ dimension, there were issues related to patient safety, unwanted and harmful events, and hospital infections.

The “quality of work life” dimension emphasizes matters related to personnel volume and work conditions. The ‘quality’ dimension is related to the quality of service provided in different parts of the hospital and possible complications in improving the quality of services. The ‘satisfaction’ dimension focuses on the satisfaction of patients, employees, and their complaints. The ‘innovation’ dimension relates to the research process and its output. The ‘appropriateness’ dimension involves proper service from clinical departments, pharmaceutical services, and patient treatment. The ‘evaluation’ dimension focuses on the indicators related to the assessment scores of the para-clinical departments of the hospital.

The ‘profitability’ dimension focuses on the overall output indicators for income and profitability. The ‘cost’ dimension focuses on indicators related to general expenditures and the average cost per bed and patient and budgeting. The ‘economy’ dimension is related to financial rates and their indicators. The ‘coherence’ dimension emphasizes the indicators related to the continuity of the service delivery process. The ‘patient-centeredness’ dimension focuses on the indicators related to the patient’s experience of the facility, environment, treatment processes, communications, and relevant support for the patient. The ‘equity’ dimension studies indicators related to social and financial justice and life expectancy. The ‘relationship’ dimension evaluates the process of consultations and discussions required during the patients’ care provided by the treatment team. The ‘sustainability’ dimension focuses on indicators related to energy standards. The ‘flexibility’ dimension focuses on the hospital’s response to the crisis.

According to Table  4 , most studies focused on ‘efficiency’, ‘productivity’, ‘safety’ and ‘effectiveness’ as performance dimensions in 54, 53, 38 and 37 studies, respectively (40–70% of studies). In the ‘efficiency’ subcategory, resource management, supportive unit assessment, and human resource management indicators were the first to third most common indicators used in 26, 23 and 22 studies, respectively (approximately 25% of the studies).

In addition, for the ‘efficiency’ dimension, ‘medical staff numbers’, ‘emergency department bed numbers’, and ‘nonmedical staff numbers’ were reported in 16, 13, and 11 studies, respectively (between 20 and 30% of the studies). For the ‘productivity’ subcategory, ‘bed utilization rate’ and ‘service delivery and treatment’ were reported in 50% and 20% of the studies, respectively (46 and 19 out of 91).

Additionally, for the ‘productivity’ dimension, the ‘length of stay’ indicator was used more than others and reported in approximately 80% of the studies (43 out of 53), followed by the ‘bed occupancy rate’ in approximately 40% of the studies (21 out of 53). The ‘bed turnover ratio’ and ‘hospitalization rate’ were also reported in 12 studies. Furthermore, for ‘safety’ dimensions, all indicators were in the ‘patient safety’ subcategory, which has been reported in 38 studies, and ‘complications’, ‘accidents or adverse events’, and ‘incidents or errors rates’ were the most concentrated indicators by researchers in 13, 12, and 11 studies, respectively. The performance dimension of ‘effectiveness’ was presented in 37 studies (40%), with only two indicators, ‘mortality rate’ in 29 studies and ‘readmission rate’ in 23 studies.

Performance categories

Considering the three categories shown in Table  4 , ‘organizational management’ indicators were more commonly used among the other two categories (‘clinical’ and ‘administrative’) and were present in more than 85% of the studies (78 out of 91). Two categories, ‘clinical management’ and ‘administrative management’, were reported in 62 and 51 studies, respectively.

Performance subcategories

Considering the 14 subcategories shown in Table  4 , both the ‘bed utilization rate’ and ‘patient safety’ indicators were mentioned in 46 studies and were more common among the other subcategories. The second most common indicator of the ‘financial management’ subcategory was reported in 38 studies. At the third level, both the ‘human resource management’ and ‘time management’ indicators were presented in 31 studies. The ‘paramedical’ subcategory indicators were presented in less than 10% of the studies [ 60 , 96 , 97 , 98 , 106 , 113 ].

Performance indicators

According to the indicator columns in Table  3 , the most used indicators in reviewed studies were the length of stay, mortality rate, and readmission rate in 47%, 32%, and 25% of studies, respectively. Bed occupancy rate and non-personnel costs were reported in 23% of studies. Additionally, among the 110 indicators, 16 indicators, namely, the lab cancellation rate, exam-physician ratios, number of coded diagnoses, number of medical records, laboratory sample/report intervals, medical information request time, safety standards in the archives, nutritional risk screening, imaging quality control failures, errors in medical reports, average impact factor, nutritional measures, laboratory scoring, imaging inspection, discharge process and emergency response rate, were reported in less than 1% of the studies.

The classification of the indicators in Table  4 was performed based on the chain model, which included the input, process, output, outcome and impact. The assignment of the indicators to each category was performed according to the experts’ opinions. For instance, the number of publications by academic member of an academic hospital and the average impact factor of those publications were considered outcome indicators. As depicted in the Table  4 , most studies (80%) focused more on output indicators. Additionally, fifteen studies focused on introducing and extracting some of the input, process, output, outcome and impact indicators; among those, only one study [ 96 ] has examined the input, process, output and impact indicators simultaneously.

Additionally, in approximately 42% (36 out of 91) of the studies, the indicators’ definitions, formulas, or descriptions have been illustrated, while less than 10% of the studies have defined measuring units, standard or benchmark units for all studied indicators [ 15 , 43 , 45 , 51 , 52 , 57 , 67 ].

Overall, nine studies related to hospital performance evaluation were conducted using systematic review methodologies (five systematic reviews [ 16 , 29 , 30 , 56 , 113 ], two literature reviews [ 79 , 80 ], one narrative review [ 98 ] and one brief review [ 92 ]). Most of these studies focused on extracting performance indicators from one or more hospital departments (e.g., the emergency department) [ 16 , 56 ], hospital laboratory and radiology information systems [ 106 ], supply chain performance [ 29 ], resources and financial results and activity [ 113 ], hospital water consumption [ 30 ], and the pharmaceutical sector [ 98 ]. Other reviews included a three-step process to review, evaluate and rank these hospital indicators in a systematic approach [ 16 ], or to evaluate performance indicator models to create an interactive network and visualize the causal relationships between performance indicators [ 79 ]; moreover, some have focused on the importance of indicators to ensure adequate coverage of the relevant areas of health care services to be evaluated [ 92 ].

Only one scoping review aimed to identify current assessments of hospital performance and compared quality measures from each method in the context of the six qualitative domains of STEEEP (safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness) of the Institute of Medicine (IOM) in accordance with Donabedian’s framework and formulating policy recommendations [ 115 ].

In addition, 21 studies divided performance indicators into 2 to 6 dimensions of performance. Also, the reviewed studies included 2–40 indicators in zero [ 29 , 30 , 98 ] to 6 domains [ 34 ]. Moreover, none of the studies have tried to comprehensively summarize and categorize the performance indicators in several categories, focusing on all the indicators reflecting the performance of the entire hospital organization, or the indicators of administrative units or clinical departments.

In this scoping review, a unique set of hospital performance evaluation indicators related to the various performance dimensions was categorized from 91 studies over the past ten years.

Similarly, in a study, 19 performance dimensions, 32 sub-dimensions, and 138 indicators were extracted from only six studies. Those dimensions were described by all studies included in the review, but only three studies specified the relevant indicators, and the list provided for all possible indicators was not comprehensive. Also, despite current review, there was no classification of indicators based on the hospital levels: managerial, clinical, or organizational levels [ 116 ]. Another study has similarly investigated the performance evaluation indicators of the hospital in such a way that among 42 studies, 111 indicators were presented in the four categories: input, output, outcome, and impact. But, there was no classification of indicators based on performance dimensions and hospital levels [ 117 ].

In this study, the importance of categorized indicators, for the first time to our knowledge, was determined based on their frequency of use in the published literature (Appendix 2 ). The ‘Organizational management’ indicators were the most common compared with the other two categories (‘clinical’ and ‘administrative’). It could be because of the fact that the indicators such as ‘bed occupancy rate’, ‘average length of stay’, ‘mortality rate’, ‘hospital infection rate’, and ‘patient safety’ are easier to be registered in hospital software compared to other indicators, and also they better reflect the overall performance of hospital. Thus, researchers are more interested in using these indicators.

Considering 14 subcategories, indicators related to three subcategories i.e. bed utilization, patient safety and financial management are the most frequent used indicators for hospital performance evaluation. It reflects the need of hospital managers to increase the profitability of hospital in one hand, and to control cost on the other hand. As a results, researchers have paid special attention to ‘cost income’, ‘profitability’, ‘economic’, etc., as indicators for evaluating hospital performance.

When considering indicators by type, more studies have focused on output indicators, while input indicators were the least common used. This might be because of the fact that at hospital level, it is difficult for managers to change those inputs such as ‘beds’, ‘human resources’, ‘equipment and facilities’. In addition, due to the complexity of interdepartmental relationships in hospitals, process indicators seemed to provide more variety for analysis than input indicators, so they were more often used. As mentioned above, output indicators were the most used indicators for hospital performance evaluation due to their ease of calculation and interpretation.

The main purpose of this paper was to identify a comprehensive set of indicators that can be used to evaluate hospital performance in various hospital settings by being distilled into a smaller and more related set of indicators for every hospital or department setting. future studies could be designed to validate each set of indicators in any specific context. In addition, they could investigate the relationship between the indicators and their outcomes of interest and the performance dimension each could address. This will enable hospital managers to build their own set of indicators for performance evaluation both at organization or at department level. Also it should be mentioned that.

Although some previous studies have provided definitions for each indicator and determined the standard criteria for them, this was not done in this study because the focus of this study was to provide a collection of all the indicators used in hospital performance evaluation, which resulted in the identification of more than a thousand indicators without limiting to specific country or context. So while preparing a smaller set of indicators, specific conditions of each country, such as the type of health system and its policy, the type of financing system, and the structure of services, should be taken into account to select appropriate indicators.

In addition, although it is important to examine the scope of each article to compare the list of indicators and the relationships between the dimensions of the hospital in terms of size and type and between the number and type of selected indicators, this was considered beyond the scope of this review due to the high number of indicators, which made the abovementioned investigations impossible. Future studies could do that while working with a smaller set of indicators.

This review aimed to categorize and present a comprehensive set of indicators for evaluating overall hospital performance in a systematic way. 1161 hospital performance indicators were drawn from 91 studies over the past ten years. They then were summarized into 110 main indicators, and categorized into three categories: 14 subcategories, and 21 performance dimensions This scoping review also highlighted the most frequent used indicators in performance evaluation studies which could reflect their importance for that purpose. The results of this review help hospital managers to build their own set of indicators for performance evaluation both at organization or at department level with regard to various performance dimensions.

As the results of this review was not limited to any specific country or context, specific conditions of each country, such as the type of health system and its policy, the type of financing system, and the structure of services, should be taken into account while selecting appropriate indicators as a smaller set of indicators for hospital performance evaluation in specific context.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Gross domestic product

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews

Emergency departments

Intensive care unit

Operating room

Surgical intensive care unit

Neonatal intensive care unit

Readmission rate

Quality Control

Medication use evaluation

safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness

Institute of Medicine

Abdullah A, Ahmad S, Athar MA, Rajpoot N, Talib F. Healthcare performance management using integrated FUCOM-MARCOS approach: the case of India. Int J Health Plann Manage. 2022;37(5):2635–68.

Article   PubMed   Google Scholar  

Pestana M, Pereira R, Moro S. Improving health care management in hospitals through a productivity dashboard. J Med Syst. 2020;44(4):87.

Amos D. A practical framework for performance measurement of facilities management services in developing countries’ public hospitals. J Facil Manag. 2022;20(5):713–31.

Article   Google Scholar  

Ahmed S, Hasan MZ, MacLennan M, Dorin F, Ahmed MW, Hasan MM, et al. Measuring the efficiency of health systems in Asia: a data envelopment analysis. BMJ Open. 2019;9(3):e022155.

Article   PubMed   PubMed Central   Google Scholar  

Mitkova Z, Doneva M, Gerasimov N, Tachkov K, Dimitrova M, Kamusheva M, et al. Analysis of healthcare expenditures in Bulgaria. Healthc. 2022;10(2):274.

Patrinely JR, Walker SH, Glassman GE, Davis MJ, Abu-Ghname A, Khan U, et al. The importance of financial metrics in physician funding and performance evaluation. Plast Reconstr Surg. 2021;147:1213–8.

Article   CAS   PubMed   Google Scholar  

Buathong S, Bangchokdee S. The use of the performance measures in Thai public hospitals. ASIAN Rev Acc. 2017;25(4):472–85.

Google Scholar  

Imani A, Alibabayee R, Golestani M, Dalal K. Key indicators affecting hospital efficiency: a systematic review. Front Public Heal. 2022;10:830102.

Mahdiyan S, Dehghani A, Tafti AD, Pakdaman M, Askari R. Hospitals’ efficiency in Iran: a systematic review and meta-analysis. J Educ Health Promot. 2019;8(1):126.

PubMed   PubMed Central   Google Scholar  

Amos D, Musa ZN, Au-Yong CP. Performance measurement of facilities management services in Ghana’s public hospitals. Build Res Inf. 2020;48(2):218–38.

Feibert DC, Andersen B, Jacobsen P. Benchmarking healthcare logistics processes–a comparative case study of Danish and US hospitals. Total Qual Manag Bus Excell. 2019;30(1–2):108–34.

Gün I, Yilmaz F, Şenel IK. Efficiency analysis of health systems in world bank countries. Arch Heal Sci Res. 2021;8(2):147–52.

Breyer JZ, Giacomazzi J, Kuhmmer R, Lima KM, Hammes LS, Ribeiro RA, et al. Hospital quality indicators: a systematic review. Int J Health Care Qual Assur. 2019;32(2):474–87.

Regragui H, Sefiani N, Azzouzi H. Improving performance through measurement: the application of BSC and AHP in healthcare organization. In: Equipe De Recherche, Ingénierie, Innovation Et Management Des Systèmes Industriels, Université Abdelmalek Saadi. Tanger, Morocco: Institute of Electrical and Electronics Engineers Inc; 2018. p. 51–6.

Ghozali MT, Latifah DN, Darayani A. Analysis of Drug Supply Management of the Pharmacy Warehouse of Prof. Dr. Soerojo Mental Health Hospital, Magelang, Indonesia. Clin Schizophr Relat Psychoses. 2021;15:1–6.

Etu EE, Monplaisir L, Aguwa C, Arslanturk S, Masoud S, Markevych I, et al. Identifying indicators influencing emergency department performance during a medical surge: a consensus-based modified fuzzy Delphi approach. PLoS ONE. 2022;17(4 April):e0265101.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lin C-Y, Shih F-C, Ho Y-H. Applying the balanced scorecard to build service performance measurements of medical institutions: An AHP-DEMATEL approach. Int J Environ Res Public Health. 2023;20(2):1022.

Backman C, Vanderloo S, Forster AJ. Measuring and improving quality in university hospitals in Canada: the collaborative for excellence in healthcare quality. Health Policy (New York). 2016;120(9):982–6.

Ghalem Â, Okar C, Chroqui R, Semma E. Performance: A concept to define. In: Performance: A concept to define. LOGISTIQUA 2016; 2016. p. 1–13.

Sonnentag S, Frese M. Performance Concepts and Performance Theory. In 2005. p. 1–25.

Tangen S. Demystifying productivity and performance. Int J Prod Perform Manag. 2005;54:34–46.

Elena-Iuliana I, Maria C. Organizational Performance – A Concept That Self-Seeks To Find Itself. Ann - Econ Ser Constantin Brancusi Univ Fac Econ. 2016;4(4):179–83.

Riratanaphong C, Van der Voordt T, Sarasoja A. Performance Measurement in the context of CREM and FM. In: Per Anker Jensen, Theo Van der Voordt CC, editor. The added value of facilities management: concepts, findings and perspectives. Lyngby Denmark: Polyteknisk Forlag; 2012. p. 1–21.

Pham M, Rajić A, Greig J, Sargeant J, Papadopoulos A, Mcewen S. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85.

Chaney M. So you want to write a narrative review article? J Cardiothorac Vasc Anesth. 2021;35:3045–9.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Tricco A, Lillie E, Zarin W, O’Brien K, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Dolatabad AH, Mahdiraji HA, Babgohari AZ, Garza-Reyes JA, Ai A. Analyzing the key performance indicators of circular supply chains by hybrid fuzzy cognitive mapping and Fuzzy DEMATEL: evidence from healthcare sector. Environ Dev Sustain. 2022;1–27.

Batista KJM, da Silva SR, Rabbani ERK, Zlatar T. Systematic review of indicators for the assessment of water consumption rates at hospitals. Water Supply. 2020;20(2):373–82.

Beta G, Role D, Berloviene D, Balkena Z. PATIENTS’ SATISFICATION AS THE QUALITY INDICATOR OF NURSING. In: Lubkina V, Kaupuzs A, Znotina D, editors. SOCIETY INTEGRATION EDUCATION, VOL VI: PUBLIC HEALTH AND SPORT, RESEARCHES IN ECONOMICS AND MANAGEMENT FOR SUSTAINABLE EDUCATION. 2020. p. 79–88.

Bhardwaj P, Joshi NK, Singh P, Suthar P, Joshi V, Jain YK, et al. Competence-based assessment of biomedical equipment management and maintenance system (e-Upkaran) using benefit evaluation framework. CUREUS J Med Sci. 2022;14(10):e30579.

Cheon O, Song M, Mccrea AM, Meier KJ. Health care in America: the relationship between subjective and objective assessments of hospitals. Int PUBLIC Manag J. 2021;24(5):596–622.

Craig KJT, McKillop MM, Huang HT, George J, Punwani ES, Rhee KB. US hospital performance methodologies: a scoping review to identify opportunities for crossing the quality chasm. BMC Health Serv Res. 2020;20(1):640.

Cristiano S, Ulgiati S, Gonella F. Systemic sustainability and resilience assessment of health systems, addressing global societal priorities: Learnings from a top nonprofit hospital in a bioclimatic building in Africa. Renew Sustain ENERGY Rev. 2021;141:110765.

Dadi D, Introna V, Santolamazza A, Salvio M, Martini C, Pastura T, et al. Private Hospital Energy Performance Benchmarking Using Energy Audit Data: An Italian Case Study. Energies. 2022;15(3):1–16.

Dawande PP, Wankhade RS, Akhtar FI, Noman O. Turnaround time: an efficacy measure for medical laboratories. CUREUS J Med Sci. 2022;14(9):e28824.

De Sousa LR, Mazzo A, De Almeida ACF, Tonello C, Lourençone LFM. Evaluation of quality indicators in the management of an operating room at a tertiary-level hospital. Med. 2022;55(1):1–8.

Drynda S, Schindler W, Slagman A, Pollmanns J, Horenkamp-Sonntag D, Schirrmeister W, et al. Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study. BMJ Open. 2020;10(9):e038776.

Fekri O, Manukyan E, Klazinga N. Appropriateness, effectiveness and safety of care delivered in Canadian hospitals: a longitudinal assessment on the utility of publicly reported performance trend data between 2012–2013 and 2016–2017. BMJ Open. 2020;10(6):e035447.

Galloa AJO, Ramírez CA. Evaluating Colombian public hospitals productivity during 2004–2015. A luenberger-indicator approach. Rev Gerenc Y Polit Salud. 2020;19:1–23.

Gebreyes M, Sisay A, Tegen D, Asnake A, Wolde M. Evaluation of laboratory performance, associated factors and staff awareness towards achieving turnaround time in tertiary hospitals, Ethiopia. Ethiop J Health Sci. 2020;30(5):767–76.

Gnanaraj J, Kulkarni RG, Sahoo D, Abhishekh B. Assessment of the Key Performance Indicator Proposed by NABH in the Blood Centre of a Tertiary Health Care Hospital in Southern India. Indian J Hematol Blood Transfus. 2022;39:308–16.

Horenberg F, Lungu DA, Nuti S. Measuring research in the big data era: the evolution of performance measurement systems in the Italian teaching hospitals. Health Policy (New York). 2020;124(12):1387–94.

Hwang DK, Cho J, Moon J. Feasibility study on energy audit and data driven analysis procedure for building energy efficiency: bench-marking in Korean hospital buildings. Energies. 2019;14(15):3006.

Jaskova D. Efficiency of management, processes in a private hospital. Entrep Sustain Issues. 2021;9(1):436–46.

Jebraeily M, Valizadeh MA, Rahimi B, Saeidi S. The development of a clinical dashboard for monitoring of key performance indicators in ICU. J Iran Med Counc. 2022;5(2):308–17.

Kang Y, Kim M, Jung K. The equity of health care spending in South Korea: testing the impact of publicness. Int J Environ Res Public Health. 2020;17(5):1775.

Abou Tarieh RR, Zayyat R, Naoufal RN, Samaha HR. A case study exploring the impact of JCI standards implementation on staff productivity and motivation at the laboratory and blood bank. Heal Sci Rep. 2022;5(1):e497.

Kadoic N, Simic D, Mesaric J, Redep NB. Measuring quality of public hospitals in croatia using a multi-criteria Approach. Int J Environ Res Public Health. 2021;18:19.

Khalilabad T, Amir N, Asl P, Raeissi Shali M, Niknam N. Assessment of clinical and paraclinical departments of military hospitals based on the Pabon Lasso Model. J Educ Health Promot. 2020;9:1–6.

Lai JHK, Hou H, Edwards DJ, Yuen PL. An analytic network process model for hospital facilities management performance evaluation. Facilities. 2022;40(5–6):333–52.

Lai J, Yuen PL. Identification, classification and shortlisting of performance indicators for hospital facilities management. Facilities. 2021;39(1–2):4–18.

Lin CS, Chiu CM, Huang YC, Lang HC, Chen MS. Evaluating the operational efficiency and quality of Tertiary hospitals in Taiwan: the application of the EBITDA Indicator to the DEA Method and TOBIT Regression. Healthcare. 2022;10(1):58.

Matos R, Ferreira D, Pedro MI. Economic analysis of portuguese public hospitals through the construction of quality, efficiency, access, and financial related composite indicators. Soc Indic Res. 2021;157(1):361–92.

Morisod K, Luta X, Marti J, Spycher J, Malebranche M, Bodenmann P. Measuring health equity in emergency care using routinely collected data: a systematic review. Heal Equity. 2021;5(1):801–17.

Nik Hisamuddin R, Tuan Hairulnizam TK. Developing key performance indicators for emergency department of teaching hospitals: a mixed fuzzy Delphi and nominal group technique approach. Malays J Med Sci. 2022;29(2):114–25.

Ramírez Calazans A, Paredes Esteban RM, Grijalva Estrada OB, Ibarra Rodríguez MR. Assessment of quality indicators in pediatric major outpatient surgery. Influence of the COVID-19 pandemic. Cir Pediatr. 2023;36(1):17–21.

PubMed   Google Scholar  

Shaqura II, Gholami M, Akbari Sari A. Assessment of public hospitals performance in Gaza governorates using the Pabón Lasso Model. Int J Health Plann Manage. 2021;36(4):1223–35.

Al-Jazairi AS, Alnakhli AO. Quantifying clinical pharmacist activities in a tertiary care hospital using key performance indicators. Hosp Pharm. 2021;56(4):321–7.

Aloh HE, Onwujekwe OE, Aloh OG, Nweke CJ. Is bed turnover rate a good metric for hospital scale efficiency? A measure of resource utilization rate for hospitals in Southeast Nigeria. Cost Eff Resour Alloc. 2020;18(1):1–8.

Bari S, Incorvia J, Ahearn O, Dara L, Sharma S, Varallo J, et al. Building safe surgery knowledge and capacity in Cambodia: a mixed-methods evaluation of an innovative training and mentorship intervention. Glob Health Action. 2021;14(1):1998996.

Bari S, Incorvia J, Iverson KR, Bekele A, Garringer K, Ahearn O, et al. Surgical data strengthening in Ethiopia: results of a Kirkpatrick framework evaluation of a data quality intervention. Glob Health Action. 2021;14(1):1–11.

Bastos LSL, Hamacher S, Zampieri FG, Cavalcanti AB, Salluh JIF, Bozza FA. Structure and process associated with the efficiency of intensive care units in low-resource settings: an analysis of the CHECKLIST-ICU trial database. J Crit Care. 2020;59:118–23.

Bastos LSL, Wortel SA, de Keizer NF, Bakhshi-Raiez F, Salluh JIF, Dongelmans DA, et al. Comparing continuous versus categorical measures to assess and benchmark intensive care unit performance. J Crit Care. 2022;70:154063.

Kocisova K, Hass-Symotiuk M, Kludacz-Alessandri M. Use of the dea method to verify the performance model for hospitals. E M Ekon A Manag. 2018;21(4):125–40.

Lee D, Yu S, Yoon SN. Analysis of hospital management based on the characteristics of hospitals: focusing on financial indicators. Glob Bus Financ Rev. 2019;24(3):1–13.

Mirzaei A, Tabibi SJ, Nasiripour AA, Riahi L. Evaluating the feasibility of financial variables of health: A hospital administrator’s viewpoint. Galen Med J. 2016;5(1):25–30.

Middleton S, Gardner G, Gardner A, Considine J, FitzGerald G, Christofis L, et al. Are service and patient indicators different in the presence or absence of nurse practitioners? The EDPRAC cohort study of Australian emergency departments. BMJ Open. 2019;9(7):e024529.

Nobakht S, Jahangiri K, Hajinabi K. Correlation of performance indicators and productivity: A cross sectional study of emergency departments in Tehran, Iran during year 2016. Trauma Mon. 2018;23(5):1–6.

Nuti S, Grillo Ruggieri T, Podetti S. Do university hospitals perform better than general hospitals? A comparative analysis among Italian regions. BMJ Open. 2016;6(8):e011426.

Petrovic GM, Vukovic M, Vranes AJ. The impact of accreditation on health care quality in hospitals. Vojnosanit Pregl. 2018;75(8):803–8.

Pirani N, Zahiri M, Engali KA, Torabipour A. Hospital efficiency measurement before and after health sector evolution plan in Southwest of Iran: a DEA-panel data study. Acta Inf Med. 2018;26(2):106–10.

Profit J, Gould JB, Bennett M, Goldstein BA, Draper D, Phibbs CS, et al. The association of level of care with NICU quality. Pediatrics. 2016;137(3):44–51.

Rahimi H, Bahmaei J, Shojaei P, Kavosi Z, Khavasi M. Developing a strategy map to improve public hospitals performance with balanced scorecard and dematel approach. Shiraz E Med J. 2018;19(7):1–12.

Ahmed S, Hasan MZ, Laokri S, Jannat Z, Ahmed MW, Dorin F, et al. Technical efficiency of public district hospitals in Bangladesh: a data envelopment analysis. COST Eff Resour Alloc. 2019;17:17.

Rahman MH, Tumpa TJ, Ali SM, Paul SK. A grey approach to predicting healthcare performance. Meas J Int Meas Confed. 2019;134:307–25.

Sajadi HS, Sajadi ZS, Sajadi FA, Hadi M, Zahmatkesh M. The comparison of hospitals’ performance indicators before and after the Iran’s hospital care transformations plan. J Educ Health Promot. 2017;6:89.

Si S-L, You X-Y, Liu H-C, Huang J. Identifying key performance indicators for holistic hospital management with a modified DEMATEL approach. Int J Environ Res Public Health. 2017;14(8): 934.

Váchová L, Hajdíková T. Evaluation of Czech hospitals performance using MCDM methods. In: A SI, G WS, C D, editors. Department of exact methods, faculty of management, university of economics, Prague, Jarošovská 1117, Jindřichuv Hradec, vol. 37701. Czech Republic: Newswood Limited; 2017. p. 732–5.

Xenos P, Yfantopoulos J, Nektarios M, Polyzos N, Tinios P, Constantopoulos A. Efficiency and productivity assessment of public hospitals in Greece during the crisis period 2009–2012. Cost Eff Resour Alloc. 2017;15(1):6.

Zhang L, Liu R, Jiang S, Luo G, Liu H-C. Identification of key performance indicators for hospital management using an extended hesitant linguistic DEMATEL Approach. Healthc (Basel Switzerland). 2019;8(1):7.

Aksezer CS. A nonparametric approach for optimal reliability allocation in health services. Int J Qual Reliab Manag. 2016;33(2):284–94.

Cagliano AC, Grimaldi S, Rafele C. Assessing warehouse centralization and outsourcing in the healthcare sector: an Italian case study. In: Department of Management and Production Engineering, Politecnico Di Torino, Corso Duca Degli Abruzzi 24, Torino, 10129. Italy: AIDI - Italian Association of Industrial Operations Professors; 2017. p. 244–50.

Cefalu MS, Elliott MN, Setodji CM, Cleary PD, Hays RD. Hospital quality indicators are not unidimensional: a reanalysis of Lieberthal and Comer. Health Serv Res. 2019;54(2):502–8.

Gao H, Chen H, Feng J, Qin X, Wang X, Liang S, et al. Balanced scorecard-based performance evaluation of Chinese county hospitals in underdeveloped areas. J Int Med Res. 2018;46(5):1947–62.

Gonnelli V, Satta F, Frosini F, Iadanza E. Evidence-based approach to medical equipment maintenance monitoring. In: V HEO, V J, editors. University of Florence, Dept. of Information Engineering. Florence, Italy: Springer; 2017. p. 258–61.

Helkio P, Aantaa R, Virolainen P, Tuominen R. Productivity benchmarks for operative service units. ACTA Anaesthesiol Scand. 2016;60(4):450–6.

Khalifa M, Zabani I. Developing emergency room key performance indicators: What to measure and why should we measure it? J. M, A. H, P. G, A. K, M.S. H, editors. Vol. 226. King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia: IOS Press BV; 2016. p. 179–182.

Ajami S, Ebadsichani A, Tofighi S, Tavakoli N. Medical records department and balanced scorecard approach. J Educ Health Promot. 2013;2:7.

Bosse G, Mtatifikolo F, Abels W, Strosing C, Breuer J-P, Spies C. Immediate outcome indicators in perioperative care: a controlled intervention study on quality improvement in hospitals in Tanzania. PLoS One. 2013;8(6):e65428.

Hung K-Y, Jerng J-S. Time to have a paradigm shift in health care quality measurement. J Formos Med Assoc. 2014;113(10):673–9.

Jeejeebhoy KN, Keller H, Gramlich L, Allard JP, Laporte M, Duerksen DR, et al. Nutritional assessment: comparison of clinical assessment and objective variables for the prediction of length of hospital stay and readmission. Am J Clin Nutr. 2015;101(5):956–65.

Kittelsen SAC, Anthun KS, Goude F, Huitfeldt IMS, Häkkinen U, Kruse M, et al. Costs and quality at the hospital level in the nordic countries. Heal Econ (United Kingdom). 2015;24:140–63.

Koné Péfoyo AJ, Wodchis WP. Organizational performance impacting patient satisfaction in Ontario hospitals: a multilevel analysis. BMC Res Notes. 2013;6: 509.

Li CH, Yu CH. Performance evaluation of public non-profit hospitals using a BP Artificial neural network: the case of Hubei Province in China. Int J Environ Res Public Health. 2013;10(8):3619–33.

Liu K, Jain S, Shi J. Physician performance assessment using a composite quality index. Stat Med. 2013;32(15):2661–80.

Lloyd GF, Bajorek B, Barclay P, Goh S. Narrative review: Status of key performance indicators in contemporary hospital pharmacy practice. J Pharm Pract Res. 2015;45(4):396–403.

Mehrtak M, Yusefzadeh H, Jaafaripooyan E. Pabon Lasso and data envelopment analysis: a complementary approach to hospital performance measurement. Glob J Health Sci. 2014;6(4):107–16.

Mohammadi M, Ziapoor A, Mahboubi M, Faroukhi A, Amani N, Pour FH, et al. Performance evaluation of hospitals under supervision of Kermanshah medical sciences using pabonlasoty diagram of a five-year period (2008–2012). Life Sci J. 2014;11:77–81 ( 1 SPECL. ISSUE) ).

Niaksu O, Zaptorius J. Applying operational research and data mining to performance based medical personnel motivation system. In: Vilnius University, Institute of Mathematics and Informatics. Lithuania: IOS; 2014. p. 63–70.

Córdoba S, Caballero I, Navalón R, Martínez-Sánchez D, Martínez-Morán C, Borbujo J. Analysis of the surgical activity in the dermatology department of Fuenlabrada University Hospital, Madrid, Spain, between 2005 and 2010: determination of the standard operating times. Actas Dermosifiliogr. 2013;104(2):141–7.

Profit J, Kowalkowski MA, Zupancic JAF, Pietz K, Richardson P, Draper D, et al. Baby-MONITOR: a composite indicator of NICU Quality. Pediatrics. 2014;134(1):74–82.

Rabar D, Pap N. Evaluation of crotia’s regional hospital effiency: an application of data envelopment analysis . Bacher U, Barkovic D, Dernoscheg KH, LamzaMaronic M, Matic B, Runzheimer B, editors. Interdisciplinary Management Research IX. 2013;9:649–59.

Ramos MCA, da Cruz LP, Kishima VC, Pollara WM, de Lira ACO, Couttolenc BF. Performance evaluation of hospitals that provide care in the public health system, Brazil. Rev Saude Publica. 2015;49:1–9.

Schuers M, Joulakian MB, Griffon N, Pachéco J, Périgard C, Lepage E, et al. In: S IN, de PM AM, editors. Quality indicators from laboratory and radiology information systems. A. G. Volume 216. France: IOS; 2015. pp. 212–6. Department of Biomedical Informatics, Rouen University Hospital, Rouen Cedex, 76031,.

Tabrizi JS, Saadati M, Sadeghi-Bazargani H, Ebadi A, Golzari SEJ. Developing indicators to improve educational governance in hospitals. Clin Gov. 2014;19(2):117–25.

Costa A Jr, aS., Leão LE, Novais MA, Zucchi P. An assessment of the quality indicators of operative and non-operative times in a public university hospital. Einstein (Sao Paulo). 2015;13(4):594–9.

Coyne JS, Helton J. How prepared are US hospitals for the affordable care act? A financial condition analysis of US hospitals in 2011. J Health Care Finance. 2015;41(3).

Davis P, Milne B, Parker K, Hider P, Lay-Yee R, Cumming J, et al. Efficiency, effectiveness, equity (E-3). Evaluating hospital performance in three dimensions. Health Policy (New York). 2013;112(1–2):19–27.

Flatow VH, Ibragimova N, Divino CM, Eshak DSA, Twohig BC, Bassily-Marcus AM, et al. Quality outcomes in the surgical intensive care unit after electronic health record implementation. Appl Clin Inf. 2015;6(4):611–8.

Article   CAS   Google Scholar  

Fonseca JRS, Ramos RMP, Santos AMP, Fonseca APSS. Policy effects on the quality of public health care: evaluating Portuguese public hospitals’ quality through customers’ views. Cent Eur J Public Policy. 2015;9(2):122–40.

Hadji B, Meyer R, Melikeche S, Escalon S, Degoulet P. Assessing the Relationships Between Hospital Resources and Activities: A Systematic Review. J Med Syst. 2014;38(10):1–21.

Hajduová Z, Herbrik G, Beslerová S. Application of DEA in the environment of Slovak hospitals. Invest Manag Financ Innov. 2015;12(4):148–53.

Thomas Craig KJ, McKillop MM, Huang HT, George J, Punwani ES, Rhee KB. U.S. hospital performance methodologies: a scoping review to identify opportunities for crossing the quality chasm. BMC Health Serv Res. 2020;20(1):640.

Carini E, Gabutti I, Frisicale EM, Di Pilla A, Pezzullo AM, de Waure C, et al. Assessing hospital performance indicators. What dimensions? Evidence from an umbrella review. BMC Health Serv Res. 2020;20(1):1038.

Rasi V, Delgoshaee B, Maleki M. Identification of common indicators of hospital performance evaluation models: a scoping review. J Educ Health Promot. 2020;9(1):63.

Xenos P, Yfantopoulos J, Nektarios M, Polyzos N, Tinios P, Constantopoulos A. Efficiency and productivity assessment of public hospitals in Greece during the crisis period 2009–2012. COST Eff Resour Alloc. 2017;15:15.

Shaqura II, Gholami M, Sari AA. Evaluation of performance at Palestinian public hospitals using Pabon Lasso model. Int J Health Plann Manage. 2021;36(3):896–910.

Li J, Seale H, Ray P, Wang Q, Yang P, Li S, et al. E-Health preparedness assessment in the context of an influenza pandemic: a qualitative study in China. BMJ Open. 2013;3(3):e002293.

Huang C-Y, Lai C-H. Effects of internal branding management in a hospital context. Serv Ind J. 2021;41(15–16):985–1006.

Download references

Acknowledgements

The authors are grateful for the support of the Vice Chancellor for Research of Isfahan University of Medical Sciences.

The present article is part of the result of a doctoral thesis approved by Isfahan University of Medical Sciences with code 55657 (IR.MUI.NUREMA.REC.1401.005), without financial source.

Author information

Authors and affiliations.

Student Research Committee, School of Management and Medical Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran

Shirin Alsadat Hadian

Health Management and Economics Research Center, Isfahan University of Medical Sciences, Isfahan, Iran

Reza Rezayatmand

Hospital Management Research Center, Health Management Research Institute, Iran University of Medical Sciences, Tehran, Iran

Nasrin Shaarbafchizadeh

Department of Management, Faculty of Administrative Sciences and Economics, University of Isfahan, Isfahan, Iran

Saeedeh Ketabi

School of Public Health and Preventive Medicine, Monash University, Victoria, Australia

Ahmad Reza Pourghaderi

You can also search for this author in PubMed   Google Scholar

Contributions

Shirin Alsadat Hadian and Reza Rezayatmans and Saeedeh Ketabi: Study conceptualization and design. Acquisition of data: Shirin Alsadat Hadian, Reza Rezayatmand. Analysis and interpretation of data: Shirin Alsadat Hadian, Reza Rezayatmand, Nasrin Shaarbafchizadeh, Saeedeh Ketabi. Drafting of the manuscript: Shirin Alsadat Hadian, Reza Rezayatmand. Critical revision of the manuscript for important intellectual content: Reza Rezayatmand, Nasrin Shaarbafchizadeh, Saeedeh Ketabi, Ahmad Reza Pourghaderi.

Corresponding author

Correspondence to Reza Rezayatmand .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hadian, S.A., Rezayatmand, R., Shaarbafchizadeh, N. et al. Hospital performance evaluation indicators: a scoping review. BMC Health Serv Res 24 , 561 (2024). https://doi.org/10.1186/s12913-024-10940-1

Download citation

Received : 03 January 2024

Accepted : 02 April 2024

Published : 01 May 2024

DOI : https://doi.org/10.1186/s12913-024-10940-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Performance evaluation

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Digital Health Device Collection
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives

Systematic Reviews

• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations
• What is a Systematic Review?

Types of Reviews

• Manuals and Reporting Guidelines
• Our Service
• 1. Assemble Your Team
• 2. Develop a Research Question
• 3. Write and Register a Protocol
• 4. Search the Evidence
• 5. Screen Results
• 6. Assess for Quality and Bias
• 7. Extract the Data
• 8. Write the Review
• Additional Resources
• Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

• Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project. 

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: What is a Systematic Review?
• Next: Manuals and Reporting Guidelines >>
• Last Updated: May 9, 2024 11:55 AM
• URL: https://guides.mclibrary.duke.edu/sysreview
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Welcome to the Purdue Online Writing Lab

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

The Online Writing Lab at Purdue University houses writing resources and instructional material, and we provide these as a free service of the Writing Lab at Purdue. Students, members of the community, and users worldwide will find information to assist with many writing projects. Teachers and trainers may use this material for in-class and out-of-class instruction.

The Purdue On-Campus Writing Lab and Purdue Online Writing Lab assist clients in their development as writers—no matter what their skill level—with on-campus consultations, online participation, and community engagement. The Purdue Writing Lab serves the Purdue, West Lafayette, campus and coordinates with local literacy initiatives. The Purdue OWL offers global support through online reference materials and services.

A Message From the Assistant Director of Content Development 

The Purdue OWL® is committed to supporting  students, instructors, and writers by offering a wide range of resources that are developed and revised with them in mind. To do this, the OWL team is always exploring possibilties for a better design, allowing accessibility and user experience to guide our process. As the OWL undergoes some changes, we welcome your feedback and suggestions by email at any time.

Please don't hesitate to contact us via our contact page  if you have any questions or comments.

All the best,

Social Media

Facebook twitter.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 06 May 2024

Robots, firm relocation, and air pollution: unveiling the unintended spatial spillover effects of emerging technology

• Yanying Wang 1 , 2 &
• Qingyang Wu 3  

Humanities and Social Sciences Communications volume  11 , Article number:  577 ( 2024 ) Cite this article

113 Accesses

2 Altmetric

Metrics details

• Development studies
• Environmental studies

Amidst the global upsurge in industrial robot deployment, there remains a notable gap in our understanding of their environmental impact. This paper explores how the introduction of industrial robots has changed air quality at both the local and neighborhood levels in China. Using the Spatial Durbin Model, we investigate the regional spillovers of PM 2.5 concentration and the diffusion of this innovative technology. Our findings reveal that the rise of robots significantly reduces air pollution in the local area, while exacerbating it in neighboring regions. This contrast is mainly because pollution-intensive industries are more inclined to relocate to neighboring regions than their cleaner counterparts, after the local use of robots increases. Throughout the process, internal costs rather than external costs dominate firms’ relocation decisions. This study provides novel insights into the complex environmental externalities associated with the spread of industrial robots and highlights the critical issue of growing environmental inequality in the era of emerging technologies.

Similar content being viewed by others

Impact of industrial robots on environmental pollution: evidence from China

The impact of industrial robot adoption on corporate green innovation in China

Automation and labour market inequalities: a comparison between cities and non-cities, introduction.

Among all the emerging technologies, the development of industrial robots stands out significantly. Since the 1990s, robotics technology has made significant advancements. In recent years, a number of developing nations have capitalized on the wave of the ongoing Industrial 4.0 revolution (Kozul-Wright, 2016 ). This trend has facilitated a rapid dissemination of robotic utilization across developing countries, exhibiting a nearly tenfold increase from 2000 to 2015. This increase is significantly faster than the growth seen in developed countries (World Bank, 2021). As shown in Fig. 1 , by the end of 2019, the global inventory of industrial robots amounted to 2.7 million units, with applications spanning various sectors such as electronics, logistics, chemicals, and processing and manufacturing. Notably, China has emerged as the leading purchaser of industrial robots starting from 2013, reflecting its remarkable economic growth over the past four decades.

Note: calculated and made by the author.

The widespread adoption of industrial robots is expected to alter economic structure (Aghion et al. 2019 ) and enhance productivity (Acemoglu and Restrepo, 2020 ; Dauth et al. 2021 ; Gihleb et al. 2022 ). However, the potential for harmonizing economic growth and environmental improvement remains uncertain. While numerous studies have explored the influence of technological progress on environmental quality, findings have been inconsistent. Specifically, research on the environmental effects of industrial robots is limited, often focusing on the average effects rather than specific impacts (Luan et al. 2022 ). Given the potential disparities in global technological progress, the environmental externalities of uneven diffusion of new technologies should be well studied.

This paper attempts to fill this gap by investigating whether the use of industrial robots generates environmental externalities from a spatial perspective. We argue that the introduction of robots could reshape the uneven distribution of production efficiency across different regions (Luan et al. 2022 ). Greater robot penetration may improve local air quality by directly improving industrial structures, increasing energy efficiency, and reducing local pollution abatement costs, which is identified as the first environmental externality (Wang et al. 2022 ; Li et al. 2023 ; Wu, 2023 ). Furthermore, a regional increase in robot usage can also make former high-polluting industries uncompetitive locally, prompting physical or operational relocation to other regions with less emerging technology coverage (Acemoglu and Restrepo, 2020 ). This scenario represents the second environmental externality.

It is particularly important for China to discuss these environmental externalities in depth. China has experienced significant regional imbalances alongside notable economic growth. In the case of industrial robots, China’s industrial robots are mainly concentrated in the economically developed coastal regions. At the same time, air quality varies widely from region to region. As society develops, people’s demand for health and environmental comfort increases, while air pollution remains the most important environmental risk factor contributing to the burden of disease, disproportionately impacting vulnerable groups in China. It is therefore important to understand whether and to what extent this emerging technology contributes to uneven pollution distribution in China.

We use the Spatial Durbin Model (SDM) to test the whether the rising industrial robots in one area would reduce local air pollution while exacerbate it in neighboring areas. We transformed industry-level robot exposure data from the International Federation of Robotics (IFR) into a measure of robot penetration rate in Chinese cities. The spatial distance between cities, and the PM2.5 concentration of the cities were also mapped out. The results suggest a negative and significant relationship between robot exposure and local PM2.5 concentration, indicating that the adoption of industrial robots was accompanied by a decrease in PM2.5 concentration by about 6.89 μg/m 3 during 2006 to 2019. However, considering the possibility of the pollution nearby transfer, the net effect of industrial robots on overall air quality may not be as straightforward. There is a positive spillover effect of robot use on PM2.5 concentrations in neighboring areas, suggesting that the increasing exposure to robots in the local area actually “pollutes” their neighborhood. Through rigorous analysis, alternative explanations were discounted. By revealing the complexity of environmental consequences associated with increased robot exposure, our study provides insights into equitable and sustainable policies in an era of emerging technologies.

This study fills in a significant gap in existing literature by offering a novel and comprehensive understanding of the complex environmental ramifications of emerging technologies. Previous research has focused on robots’ effects on carbon emissions, but the nuanced spillover effects on air pollution and their micro-mechanisms have been largely underexplored. Our paper introduces the concept of two-sided externalities from a micro perspective by examining the influence of technological advancements on corporate investment and location decisions using Chinese data samples. In addition, we delve into the theoretical underpinnings of industrial robots’ impact on environmental pollution and empirically validate two primary mechanisms: the industrial structure effect and the internal cost effect. Furthermore, our findings suggest that emerging technologies may intensify environmental inequality, supporting observations that developing countries bear a disproportionate burden of environmental pollution.

In addition to the above-mentioned, this study also contributes to numerous empirical literatures on assessing the environmental impact of industrial robots (Liu et al. 2022 ; Ding et al. 2023 ), the drivers of environmental inequality (Bell and Ebisu, 2012 ; Zheng et al. 2022 ), the analysis of externalities using spatial econometrics (Anselin, 2003 ; Autant‐Bernard and LeSage, 2011 ; Wu et al. 2023 ; Chiu et al. 2024 ), and the study of pollution industry transfer dynamics (Ben Kheder and Zugravu, 2012 ; Dou and Han, 2019 ).

The rest of the paper is organized as follows. Section “Literature review and research hypothesis” presents a theoretical model, formulates the hypotheses, and summarizes the two-sided externalities of industrial robots on air pollution. The data descriptions, variable definitions, and spatial econometric specifications are described in Section “Materials and methods”. Section “Empirics results” presents the empirical results with robustness checks. Further discussion on the space-time characteristics of industrial robot induced pollution near transfer (IR-PNT), with the embodied mechanism, is presented in Section “Further discussion on IR-PNT”. The last section concludes the paper.

Literature review and research hypothesis

This study focuses on the environmental externalities of industrial robots in China from a spatial perspective, including both local and neighboring impacts. Recognizing the multifaceted ways in which emerging technologies influence air quality, we analyze these impacts as two-sided externalities. First, industrial robots directly affect local air quality by improving the overall industrial structure of the local area, increasing energy efficiency, and reducing the cost of pollution control. Second, the diffusion of this technology may indirectly worsen the ambient quality of the neighborhood by facilitating the relocation of pollution-intensive business (Chun et al. 2015 ). The specific mechanism is discussed below.

First externality: cleaning the air

It is supposed that the industrial robots have a direct impact on local air quality by improving the local industrial structures, boosting energy efficiency, and reducing pollution control expenses.

First, the introduction of industrial robots can lead to an advanced industrial structure, which is widely acknowledged for promoting cleaner production and reducing pollutant emissions (Gao and Yuan, 2021 ). The incorporation of industrial robots facilitates knowledge and technological spillovers, accelerating the development of new technologies and products across diverse sectors, thereby reshaping the industrial landscape. As supported by a growing body of research, the introduction of robots plays a pivotal role in industrial upgrading, fostering industrialization and modernization (Hägele et al. 2016 ; Jung and Lim, 2020 ). On a detailed technological level, the application of industrial robots exhibits a dual impact on labor demand, characterized by substitution and heterogeneity. Automation primarily targets the replacement of repetitive tasks and positions typically held by low-skilled workers, consequently shifting the industrial structure towards a more capital-intensive configuration (Graetz and Michaels, 2018 ). Aghion et al. ( 2017 ) demonstrated how the integration of artificial intelligence and traditional production methods in different industrial sectors influences the evolution of industrial structure. This effect is dominant in the manufacturing sector, as the application of industrial robots promotes the expansion of this very manufacturing sector, where the use of industrial robots also stimulates growth in related service industries through the manufacturing scale effect (Acemoglu and Restrepo, 2020 ).

Second, the integration of robots in the initial stages of the production process can enhance the energy transition and boost energy efficiency, thereby reducing enterprises’ pollution emissions. China faces significant environmental pollution problems resulting from inefficient energy consumption (Zhang et al. 2013 ). Energy inefficiencies are prevalent across various stages, including development, processing, conversion, transmission, distribution, and end-use. Emerging technologies offer a pathway to the energy transition by bolstering the efficacy of energy systems and reducing transition costs (Bocca et al. 2021 ). They can also accelerate the adoption and application of renewable energy sources and equipment. For example, the application of robots in enterprises can optimize technological processes of coal combustion or encourage the adoption of cleaner energy alternatives, subsequently enhancing energy efficiency and reducing pollution emissions (Sheng and Bu, 2022 ). Overall, by facilitating the energy transition at the production level, industrial robots reduce pollution emissions.

In addition, the clean effect of robotics can be further explained from a microscopic perspective through the variation of abatement costs. For example, industrial robots can reduce material losses in manufacturing and supply chain operations. They also facilitate the implementation of digital environmental monitoring and accounting systems, enabling efficient measurement, reporting, and verification of environmental impacts. Concepts like smart recycling systems, which repurpose waste into high-quality reusable materials, further illustrate this point (Dusík et al. 2018 ; Wilts et al. 2021 ). Consequently, the adoption of industrial robots is associated with reduced abatement costs (Wang and Feng, 2022 ). However, abatement costs are not constant across firms. In regions where the majority of firms do not utilize industrial robots, abatement costs will be relatively higher due to less efficient in terms of profitability and resource efficiency. Faced with this disparity, companies will generally consider whether to leave or stay according to the level of abatement costs (Shen et al. 2017 ). They may either invest in eco-friendly technologies to manage their emissions or relocate robots to areas with lower relative emission costs to gain operational and abatement advantages. This decision-making process is further detailed in the following subsection (see the next subsection for details).

Moreover, these effects demonstrate variability linked to local developmental stages and industrial configurations. In some contexts, the rebound effect of robot usage on environment have been found in nations of different development stages and industrial structures (see, e.g., Li et al. 2022 ; Luan et al. 2022 ; Zhu et al. 2023 ). For instance, sectors such as manufacturing, agriculture, and utilities, including electricity, gas, and water supply, the deployment of industrial robots markedly enhances the efficiency of energy equipment. This leads to a notable reduction in emissions, especially in high-pollution enterprises and those in regions with more relaxed environmental regulations, as identified by Zhu et al. ( 2023 ). Therefore, Proposition 1 can be summarized as follows.

Proposition 1: The diffusion of industrial robots could generally improve local air quality, but the impact may vary due to heterogeneity in regional development stages and industrial structures.

Second externality: industrial robot induced pollution nearby transfer (IR- PNT)

From a micro perspective, technological progress not only drives local enterprises to further innovate and reduce pollution emissions, but also affects the location decisions of enterprises. According to the new economic geography theory, technological progress is a critical determinant of the location of firms (Fujita and Krugman, 2004 ), while migration (in a broad sense, including the relocation in the physical sense and transfer of heavy pollution production business) is one of the main strategies with which firms respond to the penetration of new technologies (Pellenbarg et al. 2002 ; Aduba and Asgari, 2020 ). Then, intuitively, this technological advancement intuitively reshapes the uneven distribution of production efficiency across regions (Florida et al. 2008 ). A higher local adoption rate of emerging technologies corresponds to an increased likelihood of the relocation of highly polluting firms, which tend to have lower production capacities, to neighboring areas (Milani, 2017 ). Subsequently, the movement of these polluting entities on a micro level result in a substantial reduction of local air pollution. However, this also gives rise to a contrasting trend in neighboring regions, where the risk of environmental degradation emerges (Bommer, 1999 ). Realistic examples confirm the possibility of the existence of this effect: in Langfang and Zunhua, Hebei, which are adjacent to Beijing, the share of value added in the secondary sector is still rising from 2014 to 2017.

In other words, the introduction of industrial robots will also lead to the so-called “Baumol’s disease”, where the application of new technologies changes the local industrial layout and factor endowment, leading to huge variations in the competitive advantages of different regions (Bartelsman and Doms, 2000 ; Juhász et al. 2020 ; Benhabib et al. 2021 ). Even if only a limited number of firms adopt industrial robots in their operations in a given area, the rest of the firms can more easily learn from the “forerunners” and thus benefit much more from the positive externality of knowledge diffusion through the industrial chain and business ecosystem (Aghion et al. 2017 ). In this way, the adoption of industrial robots in a certain region can affect the costs and returns of firms, which can further encourage firms to innovate or relocate.

Next, a theoretical illustration of this view is provided. The model proposed by Levinson and Taylor ( 2008 ) and Shen et al. ( 2017 ) is extended to a multi-regional model, where the penetration of industrial robots is considered as the dominant factor influencing the variation in production costs, as it determines whether it is feasible and realistic for a firm to implement automation in its production, as well as the significant savings in labor costs that can be achieved by replacing humans with robots (Soergel, 2015 ; Graetz and Michaels, 2018 ).

Three regions, numbered “I”, “II”, “III” are considered. The model is partial equilibrium, and the price of production factors and is exogenous. Without loss of generality, we have made the following three assumptions: From a regional perspective, the production cost is greatly reduced due to the improvement in productivity; 2) For a representative company without industrial robots, the relative cost of emitting pollutants is χ I  > χ II  > χ III . The introduction of robots by other firms can increase the relative cost of pollution for firms without robots; 3) The demand of investment in abatement technologies is generally higher for pollution-intensive enterprises. The intuition is that the average productivity of enterprises will increase after the popularization of industrial robots. But for representative enterprises without robots, it is equivalent to losing their relative productivity and facing a higher cost of emitting pollutants. In the initial state, the continuum of industries of region I is indexed by \(\nu \in [0,1]\) while there are no industries in regions II and III; The pollution intensity of industry \(\nu\) is \(\sigma (\nu )\) and \({\sigma }^{{\prime} }(\nu ) > 0,0 < \sigma (\nu ) < 1\) . The production cost per unit product in regions are \({c}_{I},{c}_{{II}},{c}_{{III}}\) , and \({c}_{I} < {c}_{{II}} < {c}_{{III}}\) . Labor and capital flow freely between regions, and the relocation costs of fixed assets should be taken into consideration when industry \(\nu\) in region I moves to regions II and III; The distance between region I and region III is greater than that between region II and region III, so the production cost per unit product of industry \(\nu\) in region II is higher.

θ is the share of enterprises’ inputs used in environmental-friendly technologies. The production function \(F\left(K\left(\nu \right),L\left(\nu \right)\right)\) satisfies the constant returns to scale (CRTS). The output \(Q(\nu )\) and the pollution discharge \(E(\nu )\) are as follows:

The representative enterprise in region I aims at the maximum profit by choosing θ :

The first order derivative w.r.t θ is:

This is consistent with our intuition above that the greater the penetration of local industrial robots, the greater the pollution abatement efforts of enterprises in the area, thus improving air quality.

Based on Eq. ( 1 ), the production function with pollution emission and finished products as input factors can be written as:

As \({{\rm{\chi }}}_{I}\) and \({c}_{I}\) are the shadow prices of pollution emissions and production, respectively. The first order derivative w.r.t \(\nu\) is:

For the same reason, the unit costs of enterprise in regions II and III are:

The condition for an enterprise to relocate from region I to region II is:

Suppose an interior threshold industry level \(\widetilde{v}\) meets \({C}_{{II}}(\widetilde{v})={C}_{I}(\widetilde{v})\) :

As \({\sigma }^{{\prime} }(v) > 0,0 < \sigma (v) < 1\) , the EQ. (7) holds under \(v > \widetilde{v}\) , suggesting that the rational decision for enterprise in region I is to relocate to region II.

And Eq. ( 9 ) show the negative association between \(\widetilde{v}\) and \({{\rm{\chi }}}_{I}\) :

Hence the following Propositions 2a and 2b is straightforward:

Proposition 2a: Considering the penetration of industrial robots, the pollution-intensive enterprises would choose either to innovate locally or to relocate to the neighborhood; moreover, the greater the comparative disadvantage of the pollution-intensive enterprises, the more likely they are to choose to relocate to the neighborhood.

Proposition 2b: As the penetration of industrial robots increases, the likelihood that highly polluting enterprises will relocate nearby will increase, resulting in worse air quality in the neighborhood (IR-PNT effect).

Extend the conclusion to three areas. Consider the interior threshold industries \(\overline{\overline{v}}\) meets \({C}_{I}(v) < {C}_{II}(\overline{\overline{v}})={C}_{III}(\overline{\overline{v}})\) :

As \({{\rm{\chi }}}_{{II}}\)  >  \({{\rm{\chi }}}_{{III}}\) , \({\sigma }^{{\prime} }\left(v\right) > 0,\,0 < \sigma (v) < 1\) , the enterprise would like to move again from region II to region III.

In practice, a company’s decision to relocate relies heavily on an overall cost-benefit analysis. The costs of relocation can generally be divided into external and internal costs. For example, the external cost of firm relocation usually includes transportation links and information infrastructure, which are crucial for maintaining connectivity and operational efficiency post-relocation (Audirac, 2005 ); Internal costs, on the other hand, encompass aspects like the reallocation of current assets and employee wages, which significantly influence a firm’s financial capabilities and willingness to relocate (Pennings and Sleuwaegen, 2000 ). Additionally, the distance for relocation emerges as a pivotal factor, influencing both the feasibility and cost-effectiveness of the move. This aspect forms the basis for the spatial econometric models used in our analysis (Ben Kheder and Zugravu, 2012 ). Based on the conclusion of this part of the analysis, Propositions 3a and 3b can be proposed as follows:

Proposition 3a: The IR-PNT effect is influenced by geographical distance. Firms tend to relocate to closer areas due to lower overall migration costs, resulting in a more pronounced concentration of industrial activities in nearby regions, and thereby intensifying the IR-PNT effect.

Proposition 3b: The heterogeneity of the IR-PNT effect is significantly shaped by the varying levels of external and internal migration costs faced by local firms. These costs influence firm decisions on relocation, affecting the distribution and intensity of the IR-PNT effect across different regions.

Materials and methods

Data sources.

The raw data is obtained from the IFR, which contains the number of robots by industry, country and year, based on the annual surveys of robot suppliers. The data of China used in this paper is extracted from 2006 to 2019, because the robot industry in China was undeveloped in the 90s and its boom started in 2006, the industry-specific data was only provided in 2006. Therefore, the consistent data of robots in 7 broad industries (roughly at the two-digit level), i.e., agriculture, forestry, and fishing; mining; manufacturing; utilities; construction; education, research, and development; and non-specific industries, are adopted. In manufacturing, consistent data on the use of robots are collected for 11 industries for more detail (roughly at the 3-digit level): food and beverage, plastics and chemical products, rubber and plastic products (non-automotive), fabricated metal products (non-automotive), electrical machinery, electronic components/devices, semiconductors, liquid crystal display (LCD), light-emitting diode (LED), automotive, motor vehicles, engines and bodies, unspecified auto parts, and other vehicles.

In terms of outcome variables, surface PM2.5 data from van Donkelaar et al. ( 2021 ), who lead an annual global satellite-based estimate combining the instruments of NASA MODIS, MISR, and SeaWIFS and calibrated using a geographically weighted regression, are used to construct measures of annual PM2.5 concentration for each prefecture-level city (For details, see: https://sites.wustl.edu/acag/datasets/surface-pm2-5/ ).

To control for the potentially confounding factors that are not of interest here, information from the China Statistical Yearbook, the China Energy Statistics Yearbook, the China Environmental Statistics Yearbook, and the China Energy Statistics Yearbook is collected, and a set of economic and environmental indicators are obtained to primarily mitigate the bias from omitted variables (Wu et al. 2023 ; Jiang et al. 2024 ). See Table 1 for the detailed definition of the covariates.

Indicators of exposure to robots

We focus on the industrial robot use at the city level and mainly use the IFR dataset. As the most detailed data source available on industrial robots, IFR data have been popular in many studies such as those by Giuntella and Wang ( 2019 ), Anelli et al. ( 2019 ), and Acemoglu and Restrepo ( 2020 ). However, there are still some challenges in calculating the number of industrial robots per worker using IFR data. First, the different industrial classification methods make it difficult to map the raw data one-to-one to the Chinese Industry Classification System (CICS). Second, data on the distribution of labor in some industries are not available in China. In view of this, the IFR data are mapped to CICS, and the focus is only on the robotic installations with available labor data, which account for 29.2% of the total raw data.

In light of these challenges, our methodology aligns with established practices in the existing literature (Acemoglu and Restrepo, 2020 ), where the robot usage at the city level is calculated to represent the technological impact of robots. The first step is to calculate the exposure to robots at the industry level:

where \({\rm{MR}}_{{it}}\) donates the stock of industrial robots of industry i in year t, and \({L}_{i,t=2005}\) is the number of labors of i industry based on 2005.

Second is to construct the city-level industry robot usage index:

where I represents the collection of various industries. \({L}_{i,j,t=2005}\) is the number of labors of industry i in city j in 2005. \({\rm{Mean}}\,{L}_{i,t=2005}\) is the mean of labors of industry i in 2005. It is notable that \({L}_{i,j,t=2005}/{\rm{Mean}}\,{L}_{i,t=2005}\) is the proportion of number of labors in industry \(i\) and region j to the average number of labors of the industry, representing the relative labor force proportion.

Since the industry classification standard in China’s statistical system is not exactly the same as the IFR, we apply the employment data at the city and industry level from the China Labor Statistics Yearbook and merge it with the major industries provided by the IFR, so as to obtain the relative share of labor in industry i at the city level in year t . Then, we calculate and obtain the target variable \({\rm{Robot}}_{{jt}}\) .

Finally, since the distribution of cities’ exposure to robots is substantially right-skewed, we adopt its logarithmic version, which is essentially the normal distribution in the empirics.

Spatial econometric model

We selected the SDM for its effectiveness in addressing the issue of spatial dependence in the study of the environmental impacts of emerging technologies. The SDM’s spatial weight matrix captures the geographical distribution and interactions of economic activities, which is essential for understanding the diffusion of pollutants and the influence of environmental policies across regions. Furthermore, its flexibility allows it to be adapted to specific research contexts, making it suitable for exploring the micro-mechanisms of spillover effects. In addition, our dataset fits well with the data requirements of the SDM, promising insightful results. The accuracy and theoretical relevance of this model outperform other models considered, making it the optimal choice for our study.

Spatial correlation test

Spatial correlation test reflects whether there is a correlation between any two geographical entities. The global Moran’s I value (GMI) is used to test the existence of spatial correlations, defined as:

where N is the number of observations, i and j are the two dimensions of coordinates. x i i are the observations of cell i with the mean \(\bar{x}\) . And there is a theoretical possibility that the global spatial correlation test may ignore the features of atypical local area, Anselin ( 1995 ) therefore proposed a local version Moran’s I value (LMI) formulated as follows:

where w ij is the spatial weight of two adjacent regions based on the specific spatial relationships and \(W\) is the double summation of w ij on i and j that \(W=\mathop{\sum }\nolimits_{i=1}^{n}\mathop{\sum }\nolimits_{j=1}^{n}{w}_{{ij}}\) . Take the geographical adjacency weight matrix \({w}_{{ij}}^{a}\) for instance, it follows the rule that it is equal to 1 if regions i and j are spatially adjacent, and 0 otherwise. An alternative method used in this paper is to assign weights using the decay functions, such as the geographical distance weight matrix \({w}_{{ij}}^{d}\) :

where \({\phi }_{i}\) and \({\phi }_{j}\) donate the spherical coordinates of the centroids of regions i and j respectively, \(\Delta \tau\) is the difference of coordinates between regions i and j. \(R\) is the earth radius, approximately 3958.761 miles. The geographical distance weight matrix is standardized by rows and its diagonal elements are set to 0.

Spatial autoregressive model (SAM) and spatial error model (SEM) are two typical categories for spatial econometric setting (LeSage and Pace, 2009 ). And LeSage et al. ( 2015 ) construct a spatial panel data model called SDM, in consideration of the spatial correlation of the dependent variables as well as the independent variable by adding the spatial lag terms of endogenous and exogenous factors. Since the spatial effect of industrial robot exposure and environmental pollution in a certain region may both be correlated with environmental pollution in nearby regions, we apply the SDM, and the model of this paper is expressed as follows:

where \({\mathrm{ln}{PM}}_{{it}}\) denotes the particulate matter concentrations in the logs of each city I in year t ; \({{IR}\exp }_{{jt}}\) denotes the industrial robots’ exposure in the logs of each city j in year t ; X consists of a vector of control variables, and as noted above. \({w}_{{ij}}\) represents the elements of the n×n order spatial weight matrix and the geographical adjacency weight matrices and geographical distance weight matrix is used in this paper. λ includes the prefecture-level city and year fixed effects. Besides, the \(\rho\) and \(\theta\) in Eq. ( 17 ) represent the spatial regression coefficient of the explanatory variable and the explained variable, respectively. Specifically, \(\rho\) reflects the spatial dependence of environmental pollution in neighboring regions, and \(\theta\) reflects the degree of influence of industrial robot exposure in neighboring regions on environmental pollution in the local area.

After analyzing the impact of industrial robots on environmental pollution using the spatial econometric model in Section “Empirics results”, the IR-PNT effects are further discussed in Section “Further discussion on IR-PNT”. By using SDM, it is also shown that by changing the factors of comparative resource abundance of a region, the installation of robots will inevitably affect the relocation decision of enterprises and factories, and then the air quality of neighboring areas.

Empirics results

Spatial correlation tests.

The results of the GMI (see Table 2 ) show that all Moran’s I indices of PM 2.5 from 2006 to 2019 are significantly greater than zero, indicating that the air pollution levels of cities in China are basically spatially autocorrelated. Moreover, the GMI of PM 2.5 is relatively stable during the study period, indicating the consistency of the level of spatial agglomeration. Therefore, the spatial econometric model is necessary to explore the relationship between air pollutants and other factors.

The above results are consistent with the previous literature, generally Chen et al. ( 2017 ) and Feng et al. ( 2019 ). Urban air quality in China is spatially autocorrelated with a stable trend. It is widely recognized that due to geographical relationships, the mutual transmission of pollutants exacerbates regional pollution. In fact, the formation of PM 2.5 is not only related to physical and chemical factors (such as atmospheric conditions, temperature and humidity, etc.), but also related to anthropogenic economic activities, such as the distribution of heavily polluting and energy-consuming enterprises. In this case, since neighboring cities tend to have similar socio-economic characteristics, and the PM 2.5 of these cities are also quite similar.

To investigate the local characteristics of PM 2.5, the local Moran’s I scatterplots of PM 2.5 at the beginning and end of the study period are also presented (see Fig. 2 ). The scatterplots can be divided into four quadrants, which indicate the local spatial correlations between regions (Feng et al. 2019 ). The first and third quadrants represent positive spatial correlations, while the second and fourth quadrants represent negative spatial correlations. From Fig. 2 , it is clear that PM 2.5 is positively correlated in most cities, with a shorter spatial distance between different regions indicating a similar concentration of PM 2.5. Overall, highly polluted cities in China tend to cluster together, as do clean cities throughout the study period.

A Local Moran’s I in 2006 B Local Moran’s I in 2019. Note: Due to the length of the paper, only part of the results are shown, but we accept any request for results.

Baseline regressions

In order to clarify the relationship between industrial robots and local air pollution, we first estimate how the presence of industrial robots affects the average level of air pollution, and present the results in Table 3 . Columns (1) and (3) show the case without control variables, while columns (2) and (4) include them sequentially. To test the robustness of the results, the first two columns use the (0–1) adjacency matrix, while the last two columns use the geographical distance matrix.

Unlike traditional non-spatial models, the spatial econometric model incorporates circular feedbacks between independent and dependent variables in local and nearby areas. Therefore, the estimated coefficients in the first two rows of Table 3 cannot reflect the marginal effects of the independent variables on the dependent variable. As outlined by Elhorst ( 2014 ), the marginal effects should be explained by the direct and indirect effects in the SDM, respectively, and we estimate these effects accordingly.

For the direct effect, the coefficients of IRexp indicate a negative and significant correlation between industrial robot exposure and PM2.5 after controlling for socio-economic variables and city-level fixed effects, which verifies the proposition of the first environmental externality (local cleaning effect), i.e., Proposition 1 . Taking column (4) as an example, cities with the highest exposure to industrial robots experience a 61% (0.0184 × (2.22 - 0.065) / 0.065) reduction in PM2.5 concentrations compared to that of cities with average exposure to industrial robots, simply due to the increasing exposure to industrial robots in the local area (with the highest and average values of exposure to industrial robots being 2.22 and 0.065, respectively). Moreover, the resulting value suggests that during the period 2006 - 2019, the introduction of industrial robots will reduce the average PM2.5 concentration by approximately 6.89 μg/m 3 (50.6*0.0184 × ((0.126 - 0.015) / 0.015), where 50.6 μg/m 3 is the actual PM2.5 concentration in 2006. Based on the empirical dose-response relationship estimated by Burnett et al. ( 1999 ) and Pope et al. ( 2008 ), this reduction of 6.89 μg/m 3 from 2006 to 2019 averted the risk of hospitalization for cardiopulmonary disease and heart failure by 2.27% (6.89 × (0.33%) and 9.03% (6.89 × (1.31%), respectively, nationally.

However, considering the possibility of IR-PNT, the overall cleaning effect of industrial robots may be limited. For the indirect effect, the coefficients of W_IRexp indicate that the industrial robot exerts a positive spillover effect on PM2.5 in neighboring regions, which means that the higher the exposure to industrial robots in the local area, the higher the PM2.5 in the neighboring areas will be. It can be understood as the fact that the higher exposure to industrial robots will “pollute” its neighborhood. This result confirms the proposition of the second environmental externality (IR-PNT) proposed hereby, i.e., Propositions 2a and 2b . Specifically, column 4 shows that the 1% increase in exposure to industrial robots in neighboring regions increases the PM 2.5 in the local area by 0.11% on average, although this effect is insignificant. It will be further specified in the next section.

Additionally, the coefficients of S_rho show that air pollution has a significant positive spillover effect: The 1% increase in PM 2.5 in neighboring regions leads to at least 0.23% higher PM 2.5 in the local area, which is consistent with the spatial correlation test mentioned above in Table 2 .

Overall, industrial robots have a spillover effect on air quality, leading to increased pollution. This finding is consistent with Nguyen et al. ( 2020 ) and Liu et al. ( 2022 ), which indicate that the development of new and emerging technologies is not always good for the environment. Our finding is also similar to Chen et al. ( 2022 ), which found that emission reduction efforts in clean energy development are offset by CO 2 transfer effect.

Heterogeneity analysis

In this section, the heterogeneous effects of industrial robots on air pollution are analyzed. Column (2) in Table 4 performs the spatial regression according to the location of the cities, while column (3) in Table 4 performs the spatial regression according to the size of the cities. For the direct effect, the coefficients of East_ IRexp and Metro_ IRexp are negative, unlike W_ IRexp in the baseline, indicating that as the number of industrial robots increases in the east and in the metropolis, the local cleaning effect is even greater. For the indirect effect, combining the coefficients of IRexp , East_ IRexp and Metro_ IRexp , there will be a “cleaning” effect in the metropolis and a small one in the eastern areas, although not significant. This result differs from the baseline. This may be because there is a stronger climate in the east that encourages firms to invest in green and innovative technologies rather than relocating to pollute neighboring areas (Zhang et al. 2024 ).

Our findings are not consistent with Ding et al. ( 2023 ), whose heterogeneity analysis suggests that AI reduces carbon emissions through spatial spillovers, with the effect being stronger in the Midwest. In contrast, our analysis indicates a stronger effect in the Eastern regions. This discrepancy may be attributed to differences in the data scales used: Ding et al. ( 2023 ) employed provincial panel data, our study is based on city-level data. At the same time, the firms that emit large amounts of carbon and cause air pollution do not exactly overlap. This variation in the distribution of high-emission firms may significantly influence the regional impact of AI on environmental outcomes.

From Table 5 , we can see that the regression results of PM2.5 at different quartiles show significant variations, and the local cleaning effect of industrial robots is stronger in areas with higher PM2.5. From the perspective of effect decomposition, the indirect effect dominates the total effect of the pollution reduction effect of industrial robots in the cities with PM2.5 at different quantiles. Surprisingly, the increase in robots in clean areas increases the pollution of neighboring regions considerably, and this effect gradually declines as PM2.5 is mitigated into a cleaning effect in neighboring regions. This suggests that the IR-PNT effect that we find in the baseline regression is mainly caused by the introduction of industrial robots in the cleaning area. This may be because polluters in cleaner areas already face high marginal abatement costs, exacerbating their relative local disadvantage. As a result, they are more likely to relocate to nearby areas to avoid competition.

Robustness checks

To assess the robustness of the effect of industrial robots on air pollution, the sample window from 2009 to 2017 in column (1) is narrowed to eliminate the boundary effect of the data, as shown in Table 6 . In addition, a region-based placebo test is adopted in column (2) by randomly assigning the exposure to industrial robots of each region in a normal distribution manner. Column (3) includes the first and second lags of the dependent variables to overcome the serial correlation. Lastly, in column (4), a difference-in-difference model is set up to test the effect of industrial robots on air pollution, taking the document of Guiding Opinions of the Ministry of Industry and Information Technology on Promoting the Development of Industrial Robot Industry as a shock that occurred in 2013 and taking the proportion of employees in tertiary industries as the policy treatment effect, a difference-in-difference model is established to check the effects of industrial robots on air pollution.

In summary, the result drawn here has passed all the robustness checks, and the pseudo-shock cannot reject the null hypothesis even at the 10% significance level. It is clear that the effect of industrial robots on PM2.5 is still significant and negative, which is consistent with the previous result and the main conclusion here.

Alternative interpretations

This section discusses several latent competing stories that may also create a spurious link between industrial robots and pollution. Possible confounding factors fall into three categories: differentiated environmental regulation (including formal or informal), contemporaneous historical events (e.g., the Air Pollution Prevention and Control Action Plan, carbon emission trading system, low-carbon city pilots), and other emerging technologies (the development of information and communication technology).

Environmental regulations

Increasing environmental awareness and the quest for better living conditions is an inevitable trend that accompanies socio-economic development. To meet the demand for a better environment, the Chinese government has implemented a series of policies to control pollution and improve environmental quality. This process is synchronized with socio-economic development, and may be a latent factor in comprehensively reducing environmental pollution. If the increase in environmental awareness and regulation overlaps with the penetration of industrial robots, the resulting air pollution control effect of robots is likely to be overestimated.

To assess the potential problem, the effect of the level of local environmental regulation (broadly defined) on PM2.5 is examined. Environmental regulation includes not only the mandatory environmental policies set by the government (formal regulation), but also the participation of environmentalists and the general public, who assume environmental rights and responsibilities, and will negotiate and consult with polluters (informal regulation). Theoretically, the level of environmental regulation can exert pressure on the behavior of heavy polluters and residents. The frequency of environmental-related terms, such as “green,” “low-carbon,” “ecology,” “pollution discharge,” “emission reduction,” “environmental protection,”, etc., is collected from the Government Work Reports of each city as a proxy for formal environmental regulation. In addition, following the work of Pargal and Wheeler ( 1996 ), indicators such as income level, education level, population density, and age structure are selected to comprehensively measure the intensity of informal regulation in each city. The results of columns (1) and (2) in Table 6 show that the estimated cleaning effect of industrial robots on PM 2.5 remains highly significant, suggesting that it is unlikely that the present results can be explained by the effect of environmental regulations.

Contemporaneous historical events

Some might argue that it is not industrial robots that have improved local air quality, but rather the impact of concurrent events, such as the introduction of more direct air pollution policies during the study period. In China, the Air Pollution Prevention and Control Action Plan (Action Plan) from 2013 to 2017 sets clear and quantifiable targets for the reduction of atmospheric particulate matter, and is considered a milestone for the improvement of China’s ecological environment (Geng et al. 2021 ), while the introduction of the carbon emission trading scheme (ETS) market is considered an effective way to reduce carbon emissions (Cui et al. 2021 ). In turn, carbon dioxide has a similar source with PM2.5, and promoting carbon markets may have synergistic effects in reducing PM2.5. Similarly, the two waves of low-carbon city pilot projects launched in 2010 and 2012 are also considered to be the driving force for improving air quality (Yu and Zhang, 2021 ).

In order to assess the potential influence of the above contemporaneous events, the data in the year after 2013 are excluded in the baseline specification when the Action Plan was implemented. A number of carbon ETS market and low-carbon city pilots are also excluded by assigning a value of 0 to cities with a carbon ETS market or low-carbon cities and a value of 1 to cities without a carbon ETS market or non-low-carbon cities, in order to control for the effect. The results in columns (3), (4), and (5) of Table 6 indicate that the estimated cleaning effect of industrial robots remains significant. Although the magnitudes become somewhat smaller when excluding the carbon ETS market and low-carbon cities, this does not indicate that the impact of industrial robots can be ignored.

Other emerging technologies

A potential concern is that the increase in industrial robots may be accompanied by other emerging technologies that have actually been the main contributors to pollution reduction. For example, the development of information and communication technology (ICT) is considered to be a feasible possibility to reduce PM2.5 concentrations, as it increases production efficiency and technological innovation, and promotes the evolution of the regional industrial structure in a progressive way (Gouvea et al. 2018 ). Moreover, ICT is almost synchronized with the development of industrial robots. To exclude the potential inference of ICT in the present baseline model, the level of ICT development in cities is calculated by extracting the first principal component of total telecommunication services, the number of local telephone users, and the number of internet users. Cities with an above-average level of digital economy are then assigned a value of 1 and otherwise 0 to eliminate the confounding effect of ICT development. The results in column (6) of Table 7 show that the estimated cleaning effect of industrial robots on PM2.5 remains highly significant, although its magnitudes become smaller. Overall, there is convincing evidence that the main results are unlikely to be overturned by the alternative interpretations mentioned above.

Further discussion on IR-PNT

In order to unravel the more specific characteristics of IR-PNT, this section discusses its spatial and temporal variations, as well as the main driving factors behind, including the changing industrial layout (from a macro perspective) and the relocation of firms (from a micro perspective).

Spatial variation of IR-PNT

Based on the spatial weight matrix of reciprocal geographical distance, we set different thresholds d * to study the pollution nearby transfer effects. Specifically, starting from 25 km, we withhold the spatial weight of neighboring cities with distance d ij smaller than the threshold d * , and set the weight of more distant cities to 0. This geographical distance threshold model is expressed as follows:

where \({{PR}}_{{it}}\) denotes the PM2.5 concentrations in logs of each city i in year t , Wij is the spatial weight as defined above, \({{AI}\exp }_{{jt}}\) denotes the industrial robots’s exposure in logs of each city j in year t , \({\boldsymbol{X}}\) consists of a vector of control variables, \({\boldsymbol{\lambda }}\) i ncludes the city and year fixed effects at the prefecture level. Other variables are defined as in Eq. ( 17 ). In this part, we focus on the coefficient of \({\beta }_{d}^{\dagger }\) in this specification.

Figure 3 shows the estimated coefficient results of the PNT effect of industrial robots. It can be seen that the effect of IR-PNT shows an inverted U-shaped curve, which first increases and then decreases with the increase of geographical distance, and the peak point is around 90 km. This result confirms the Proposition 3a . The reasons for this curve may be that: (1) due to technology spillovers, firms in local and very nearby areas tend to experience similar impacts of industrial robots, as well as similar environmental regulations, so there is less motivation for polluting enterprises to relocate their business within a certain distance (Porter, 2000 ); (2) when the distance exceeds 150 km, the IR-PNT effect gradually decreases, probably due to the fact that as the distance increases, the costs of supporting facilities and relocation increase, so heavy polluters are reluctant to relocate. Examples of this can be seen in reality. The IR-PNT effect is more pronounced in neighboring regions, such as Langfang near Beijing and Kaifeng near Zhengzhou, when industrial robots penetrate.

Note: The black solid line is the estimated coefficient of IR-PNT, and the gray dashed lines are the lower and upper bounds of its confidence interval (CI) at the 95% level of statistical significance.

Temporal Variation of IR-PNT

In this part, the cross-terms between the spatial lag terms of industrial robots and the year dummy variables are introduced into the regression to study the temporal variation of the IR-PNT effect. The model is expressed as follows:

where the variables are all defined as in Eq. ( 17 ). We focus on the coefficient of β k in this specification.

The results are shown in Fig. 4 . According to the estimated results of the full sample, the overall magnitudes of IR-PNT vary in the same pattern, and there is an inverted U-shaped curve of IR-PNT with the increase of the geographical distance as shown in Fig. 3 . However, over time, the effect increases after an initial decrease, with the extreme value in 2014. Specifically, the IR-PNT effect is similar to that of the overall estimate, showing a downward trend from 2006 to 2013, which becomes an upward trend from 2014 to 2019. The possible explanations are that the Chinese government started to attach importance to environmental protection and introduced a package of laws and regulations during this period, including the inclusion of air quality in the evaluation of local officials’ promotion, which led to an increase in PNT penalties (Zhang et al. 2013 ). However, the IR-PNT effect has rebounded since 2015, showing that more attention needs to be paid to these years and justifying the practical significance of the present research.

Note: The black solid line is the estimated coefficient of IR-PNT, and the gray dashed line is the fit line smoothed with a sixth-order polynomial function.

Scale-structure trade-offs and mechanism embodied in IR-PNT

The PNT can be attributed to the economic scale and industrial structure change (Antweiler et al. 2001 ). Therefore, we try to explore the scale-structure trade-offs embodied in the IR-PNT mechanism: As industrial robots become more widespread, does the city, to which companies are relocated, changes in the absolute number of pollution-intensive industries (scale effect) or rather in the relative amount (structural effect)? A priori, if the relative share of pollution-intensive industries in neighboring cities is not significantly affected, but the absolute number is, the scale effect dominates; otherwise, the structural effect dominates.

Therefore, two indicators, i.e., P_ratio and P_scale , are constructed with the proportion and amount of the output value of pollution-intensive industries, respectively. The selection of pollution-intensive industries includes 11 industries from the First National Pollution Source Census released by the China’s State Council in 2006. The data are from China Bureau of Statistics.

Table 8 shows the results of the scale structure trade-offs embodied in the IR-PNT mechanism. Columns (1) and (2) show that in the local area, the installation of industrial robots can promote the transformation of the local industrial structure and make it cleaner, while it can significantly worsen the pollution intensities of the local industrial structure in neighboring regions. There is significant impact of the penetration of industrial robots on the proportion of the output value of polluting industries, but no significant impact on the absolute amount of the output value of polluting industries, suggesting that the effects of IR-PNT are led by the structural effect.

Further examination is conducted over whether the structure and scale of the pollution-intensive industries have a significant impact on the PM2.5 concentrations. As shown in Columns (3) and (5), the increase in the proportion of the output value of pollution-intensive industries will significantly increase the local PM2.5 concentrations at a 1% statistical level, and the rise in the amount of the output value of pollution-intensive industries in adjacent areas can also benefit the local ambient air quality at a 10% statistical level. In any case, considering the changes of the absolute and relative quantity of the output value of pollution-intensive industries, it is found that the increase in the PM2.5 concentration in the adjacent areas is dominated by the fact that the industrial structure changes toward a more pollution-intensive state.

At the firm level, as the penetration of industrial robots increases, the pollution-intensive firms will choose either to innovate locally or to relocate to the neighborhood. There are two factors that can affect their relocation decision, thus forming the mechanism of IR-PNT, namely external and internal costs. For the external cost of relocation, two principal components are used as proxies. The first principal component is extracted from the volume of freight among road, air, water and rail to proxy the degree of transport connectivity. In addition, the second principal component is extracted from the gross telecommunications business, telephone and internet users to proxy the level of information infrastructure. We assume that the more transport links and information infrastructures there are, the lower the barriers to relocation. For the internal costs of relocation, we focus on the inertia of production factor inputs. The share of current assets and employee wages are constructed to measure the internal costs of relocation.

Table 9 shows the results for external costs in the IR-PNT mechanism. According to columns (1) and (3), the estimated coefficients of W_IRexp×W_Trans and W_IRexp×W_ICT are negative, although the coefficients of W_IRexp×W_Trans are not significant, indicating that on average the higher the external costs of relocation, the more reluctant the pollution-intensive firms would be to relocate, thus there will be no strong effect of IN-PNT.

Table 10 shows the results for internal costs in the IR-PNT mechanism. The estimated coefficients of W_IRexp×W_Quick are positive at the 5% statistical level, while those of W_IRexp×W_Wages are negative at the 1% statistical level. The results in columns (1) and (2) indicate that the higher the current asset ratio of pollution-intensive industries, the higher the probability of relocation. However, the conclusion from column (3) that rising wages can prevent the relocation of pollution-intensive industries is relatively unexpected. One explanation is that higher wages imply a higher level of human capital, which makes firms reluctant to relocate to neighboring regions to avoid the sunk costs of employee training (Weiss, 1995 ; Philippon and Reshef, 2012 ). Overall, compared to the coefficients of external costs, the internal nature is more dominant in the willingness to relocate. This result confirms the Proposition 3b .

These findings complement the literature on the relationship between firm location behavior and pollution (see, e.g., Dou and Han, 2019 ). We verify that, in addition to environmental regulations, firms make location decisions that take into account their relative competitiveness in emerging technologies and adapt by relocating their operations or flexing their industrial layout, which affects local and neighboring air quality.

Conclusions and policy implications

Environmental problems are frequently the result of externalities. With the rapid development of industrial robots in China, this emerging technology is anticipated to profoundly transform economic structure and productivity. However, whether it can harmonize the goals of economic growth and environmental improvement remains a question. Grounded in the new economic geography theory, this study focuses on the two-sided environmental externalities caused by industrial robots and investigates how the greater penetration of robots in the economy affects local and neighboring air quality using the spatial econometric model.

Our theoretical model suggests that the level of penetration of industrial robots in different regions will shape the locational choices of diverse firm types by changing their relative competitive advantages. The empirical results proved that the presence of robots significantly mitigates air pollution in the local area, yet paradoxically intensify it in neighboring areas. This phenomenon, termed IR-PNT, underscores the presence of pollution transfer at a regional level. To investigate the cause of the IR-PNT, a mechanism analysis is further conducted. Consistent with our theoretical model, it is found that the structural effect mainly contributes to the increase of PM2.5, which shows that as the penetration of industrial robots increases, pollution-intensive business is more likely than others to relocate to neighboring cities, often replacing cleaning firms in their neighboring cities. It is shown that developed cities with advanced technologies tend to relocate highly polluting business to neighboring cities, imposing an unfair burden on their “neighbors”.

Furthermore, we identified both heterogeneous and rebound effects of robots on air quality. Firstly, an increase in industrial robots in the east and in the metropolis has larger local cleaning and IR-PNT effect. Conversely, cities with initially cleaner air tend to experience a rebound effect post robot integration, linked to their distinct industrial structures and advanced robotic technologies. Notably, operational robots can produce waste, like discarded batteries, and their deployment necessitates considerable material use. If these materials are sourced or processed in a way that harms the environment, it can counteract the initial environmental gains. In China, the penetration rate of robots in cleaner areas is relatively lower and more likely to be in the stage of triggering rebound effects.

Due to limitations in data availability, the analyses in this study do not cover all industries in China, nor do they address the spatial spillovers of robotics across sectors. In addition, although not a central aspect of this study, the identification of causality remains potentially underdeveloped. This study has improved causal validity to some extent through robustness tests and the exclusion of competing hypotheses. Future research should aim to more definitively establish and confirm the relationship between industrial robotics and environmental quality using more robust causal inference methods. In addition, it should be emphasized that our conclusions regarding the local cleaning effect and the IR-PNT effect are derived from a specific Chinese sample. In order to formulate more generalizable findings, further verification through cross-national studies is essential.

In general, considering the IR-PNT effect, policy makers may need to consider how to minimize, reduce and offset the externalities caused by the unequal distribution of industrial robots. First, researchers are expected to assess the pollution nearby transfer effect of emerging technologies in different policy scenarios (Moni et al. 2020 ). For example, the tradable permit policy, such as cap-and-trade, can improve the optimal allocation of resources in technological change through the improvement of pollutant property rights, making it easier to achieve the balanced goals of pollution control and economic development (Liu et al. 2020 ; Wu and Wang, 2022 ).

Second, we find that the IR-PNT effect is mainly caused by the relocation of polluting firms from clean areas to neighboring regions. This suggests that regional cooperation in pollution reduction is crucial. Particular attention must be paid to the air quality around the clean area., to avoid cleaning at the expense of polluting someone else’s backyard. To achieve this, a synergistic approach is essential, integrating technological, financial and administrative strategies in regional cooperation.

Lastly, the study shows that IR-PNT effects are lower in metropolitan areas and urban agglomerations, and there is even a cleaning effect. In this case, the expansion of urban agglomerations tends to reduce the environmental inequality caused by technological progress. On the one hand, the externalities of pollution are resolved by internalizing them in larger urban clusters; on the other hand, the expanded urban agglomerations increase the area of cities and the distance and costs of relocating polluting firms (Tabuchi, 1998 ), thus reducing pollution transfer and reducing environmental inequality.

Data availability

The datasets generated during and/or analyzed during the current study are available in the Harvard Dataverse repository, https://doi.org/10.7910/DVN/9BYJYD .

Acemoglu D, Restrepo P (2017) Secular stagnation? The effect of aging on economic growth in the age of automation. Am Econ Rev 107:174–179. https://doi.org/10.1257/aer.p20171101

Article   Google Scholar  

Acemoglu D, Restrepo P (2019) Automation and new tasks: how technology displaces and reinstates labor. J Econ Perspect 33:3–30. https://doi.org/10.1257/jep.33.2.3

Acemoglu D, Restrepo P (2020) Robots and jobs: evidence from US labor markets. J Polit Econ 128:2188–2244. https://doi.org/10.1086/705716

Aduba JJ, Asgari B (2020) Productivity and technological progress of the Japanese manufacturing industries, 2000–2014: estimation with data envelopment analysis and log-linear learning model. Asia-Pacific. J Reg Sci 4(2):343–387

Google Scholar  

Aghion P, Jones BF, Jones CI (2017) Artificial Intelligence and Economic Growth. Working Paper Series. https://doi.org/10.3386/w23928

Aghion P, Antonin C, Bunel S (2019) Artificial intelligence, growth and employment: the role of policy. Econ Stat 510(1):149–164

Anderson D (2001) Technical progress and pollution abatement: an economic view of selected technologies and practices. Environ Dev Econ 6:283–311. https://doi.org/10.1017/S1355770X01000171

Anelli M, Colantone I, Stanig P (2019) We were the robots: automation and voting behavior in western europe. BAFFI CAREFIN Centre Research Paper, (2019-115)

Anselin L (1995) Local indicators of spatial association—LISA. Geogr Anal 27(2):93–115

Anselin L (2003) Spatial externalities, spatial multipliers, and spatial econometrics. Int Reg Sci Rev 26(2):153–166

Antweiler W, Copeland BR, Taylor MS (2001) Is free trade good for the environment? Am Econ Rev 91:877–908. https://doi.org/10.1257/aer.91.4.877

Arbia, G, Espa, G, Giuliani, D (2021) Spatial microeconometrics. Routledge

Atanu S, Love HA, Schwart R (1994) Adoption of emerging technologies under output uncertainty. Am J Agric Econ 76(4):836–846

Audirac I (2005) Information technology and urban form: challenges to smart growth. Int Reg Sci Rev 28(2):119–145

Autant-Bernard C, LeSage JP (2011) Quantifying knowledge spillovers using spatial econometric models. J Reg Sci 51(3):471–496

Autor, D, Chin, C, Salomons, AM, Seegmiller, B, 2022. New Frontiers: The Origins and Content of New Work, 1940–2018. Working Paper Series. https://doi.org/10.3386/w30389

Bartelsman EJ, Doms M (2000) Understanding productivity: lessons from longitudinal microdata. J Econ Lit 38(3):569–594

Bell ML, Ebisu K (2012) Environmental inequality in exposures to airborne particulate matter components in the United States. Environ Health Perspect 120(12):1699–1704

Article   CAS   PubMed   PubMed Central   Google Scholar  

Benhabib J, Perla J, Tonetti C (2021) Reconciling models of diffusion and innovation: a theory of the productivity distribution and technology frontier. Econometrica 89(5):2261–2301

Article   MathSciNet   Google Scholar  

Ben Kheder S, Zugravu N (2012) Environmental regulation and French firms location abroad: an economic geography model in an international comparative study. Ecol Econ 77:48–61. https://doi.org/10.1016/j.ecolecon.2011.10.005

Bommer R (1999) Environmental policy and industrial competitiveness: the pollution‐haven hypothesis reconsidered. Rev Int Econ 7(2):342–355

Bocca, R, Ashraf, M, Jamison, S (2021) Fostering Effective Energy Transition 2021 Edition. In World Economic Forum

Buera, FJ, Fattal Jaef, RN (2018) The dynamics of development: innovation and reallocation. World Bank Policy Research Working Paper (8505)

Burnett RT, Smith-doiron M, Stieb D, Cakmak S, Brook JR (1999) Effects of particulate and gaseous air pollution on cardiorespiratory hospitalizations. Arch Environ Health 54:130–139. https://doi.org/10.1080/00039899909602248

Article   CAS   PubMed   Google Scholar  

Chen SM, He LY (2014) Welfare loss of China’s air pollution: How to make personal vehicle transportation policy. China Econ Rev 31:106–118

Chen X, Shao S, Tian Z, Xie Z, Yin P (2017) Impacts of air pollution and its spatial spillover effect on public health based on China’s big data sample. J Clean Prod 142:915–925. https://doi.org/10.1016/j.jclepro.2016.02.119 . Special Volume on Improving natural resource management and human health to ensure sustainable societal development based upon insights gained from working within ‘Big Data Environments’

Article   CAS   Google Scholar  

Chen Y, Shao S, Fan M, Tian Z, Yang L (2022) One man’s loss is another’s gain: does clean energy development reduce CO2 emissions in China? Evidence based on the spatial Durbin model. Energy Econ 107:105852

Cheng H, Jia R, Li D, Li H (2019) The rise of robots in China. J Econ Perspect 33(2):71–88

Chiu S-H, Lin T-Y, Wang W-C (2024) Investigating the spatial effect of operational performance in China’s regional tourism system. Human Soc Sci Commun 11(1):14. https://doi.org/10.1057/s41599-024-02741-y

Chun H, Kim JW, Lee J (2015) How does information technology improve aggregate productivity? A new channel of productivity dispersion and reallocation. Res Policy 44(5):999–1016

Cui J, Wang C, Zhang J, Zheng Y (2021) The effectiveness of China’s regional carbon market pilots in reducing firm emissions. Proc Natl Acad Sci USA 118(52):e2109912118

Dauth W, Findeisen S, Suedekum J, Woessner N (2021) The adjustment of labor markets to robots. J Eur Econ Assoc 19:3104–3153. https://doi.org/10.1093/jeea/jvab012

Dusík, J, Fischer, T, Sadler, B, Therivel, R, Saric, I, (2018) Strategic Environmental and Social Assessment of Automation: Scoping Working Paper

Ding T, Li J, Shi X, Li X, Chen Y (2023) Is artificial intelligence associated with carbon emissions reduction? Case of China. Resour Policy 85:103892. https://doi.org/10.1016/j.resourpol.2023.103892

Dou J, Han X (2019) How does the industry mobility affect pollution industry transfer in China: empirical test on Pollution Haven Hypothesis and Porter Hypothesis. J Clean Prod 217:105–115. https://doi.org/10.1016/j.jclepro.2019.01.147

Elhorst JP (2014) Spatial Econometrics: From Cross-Sectional Data to Spatial Panels. Springer, Berlin, Heidelberg, 10.1007/978-3-642-40340-8

Book   Google Scholar  

Ehrlich PR, Holdren JP (1971) Impact of population growth. Science 171:1212–1217. cq9zm8

Article   ADS   CAS   PubMed   Google Scholar  

Feng Y, Cheng J, Shen J, Sun H (2019) Spatial effects of air pollution on public health in China. Environ Resour Econ 73:229–250. https://doi.org/10.1007/s10640-018-0258-4

Florida R, Mellander C, Stolarick K (2008) Inside the black box of regional development—human capital, the creative class and tolerance. J Econ Geogr 8(5):615–649

Fujita M, Krugman P (2004) The new economic geography: past, present and the future. In Fifty years of regional science (pp. 139-164). Springer, Berlin, Heidelberg

Gao K, Yuan Y (2021) The effect of innovation-driven development on pollution reduction: empirical evidence from a quasi-natural experiment in China. Technol Forecast Soc Change 172:121047

Geng G, Zheng Y, Zhang Q, Xue T, Zhao H, Tong D, Davis SJ (2021) Drivers of PM2. 5 air pollution deaths in China 2002–2017. Nat Geosci 14(9):645–650

Article   ADS   CAS   Google Scholar  

Gihleb R, Giuntella O, Stella L, Wang T (2022) Industrial robots, Workers’ safety, and health. Labour Econ 78:102205. https://doi.org/10.1016/j.labeco.2022.102205

Giuntella O, Wang T (2019) Is an Army of Robots Marching on Chinese Jobs? https://doi.org/10.2139/ssrn.3390271

Gouvea R, Kapelianis D, Kassicieh S (2018) Assessing the nexus of sustainability and information & communications technology. Technol Forecast Soc Change 130:39–44

Graetz G, Michaels G (2018) Robots at work. Rev Econ Stat 100:753–768. https://doi.org/10.1162/rest_a_00754

Grossman GM, Krueger AB (1991) Environmental Impacts of a North American Free Trade Agreement. Working Paper Series. https://doi.org/10.3386/w3914

Hägele M, Nilsson K, Pires JN, Bischoff R (2016) Industrial robotics. In Springer handbook of robotics (pp. 1385-1422). Springer, Cham

IFR (2020) World Robotics 2020. International Federation of Robotics and the United Nations

Jiang L, Yang Y, Wu Q, Yang L, Yang Z (2024) Hotter days, dirtier air: the impact of extreme heat on energy and pollution intensity in China. Energy Econ 130:107291

Juhász R, Squicciarini MP, Voigtländer N (2020) Technology adoption and productivity growth: Evidence from industrialization in France (No. w27503). National Bureau of Economic Research

Jung JH, Lim DG (2020) Industrial robots, employment growth, and labor cost: a simultaneous equation analysis. Technol Forecast Soc Change 159:120202

Kozul-Wright R (2016, October). Robots and industrialization in developing countries. In United Nations Conference on Trade and Development (No. 60)

Kromann L, Malchow-Møller N, Skaksen JR, Sørensen A (2020) Automation and productivity—a cross-country, cross-industry comparison. Ind Corp Change 29:265–287. https://doi.org/10.1093/icc/dtz039

LeSage J, Pace RK (2009) Introduction to Spatial Econometrics. Chapman and Hall/CRC, New York, p 340. 10.1201/9781420064254

LeSage J (2015) Spatial econometrics. In Handbook of research methods and applications in economic geography (pp. 23-40). Edward Elgar Publishing

Levinson A, Taylor MS (2008) Unmasking the pollution haven effect. Int Econ Rev 49(1):223–254

Li J, Ma S, Qu Y, Wang J (2023) The impact of artificial intelligence on firms’ energy and resource efficiency: empirical evidence from China. Resour Policy 82:103507. https://doi.org/10.1016/j.resourpol.2023.103507

Li Y, Zhang Y, Pan A, Han M, Veglianti E (2022) Carbon emission reduction effects of industrial robot applications: Heterogeneity characteristics and influencing mechanisms. Technol Soc 70:102034

Lin F (2017) Trade openness and air pollution: city-level empirical evidence from China. China Econ Rev 45:78–88

Liu H, Owens KA, Yang K, Zhang C (2020) Pollution abatement costs and technical changes under different environmental regulations. China Econ Rev 62:101497

Liu J, Yu Q, Chen Y, Liu J (2022) The impact of digital technology development on carbon emissions: a spatial effect analysis for China. Resour Conserv Recycl 185:106445. https://doi.org/10.1016/j.resconrec.2022.106445

Liu J, Liu L, Qian Y, Song S (2022) The effect of artificial intelligence on carbon intensity: evidence from China’s industrial sector. Socioecon Plann Sci 83:101002. https://doi.org/10.1016/j.seps.2020.101002

Liu Y, Ren T, Liu L, Ni J, Yin Y (2022) Heterogeneous industrial agglomeration, technological innovation and haze pollution. China Econ Rev https://doi.org/10.1016/j.chieco.2022.101880

Luan F, Yang X, Chen Y, Regis PJ (2022) Industrial robots and air environment: a moderated mediation model of population density and energy consumption. Sustain Prod Consum. 30:870–888

Luo Z, Wan G, Wang C, Zhang X (2018) Urban pollution and road infrastructure: a case study of China. China Econ Rev 49:171–183

Milani S (2017) The impact of environmental policy stringency on industrial R&D conditional on pollution intensity and relocation costs. Environ Resour Econ 68:595–620

Moni SM, Mahmud R, High K, Carbajales‐Dale M (2020) Life cycle assessment of emerging technologies: a review. J Ind Ecol 24(1):52–63

Nguyen TT, Pham TAT, Tram HTX (2020) Role of information and communication technologies and innovation in driving carbon emissions and economic growth in selected G-20 countries. J Environ Manag 261:110162. https://doi.org/10.1016/j.jenvman.2020.110162

Pargal S, Wheeler D (1996) Informal regulation of industrial pollution in developing countries: evidence from Indonesia. J Polit Econ 104:1314–1327. https://doi.org/10.1086/262061

Pellenbarg PH, Van Wissen LJ, Van Dijk J (2002) Firm Relocation: State of the Art and Research Prospects. University of Groningen, Groningen, p 1–42

Pennings E, Sleuwaegen L (2000) International relocation: firm and industry determinants. Economics Letters 67(2):179–186

Philippon T, Reshef A (2012) Wages and human capital in the US finance industry: 1909–2006. Q J Econ 127(4):1551–1609

Pires JN (2007) Introduction to the Industrial Robotics World. https://doi.org/10.1007/978-0-387-23326-0_1

Pope III CA, Renlund DG, Kfoury AG, May HT, Horne BD (2008) Relation of heart failure hospitalization to exposure to fine particulate air pollution. Am J Cardiol 102(9):1230–1234

Article   PubMed   Google Scholar  

Porter ME (2000) Locations, clusters, and company strategy. The Oxford Handbook of Economic Geography 253:274

Shen K, Gang J, Xian F, University N, University F (2017) Does environmental regulation cause pollution to transfer nearby? Econ Res J 52:44–59

Sheng D, Bu W (2022) The usage of robots and enterprieses’ pollution emissions in China. J Quant Technol Econ 2022(9):157–176

Shi B, Feng C, Qiu M, Ekeland A (2018) Innovation suppression and migration effect: The unintentional consequences of environmental regulation. China Econ Rev 49:1–23

Soergel A (2015) Robots could cut labor costs 16 percent by 2025 - US News & World Report. Retrieved August 7, 2022, from https://www.usnews.com/news/articles/2015/02/10/robots-could-cut-international-labor-costs-16-percent-by-2025-consulting-group-says

Tabuchi T (1998) Urban agglomeration and dispersion: a synthesis of Alonso and Krugman. J Urb Econ 44(3):333–351

Uhlmann E, Reinkober S, Hollerbach T (2016) Energy efficient usage of industrial robots for machining processes. Proc CIRP 48:206–211. https://doi.org/10.1016/j.procir.2016.03.241

Van Donkelaar A, Hammer MS, Bindle L, Brauer M, Brook JR, Garay MJ, Martin RV (2021) Monthly global estimates of fine particulate matter and their uncertainty. Environ Sci Technol 55(22):15287–15300

Article   ADS   PubMed   Google Scholar  

Wang E-Z, Lee C-C, Li Y (2022) Assessing the impact of industrial robots on manufacturing energy intensity in 38 countries. Energy Econ 105:105748. https://doi.org/10.1016/j.eneco.2021.105748

Wang Y, Feng J (2022) The Adoption Of Industrial Robots And Pollution Abatement In China

Weiss, Andrew (1995) Human Capital vs. Signalling Explanations of Wages. Journal of Economic Perspectives 9(4):133–154

Wilts H, Garcia BR, Garlito RG, Gómez LS, Prieto EG (2021) Artificial intelligence in the sorting of municipal waste as an enabler of the circular economy. Resources 10:28. https://doi.org/10.3390/resources10040028

World Health Organization. (2021) New WHO Global Air Quality Guidelines aim to save millions of lives from air pollution. Air Pollution is One of the Biggest Environmental Threats to Human Health, Alongside Climate Change

Wu Q, Wang Y (2022) How does carbon emission price stimulate enterprises’ total factor productivity? Insights from China’s emission trading scheme pilots. Energy Econ 109:105990. https://doi.org/10.1016/j.eneco.2022.105990

Wu Q (2023) Sustainable growth through industrial robot diffusion: quasi-experimental evidence from a Bartik shift-share design. Econ Transit Inst Change 31:1107–1133. https://doi.org/10.1111/ecot.12367

Wu Q, Sun Z, Jiang L, Jiang L (2023) “Bottom-up” abatement on climate from the “top-down” design: lessons learned from China’s low-carbon city pilot policy. Empir Econ 66:1223–1257

Wu Y, Al-Duais ZAM, Peng B (2023) Towards a low-carbon society: spatial distribution, characteristics and implications of digital economy and carbon emissions decoupling. Human Soc Sci Commun 10:1–13. https://doi.org/10.1057/s41599-023-02233-5

Yu Y, Zhang N (2021) Low-carbon city pilot and carbon emission efficiency: quasi-experimental evidence from China. Energy Econ 96:105125

Zhang X, Wu L, Zhang R, Deng S, Zhang Y, Wu J, Li Y, Lin L, Li L, Wang Y, Wang L (2013) Evaluating the relationships among economic growth, energy consumption, air emissions and air environmental protection investment in China. Renew Sustain Energy Rev 18:259–270. https://doi.org/10.1016/j.rser.2012.10.029

Zhang Z, Zhang W, Wu Q, Liu J, Jiang L (2024) Climate adaptation through trade: evidence and mechanism from heatwaves on firms’ imports. China Econ Rev 84:102–133

Zheng W, Chen P (2020) The political economy of air pollution: Local development, sustainability, and political incentives in China. Energy Res Soc Sci 69:101707

Zheng S, Yao R, Zou K (2022) Provincial environmental inequality in China: measurement, influence, and policy instrument choice. Ecol Econ 200:107537. https://doi.org/10.1016/j.ecolecon.2022.107537

Zhu H, Sang B, Zhang C, Guo L (2023) Have industrial robots improved pollution reduction? A theoretical approach and empirical analysis. China World Econ 31(4):153–172

Download references

Author information

Authors and affiliations.

College of Environmental Sciences and Engineering, Peking University, Beijing, 100871, China

Yanying Wang

Centre for Environment, Energy and Natural Resource Governance (CEENRG), Department of Land Economy, University of Cambridge, Cambridge, CB2 3QZ, UK

Department of Environmental Health Sciences, Fielding School of Public Health, University of California, Los Angeles, LA, CA, 90095, USA

Qingyang Wu

You can also search for this author in PubMed   Google Scholar

Contributions

The authors confirm their contribution to the paper as follows: study conception and design: Q Wu; data collection: Y Wang and Q Wu; analysis and interpretation of the results: Y Wang and Q Wu; draft the paper preparation and revision: Y Wang and Q Wu. All authors reviewed the results and approved the final version of the paper.

Corresponding author

Correspondence to Qingyang Wu .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

Additional information.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wang, Y., Wu, Q. Robots, firm relocation, and air pollution: unveiling the unintended spatial spillover effects of emerging technology. Humanit Soc Sci Commun 11 , 577 (2024). https://doi.org/10.1057/s41599-024-03100-7

Download citation

Received : 28 November 2023

Accepted : 17 April 2024

Published : 06 May 2024

DOI : https://doi.org/10.1057/s41599-024-03100-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies