types of research materials for literature review

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters & Hedges
• Grey Literature
• Managing & Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Mar 1, 2024 10:55 AM
• URL: https://guides.library.ucla.edu/systematicreviews

Literature Review: Types of literature reviews

• Traditional or narrative literature reviews
• Scoping Reviews
• Systematic literature reviews
• Annotated bibliography
• Keeping up to date with literature
• Finding a thesis
• Evaluating sources and critical appraisal of literature
• Managing and analysing your literature
• Further reading and resources

Types of literature reviews

The type of literature review you write will depend on your discipline and whether you are a researcher writing your PhD, publishing a study in a journal or completing an assessment task in your undergraduate study.

A literature review for a subject in an undergraduate degree will not be as comprehensive as the literature review required for a PhD thesis.

An undergraduate literature review may be in the form of an annotated bibliography or a narrative review of a small selection of literature, for example ten relevant articles. If you are asked to write a literature review, and you are an undergraduate student, be guided by your subject coordinator or lecturer.

The common types of literature reviews will be explained in the pages of this section.

• Narrative or traditional literature reviews
• Critically Appraised Topic (CAT)
• Scoping reviews
• Annotated bibliographies

These are not the only types of reviews of literature that can be conducted. Often the term "review" and "literature" can be confusing and used in the wrong context. Grant and Booth (2009) attempt to clear up this confusion by discussing 14 review types and the associated methodology, and advantages and disadvantages associated with each review.

Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26 , 91–108. doi:10.1111/j.1471-1842.2009.00848.x

What's the difference between reviews?

Researchers, academics, and librarians all use various terms to describe different types of literature reviews, and there is often inconsistency in the ways the types are discussed. Here are a couple of simple explanations.

• The image below describes common review types in terms of speed, detail, risk of bias, and comprehensiveness:

"Schematic of the main differences between the types of literature review" by Brennan, M. L., Arlt, S. P., Belshaw, Z., Buckley, L., Corah, L., Doit, H., Fajt, V. R., Grindlay, D., Moberly, H. K., Morrow, L. D., Stavisky, J., & White, C. (2020). Critically Appraised Topics (CATs) in veterinary medicine: Applying evidence in clinical practice. Frontiers in Veterinary Science, 7 , 314. https://doi.org/10.3389/fvets.2020.00314 is licensed under CC BY 3.0

• The table below lists four of the most common types of review , as adapted from a widely used typology of fourteen types of reviews (Grant & Booth, 2009).  

Grant, M.J. & Booth, A. (2009).  A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26 (2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

See also the Library's  Literature Review guide.

Critical Appraised Topic (CAT)

For information on conducting a Critically Appraised Topic or CAT

Callander, J., Anstey, A. V., Ingram, J. R., Limpens, J., Flohr, C., & Spuls, P. I. (2017).  How to write a Critically Appraised Topic: evidence to underpin routine clinical practice.  British Journal of Dermatology (1951), 177(4), 1007-1013. https://doi.org/10.1111/bjd.15873 

Books on Literature Reviews

• << Previous: Home
• Next: Traditional or narrative literature reviews >>
• Last Updated: Jan 16, 2024 1:39 PM
• URL: https://libguides.csu.edu.au/review

Charles Sturt University is an Australian University, TEQSA Provider Identification: PRV12018. CRICOS Provider: 00005F.

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

Types of Literature Review — A Guide for Researchers

Table of Contents

Researchers often face challenges when choosing the appropriate type of literature review for their study. Regardless of the type of research design and the topic of a research problem , they encounter numerous queries, including:

What is the right type of literature review my study demands?

• How do we gather the data?
• How to conduct one?
• How reliable are the review findings?
• How do we employ them in our research? And the list goes on.

If you’re also dealing with such a hefty questionnaire, this article is of help. Read through this piece of guide to get an exhaustive understanding of the different types of literature reviews and their step-by-step methodologies along with a dash of pros and cons discussed.

Heading from scratch!

What is a Literature Review?

A literature review provides a comprehensive overview of existing knowledge on a particular topic, which is quintessential to any research project. Researchers employ various literature reviews based on their research goals and methodologies. The review process involves assembling, critically evaluating, and synthesizing existing scientific publications relevant to the research question at hand. It serves multiple purposes, including identifying gaps in existing literature, providing theoretical background, and supporting the rationale for a research study.

What is the importance of a Literature review in research?

Literature review in research serves several key purposes, including:

• Background of the study: Provides proper context for the research. It helps researchers understand the historical development, theoretical perspectives, and key debates related to their research topic.
• Identification of research gaps: By reviewing existing literature, researchers can identify gaps or inconsistencies in knowledge, paving the way for new research questions and hypotheses relevant to their study.
• Theoretical framework development: Facilitates the development of theoretical frameworks by cultivating diverse perspectives and empirical findings. It helps researchers refine their conceptualizations and theoretical models.
• Methodological guidance: Offers methodological guidance by highlighting the documented research methods and techniques used in previous studies. It assists researchers in selecting appropriate research designs, data collection methods, and analytical tools.
• Quality assurance and upholding academic integrity: Conducting a thorough literature review demonstrates the rigor and scholarly integrity of the research. It ensures that researchers are aware of relevant studies and can accurately attribute ideas and findings to their original sources.

Types of Literature Review

Literature review plays a crucial role in guiding the research process , from providing the background of the study to research dissemination and contributing to the synthesis of the latest theoretical literature review findings in academia.

However, not all types of literature reviews are the same; they vary in terms of methodology, approach, and purpose. Let's have a look at the various types of literature reviews to gain a deeper understanding of their applications.

1. Narrative Literature Review

A narrative literature review, also known as a traditional literature review, involves analyzing and summarizing existing literature without adhering to a structured methodology. It typically provides a descriptive overview of key concepts, theories, and relevant findings of the research topic.

Unlike other types of literature reviews, narrative reviews reinforce a more traditional approach, emphasizing the interpretation and discussion of the research findings rather than strict adherence to methodological review criteria. It helps researchers explore diverse perspectives and insights based on the research topic and acts as preliminary work for further investigation.

Steps to Conduct a Narrative Literature Review

Source:- https://www.researchgate.net/figure/Steps-of-writing-a-narrative-review_fig1_354466408

Define the research question or topic:

The first step in conducting a narrative literature review is to clearly define the research question or topic of interest. Defining the scope and purpose of the review includes — What specific aspect of the topic do you want to explore? What are the main objectives of the research? Refine your research question based on the specific area you want to explore.

Conduct a thorough literature search

Once the research question is defined, you can conduct a comprehensive literature search. Explore and use relevant databases and search engines like SciSpace Discover to identify credible and pertinent, scholarly articles and publications.

Select relevant studies

Before choosing the right set of studies, it’s vital to determine inclusion (studies that should possess the required factors) and exclusion criteria for the literature and then carefully select papers. For example — Which studies or sources will be included based on relevance, quality, and publication date?

*Important (applies to all the reviews): Inclusion criteria are the factors a study must include (For example: Include only peer-reviewed articles published between 2022-2023, etc.). Exclusion criteria are the factors that wouldn’t be required for your search strategy (Example: exclude irrelevant papers, preprints, written in non-English, etc.)

Critically analyze the literature

Once the relevant studies are shortlisted, evaluate the methodology, findings, and limitations of each source and jot down key themes, patterns, and contradictions. You can use efficient AI tools to conduct a thorough literature review and analyze all the required information.

Synthesize and integrate the findings

Now, you can weave together the reviewed studies, underscoring significant findings such that new frameworks, contrasting viewpoints, and identifying knowledge gaps.

Discussion and conclusion

This is an important step before crafting a narrative review — summarize the main findings of the review and discuss their implications in the relevant field. For example — What are the practical implications for practitioners? What are the directions for future research for them?

Write a cohesive narrative review

Organize the review into coherent sections and structure your review logically, guiding the reader through the research landscape and offering valuable insights. Use clear and concise language to convey key points effectively.

Structure of Narrative Literature Review

A well-structured, narrative analysis or literature review typically includes the following components:

• Introduction: Provides an overview of the topic, objectives of the study, and rationale for the review.
• Background: Highlights relevant background information and establish the context for the review.
• Main Body: Indexes the literature into thematic sections or categories, discussing key findings, methodologies, and theoretical frameworks.
• Discussion: Analyze and synthesize the findings of the reviewed studies, stressing similarities, differences, and any gaps in the literature.
• Conclusion: Summarizes the main findings of the review, identifies implications for future research, and offers concluding remarks.

Pros and Cons of Narrative Literature Review

• Flexibility in methodology and doesn’t necessarily rely on structured methodologies
• Follows traditional approach and provides valuable and contextualized insights
• Suitable for exploring complex or interdisciplinary topics. For example — Climate change and human health, Cybersecurity and privacy in the digital age, and more
• Subjectivity in data selection and interpretation
• Potential for bias in the review process
• Lack of rigor compared to systematic reviews

Example of Well-Executed Narrative Literature Reviews

Paper title:  Examining Moral Injury in Clinical Practice: A Narrative Literature Review

Source: SciSpace

While narrative reviews offer flexibility, academic integrity remains paramount. So, ensure proper citation of all sources and maintain a transparent and factual approach throughout your critical narrative review, itself.

2. Systematic Review

A systematic literature review is one of the comprehensive types of literature review that follows a structured approach to assembling, analyzing, and synthesizing existing research relevant to a particular topic or question. It involves clearly defined criteria for exploring and choosing studies, as well as rigorous methods for evaluating the quality of relevant studies.

It plays a prominent role in evidence-based practice and decision-making across various domains, including healthcare, social sciences, education, health sciences, and more. By systematically investigating available literature, researchers can identify gaps in knowledge, evaluate the strength of evidence, and report future research directions.

Steps to Conduct Systematic Reviews

Source:- https://www.researchgate.net/figure/Steps-of-Systematic-Literature-Review_fig1_321422320

Here are the key steps involved in conducting a systematic literature review

Formulate a clear and focused research question

Clearly define the research question or objective of the review. It helps to centralize the literature search strategy and determine inclusion criteria for relevant studies.

Develop a thorough literature search strategy

Design a comprehensive search strategy to identify relevant studies. It involves scrutinizing scientific databases and all relevant articles in journals. Plus, seek suggestions from domain experts and review reference lists of relevant review articles.

Screening and selecting studies

Employ predefined inclusion and exclusion criteria to systematically screen the identified studies. This screening process also typically involves multiple reviewers independently assessing the eligibility of each study.

Data extraction

Extract key information from selected studies using standardized forms or protocols. It includes study characteristics, methods, results, and conclusions.

Critical appraisal

Evaluate the methodological quality and potential biases of included studies. Various tools (BMC medical research methodology) and criteria can be implemented for critical evaluation depending on the study design and research quetions .

Data synthesis

Analyze and synthesize review findings from individual studies to draw encompassing conclusions or identify overarching patterns and explore heterogeneity among studies.

Interpretation and conclusion

Interpret the findings about the research question, considering the strengths and limitations of the research evidence. Draw conclusions and implications for further research.

The final step — Report writing

Craft a detailed report of the systematic literature review adhering to the established guidelines of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). This ensures transparency and reproducibility of the review process.

By following these steps, a systematic literature review aims to provide a comprehensive and unbiased summary of existing evidence, help make informed decisions, and advance knowledge in the respective domain or field.

Structure of a systematic literature review

A well-structured systematic literature review typically consists of the following sections:

• Introduction: Provides background information on the research topic, outlines the review objectives, and enunciates the scope of the study.
• Methodology: Describes the literature search strategy, selection criteria, data extraction process, and other methods used for data synthesis, extraction, or other data analysis..
• Results: Presents the review findings, including a summary of the incorporated studies and their key findings.
• Discussion: Interprets the findings in light of the review objectives, discusses their implications, and identifies limitations or promising areas for future research.
• Conclusion: Summarizes the main review findings and provides suggestions based on the evidence presented in depth meta analysis.

*Important (applies to all the reviews): Remember, the specific structure of your literature review may vary depending on your topic, research question, and intended audience. However, adhering to a clear and logical hierarchy ensures your review effectively analyses and synthesizes knowledge and contributes valuable insights for readers.

Pros and Cons of Systematic Literature Review

• Adopts rigorous and transparent methodology
• Minimizes bias and enhances the reliability of the study
• Provides evidence-based insights
• Time and resource-intensive
• High dependency on the quality of available literature (literature research strategy should be accurate)
• Potential for publication bias

Example of Well-Executed Systematic Literature Review

Paper title: Systematic Reviews: Understanding the Best Evidence For Clinical Decision-making in Health Care: Pros and Cons.

Read this detailed article on how to use AI tools to conduct a systematic review for your research!

3. Scoping Literature Review

A scoping literature review is a methodological review type of literature review that adopts an iterative approach to systematically map the existing literature on a particular topic or research area. It involves identifying, selecting, and synthesizing relevant papers to provide an overview of the size and scope of available evidence. Scoping reviews are broader in scope and include a diverse range of study designs and methodologies especially focused on health services research.

The main purpose of a scoping literature review is to examine the extent, range, and nature of existing studies on a topic, thereby identifying gaps in research, inconsistencies, and areas for further investigation. Additionally, scoping reviews can help researchers identify suitable methodologies and formulate clinical recommendations. They also act as the frameworks for future systematic reviews or primary research studies.

Scoping reviews are primarily focused on —

• Emerging or evolving topics — where the research landscape is still growing or budding. Example — Whole Systems Approaches to Diet and Healthy Weight: A Scoping Review of Reviews .
• Broad and complex topics : With a vast amount of existing literature.
• Scenarios where a systematic review is not feasible: Due to limited resources or time constraints.

Steps to Conduct a Scoping Literature Review

While Scoping reviews are not as rigorous as systematic reviews, however, they still follow a structured approach. Here are the steps:

Identify the research question: Define the broad topic you want to explore.

Identify Relevant Studies: Conduct a comprehensive search of relevant literature using appropriate databases, keywords, and search strategies.

Select studies to be included in the review: Based on the inclusion and exclusion criteria, determine the appropriate studies to be included in the review.

Data extraction and charting : Extract relevant information from selected studies, such as year, author, main results, study characteristics, key findings, and methodological approaches.  However, it varies depending on the research question.

Collate, summarize, and report the results: Analyze and summarize the extracted data to identify key themes and trends. Then, present the findings of the scoping review in a clear and structured manner, following established guidelines and frameworks .

Structure of a Scoping Literature Review

A scoping literature review typically follows a structured format similar to a systematic review. It includes the following sections:

• Introduction: Introduce the research topic and objectives of the review, providing the historical context, and rationale for the study.
• Methods : Describe the methods used to conduct the review, including search strategies, study selection criteria, and data extraction procedures.
• Results: Present the findings of the review, including key themes, concepts, and patterns identified in the literature review.
• Discussion: Examine the implications of the findings, including strengths, limitations, and areas for further examination.
• Conclusion: Recapitulate the main findings of the review and their implications for future research, policy, or practice.

Pros and Cons of Scoping Literature Review

• Provides a comprehensive overview of existing literature
• Helps to identify gaps and areas for further research
• Suitable for exploring broad or complex research questions
• Doesn’t provide the depth of analysis offered by systematic reviews
• Subject to researcher bias in study selection and data extraction
• Requires careful consideration of literature search strategies and inclusion criteria to ensure comprehensiveness and validity.

In short, a scoping review helps map the literature on developing or emerging topics and identifying gaps. It might be considered as a step before conducting another type of review, such as a systematic review. Basically, acts as a precursor for other literature reviews.

Example of a Well-Executed Scoping Literature Review

Paper title: Health Chatbots in Africa Literature: A Scoping Review

Check out the key differences between Systematic and Scoping reviews — Evaluating literature review: systematic vs. scoping reviews

4. Integrative Literature Review

Integrative Literature Review (ILR) is a type of literature review that proposes a distinctive way to analyze and synthesize existing literature on a specific topic, providing a thorough understanding of research and identifying potential gaps for future research.

Unlike a systematic review, which emphasizes quantitative studies and follows strict inclusion criteria, an ILR embraces a more pliable approach. It works beyond simply summarizing findings — it critically analyzes, integrates, and interprets research from various methodologies (qualitative, quantitative, mixed methods) to provide a deeper understanding of the research landscape. ILRs provide a holistic and systematic overview of existing research, integrating findings from various methodologies. ILRs are ideal for exploring intricate research issues, examining manifold perspectives, and developing new research questions.

Steps to Conduct an Integrative Literature Review

• Identify the research question: Clearly define the research question or topic of interest as formulating a clear and focused research question is critical to leading the entire review process.
• Literature search strategy: Employ systematic search techniques to locate relevant literature across various databases and sources.
• Evaluate the quality of the included studies : Critically assess the methodology, rigor, and validity of each study by applying inclusion and exclusion criteria to filter and select studies aligned with the research objectives.
• Data Extraction: Extract relevant data from selected studies using a structured approach.
• Synthesize the findings : Thoroughly analyze the selected literature, identify key themes, and synthesize findings to derive noteworthy insights.
• Critical appraisal: Critically evaluate the quality and validity of qualitative research and included studies by using BMC medical research methodology.
• Interpret and present your findings: Discuss the purpose and implications of your analysis, spotlighting key insights and limitations. Organize and present the findings coherently and systematically.

Structure of an Integrative Literature Review

• Introduction : Provide an overview of the research topic and the purpose of the integrative review.
• Methods: Describe the opted literature search strategy, selection criteria, and data extraction process.
• Results: Present the synthesized findings, including key themes, patterns, and contradictions.
• Discussion: Interpret the findings about the research question, emphasizing implications for theory, practice, and prospective research.
• Conclusion: Summarize the main findings, limitations, and contributions of the integrative review.

Pros and Cons of Integrative Literature Review

• Informs evidence-based practice and policy to the relevant stakeholders of the research.
• Contributes to theory development and methodological advancement, especially in the healthcare arena.
• Integrates diverse perspectives and findings
• Time-consuming process due to the extensive literature search and synthesis
• Requires advanced analytical and critical thinking skills
• Potential for bias in study selection and interpretation
• The quality of included studies may vary, affecting the validity of the review

Example of Integrative Literature Reviews

Paper Title: An Integrative Literature Review: The Dual Impact of Technological Tools on Health and Technostress Among Older Workers

5. Rapid Literature Review

A Rapid Literature Review (RLR) is the fastest type of literature review which makes use of a streamlined approach for synthesizing literature summaries, offering a quicker and more focused alternative to traditional systematic reviews. Despite employing identical research methods, it often simplifies or omits specific steps to expedite the process. It allows researchers to gain valuable insights into current research trends and identify key findings within a shorter timeframe, often ranging from a few days to a few weeks — unlike traditional literature reviews, which may take months or even years to complete.

When to Consider a Rapid Literature Review?

• When time impediments demand a swift summary of existing research
• For emerging topics where the latest literature requires quick evaluation
• To report pilot studies or preliminary research before embarking on a comprehensive systematic review

Steps to Conduct a Rapid Literature Review

• Define the research question or topic of interest. A well-defined question guides the search process and helps researchers focus on relevant studies.
• Determine key databases and sources of relevant literature to ensure comprehensive coverage.
• Develop literature search strategies using appropriate keywords and filters to fetch a pool of potential scientific articles.
• Screen search results based on predefined inclusion and exclusion criteria.
• Extract and summarize relevant information from the above-preferred studies.
• Synthesize findings to identify key themes, patterns, or gaps in the literature.
• Prepare a concise report or a summary of the RLR findings.

Structure of a Rapid Literature Review

An effective structure of an RLR typically includes the following sections:

• Introduction: Briefly introduce the research topic and objectives of the RLR.
• Methodology: Describe the search strategy, inclusion and exclusion criteria, and data extraction process.
• Results: Present a summary of the findings, including key themes or patterns identified.
• Discussion: Interpret the findings, discuss implications, and highlight any limitations or areas for further research
• Conclusion: Summarize the key findings and their implications for practice or future research

Pros and Cons of Rapid Literature Review

• RLRs can be completed quickly, authorizing timely decision-making
• RLRs are a cost-effective approach since they require fewer resources compared to traditional literature reviews
• Offers great accessibility as RLRs provide prompt access to synthesized evidence for stakeholders
• RLRs are flexible as they can be easily adapted for various research contexts and objectives
• RLR reports are limited and restricted, not as in-depth as systematic reviews, and do not provide comprehensive coverage of the literature compared to traditional reviews.
• Susceptible to bias because of the expedited nature of RLRs. It would increase the chance of overlooking relevant studies or biases in the selection process.
• Due to time constraints, RLR findings might not be robust enough as compared to systematic reviews.

Example of a Well-Executed Rapid Literature Review

Paper Title: What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature

A Summary of Literature Review Types

Tools and resources for conducting different types of literature reviews, online scientific databases.

Platforms such as SciSpace , PubMed , Scopus , Elsevier , and Web of Science provide access to a vast array of scholarly literature, facilitating the search and data retrieval process.

Reference management software

Tools like SciSpace Citation Generator , EndNote, Zotero , and Mendeley assist researchers in organizing, annotating, and citing relevant literature, streamlining the review process altogether.

Automate Literature Review with AI tools

Automate the literature review process by using tools like SciSpace literature review which helps you compare and contrast multiple papers all on one screen in an easy-to-read matrix format. You can effortlessly analyze and interpret the review findings tailored to your study. It also supports the review in 75+ languages, making it more manageable even for non-English speakers.

Goes without saying — literature review plays a pivotal role in academic research to identify the current trends and provide insights to pave the way for future research endeavors. Different types of literature review has their own strengths and limitations, making them suitable for different research designs and contexts. Whether conducting a narrative review, systematic review, scoping review, integrative review, or rapid literature review, researchers must cautiously consider the objectives, resources, and the nature of the research topic.

If you’re currently working on a literature review and still adopting a manual and traditional approach, switch to the automated AI literature review workspace and transform your traditional literature review into a rapid one by extracting all the latest and relevant data for your research!

There you go!

You might also like

Cybersecurity in Higher Education: Safeguarding Students and Faculty Data

How To Write An Argumentative Essay

Beyond Google Scholar: Why SciSpace is the best alternative

• University of Wisconsin–Madison
• University of Wisconsin-Madison
• Research Guides
• Evidence Synthesis, Systematic Review Services
• Literature Review Types, Taxonomies

Evidence Synthesis, Systematic Review Services : Literature Review Types, Taxonomies

• Develop a Protocol
• Develop Your Research Question
• Select Databases
• Select Gray Literature Sources
• Write a Search Strategy
• Manage Your Search Process
• Register Your Protocol
• Citation Management
• Article Screening
• Risk of Bias Assessment
• Synthesize, Map, or Describe the Results
• Find Guidance by Discipline
• Manage Your Research Data
• Browse Evidence Portals by Discipline
• Additional Resources

Choosing a Literature Review Methodology

Growing interest in evidence-based practice has driven an increase in review methodologies. Your choice of review methodology (or literature review type) will be informed by the intent (purpose, function) of your research project and the time and resources of your team. 

• Decision Tree (What Type of Review is Right for You?) Developed by Cornell University Library staff, this "decision-tree" guides the user to a handful of review guides given time and intent.

Types of Evidence Synthesis*

Critical Review - Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or model.

Mapping Review (Systematic Map) - Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature.

Meta-Analysis - Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results.

Mixed Studies Review (Mixed Methods Review) - Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies.

Narrative (Literature) Review - Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness.

Overview - Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics.

Qualitative Systematic Review or Qualitative Evidence Synthesis - Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies.

Rapid Review - Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research.

Scoping Review or Evidence Map - Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research.

State-of-the-art Review - Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives on issue or point out area for further research.

Systematic Review - Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review. (An emerging subset includes Living Reviews or Living Systematic Reviews - A [review or] systematic review which is continually updated, incorporating relevant new evidence as it becomes available.)

Systematic Search and Review - Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis.’

Umbrella Review - Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results.

*These definitions are in Grant & Booth's "A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies."

Literature Review Types/Typologies, Taxonomies

Grant, M. J., and A. Booth. "A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies."  Health Information and Libraries Journal  26.2 (2009): 91-108.  DOI: 10.1111/j.1471-1842.2009.00848.x  Link

Munn, Zachary, et al. “Systematic Review or Scoping Review? Guidance for Authors When Choosing between a Systematic or Scoping Review Approach.” BMC Medical Research Methodology , vol. 18, no. 1, Nov. 2018, p. 143. DOI: 10.1186/s12874-018-0611-x. Link

Sutton, A., et al. "Meeting the Review Family: Exploring Review Types and Associated Information Retrieval Requirements."  Health Information and Libraries Journal  36.3 (2019): 202-22.  DOI: 10.1111/hir.12276  Link

• << Previous: Home
• Next: The Systematic Review Process >>
• Last Updated: Mar 21, 2024 2:02 PM
• URL: https://researchguides.library.wisc.edu/literature_review

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Mar 26, 2024 10:40 AM
• URL: https://libguides.usc.edu/writingguide

Systematic Reviews: Types of literature review, methods, & resources

• Types of literature review, methods, & resources
• Protocol and registration
• Search strategy
• Medical Literature Databases to search
• Study selection and appraisal
• Data Extraction/Coding/Study characteristics/Results
• Reporting the quality/risk of bias
• Manage citations using RefWorks This link opens in a new window
• GW Box file storage for PDF's This link opens in a new window

Analytical reviews

GUIDELINES FOR HOW TO CARRY OUT AN ANALYTICAL REVIEW OF QUANTITATIVE RESEARCH

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Randomised trials, Systematic reviews, Study protocols, Diagnostic/prognostic studies, Case reports, Clinical practice guidelines, Animal pre-clinical studies, etc). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

When comparing therapies :

PRISMA (Guideline on how to perform and write-up a systematic review and/or meta-analysis of the outcomes reported in multiple clinical trials of therapeutic interventions. PRISMA  replaces the previous QUORUM statement guidelines ):  Liberati, A,, Altman, D,, Moher, D, et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.  Plos Medicine, 6 (7):e1000100. doi:10.1371/journal.pmed.1000100 

When comparing diagnostic methods :

Checklist for Artificial Intelligence in Medical Imaging (CLAIM). CLAIM is modeled after the STARD guideline and has been extended to address applications of AI in medical imaging that include classification, image reconstruction, text analysis, and workflow optimization. The elements described here should be viewed as a “best practice” to guide authors in presenting their research. Reported in Mongan, J., Moy, L., & Kahn, C. E., Jr (2020). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers.  Radiology. Artificial intelligence ,  2 (2), e200029. https://doi.org/10.1148/ryai.2020200029

STAndards for the Reporting of Diagnostic accuracy studies (STARD) Statement. (Reporting guidelines for writing up a study comparing the accuracy of competing diagnostic methods)  http://www.stard-statement.org/

When evaluating clinical practice guidelines :

AGREE Research Trust (ART) (2013).  Appraisal of Guidelines for Research & Evaluation (AGREE-II) . (A 23-item instrument for as sessing th e quality of Clinical Practice Guidelines. Used internationally for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.)

National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument (2019). (A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.)

When reviewing genetics studies

Human genetics review reporting guidelines.  Little J, Higgins JPT (eds.). The HuGENet™ HuGE Review Handbook, version 1.0 . 

When you need to re-analyze individual participant data

If you wish to collect, check, and re-analyze individual participant data (IPD) from clinical trials addressing a particular research question, you should follow the  PRISMA-IPD  guidelines as reported in  Stewart, L.A., Clarke, M., Rovers, M., et al. (2015). Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement. JAMA, 313(16):1657-1665. doi:10.1001/jama.2015.3656 .

When comparing Randomized studies involving animals, livestock, or food:

O’Connor AM, et al. (2010).  The REFLECT statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety by modifying the CONSORT statement.  Zoonoses Public Health. 57(2):95-104. Epub 2010/01/15. doi: 10.1111/j.1863-2378.2009.01311.x. PubMed PMID: 20070653.

Sargeant JM, et al. (2010).  The REFLECT Statement: Reporting Guidelines for Randomized Controlled Trials in Livestock and Food Safety: Explanation and Elaboration.  Zoonoses Public Health. 57(2):105-36. Epub 2010/01/15. doi: JVB1312 [pii] 10.1111/j.1863-2378.2009.01312.x. PubMed PMID: 20070652.

GUIDELINES FOR HOW TO WRITE UP FOR PUBLICATION THE RESULTS OF ONE QUANTITATIVE CLINICAL TRIAL

When reporting the results of a Randomized Controlled Trial :

Consolidated Standards of Reporting Trials (CONSORT) Statement. (2010 reporting guideline for writing up a Randomized Controlled Clinical Trial).  http://www.consort-statement.org . Since updated in 2022, see Butcher, M. A., et al. (2022). Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension . JAMA : the Journal of the American Medical Association, 328(22), 2252–2264. https://doi.org/10.1001/jama.2022.21022

Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biology, 8(6), e1000412–e1000412. https://doi.org/10.1371/journal.pbio.1000412 (A 20-item checklist, following the CONSORT approach, listing the information that published articles reporting research using animals should include, such as the number and specific characteristics of animals used; details of housing and husbandry; and the experimental, statistical, and analytical methods used to reduce bias.)

Narrative reviews

GUIDELINES  FOR HOW TO CARRY OUT  A  NARRATIVE REVIEW / QUALITATIVE RESEARCH /  OBSERVATIONAL STUDIES

Campbell, M. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ, 368. doi: https://doi.org/10.1136/bmj.l6890  (guideline on how to analyse evidence for a narrative review, to provide a recommendation based on heterogenous study types).

Community Preventive Services Task Force (2021).  The Methods Manual for Community Guide Systematic Reviews . (Public Health Prevention systematic review guidelines)

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Observational studies, Qualitative research, Quality improvement studies, and Economic evaluations). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from  https://methods.cochrane.org/qi/training-resources . (Training materials for how to do a meta-synthesis, or qualitative evidence synthesis). 

Cornell University Library (2019). Planning worksheet for structured literature reviews. Retrieved 4/8/22 from  https://osf.io/tnfm7/  (offers a framework for a narrative literature review).

Green, B. N., Johnson, C. D., & Adams, A. (2006).  Writing narrative literature reviews for peer-reviewed journals: secrets of the trade . Journal of Chiropractic Medicine, 5(3): 101-117. DOI: 10.1016/ S0899-3467 (07)60142-6.  This is a very good article about what to take into consideration when writing any type of narrative review.

When reviewing observational studies/qualitative research :

STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement. (Reporting guidelines for various types of health sciences observational studies).  http://www.strobe-statement.org 

Meta-analysis of Observational Studies in Epidemiology (MOOSE)  http://jama.jamanetwork.com/article.aspx?articleid=192614

RATS Qualitative research systematic review guidelines.  https://www.equator-network.org/reporting-guidelines/qualitative-research-review-guidelines-rats/

Methods/Guidance

Right Review , this decision support website provides an algorithm to help reviewers choose a review methodology from among 41 knowledge synthesis methods.

The Systematic Review Toolbox , an online catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Maintained by the UK University of York Health Economics Consortium, Newcastle University NIHR Innovation Observatory, and University of Sheffield School of Health and Related Research.

Institute of Medicine. (2011).  Finding What Works in Health Care: Standards for Systematic Reviews . Washington, DC: National Academies  (Systematic review guidelines from the Health and Medicine Division (HMD) of the U.S. National Academies of Sciences, Engineering, and Medicine (formerly called the Institute of Medicine)).

International Committee of Medical Journal Editors (2022).  Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals . Guidance on how to prepare a manuscript for submission to a Medical journal.

Cochrane Handbook of Systematic Reviews of Interventions (International Cochrane Collaboration systematic review guidelines). The various Cochrane review groups comporise around 30,000 physicians around the world working in the disciplines on reviews of interventions with very detailed methods for verifying the validity of the research methods and analysis performed in screened-in Randmized Controlled Clinical Trials. Typically published Cochrane Reviews are the most exhaustive review of the evidence of effectiveness of a particular drug or intervention, and include a statistical meta-analysis. Similar to practice guidelines, Cochrane reviews are periodically revised and updated.

Joanna Briggs Institute (JBI) Manual of Evidence Synthesis . (International systematic review guidelines). Based at the University of Adelaide, South Australia, and collaborating with around 80 academic and medical entities around the world. Unlike Cochrane Reviews that strictly focus on efficacy of interventions, JBI offers a broader, inclusive approach to evidence, to accommodate a range of diverse questions and study designs. The JBI manual provides guidance on how to analyse and include both quantitative and qualitative research.

Cochrane Methods Support Unit, webinar recordings on methodological support questions 

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from https://methods.cochrane.org/qi/training-resources . (How to do a meta-synthesis, or qualitative evidence synthesis). 

Center for Reviews and Dissemination (University of York, England) (2009).  Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care . (British systematic review guidelines). 

Agency for Health Research & Quality (AHRQ) (2013). Methods guide for effectiveness and comparative effectiveness reviews . (U.S. comparative effectiveness review guidelines)

Hunter, K. E., et al. (2022). Searching clinical trials registers: guide for systematic reviewers.  BMJ (Clinical research ed.) ,  377 , e068791. https://doi.org/10.1136/bmj-2021-068791

Patient-Centered Outcomes Research Institute (PCORI).  The PCORI Methodology Report . (A 47-item methodology checklist for U.S. patient-centered outcomes research. Established under the Patient Protection and Affordable Care Act, PCORI funds the development of guidance on the comparative effectivess of clinical healthcare, similar to the UK National Institute for Clinical Evidence but without reporting cost-effectiveness QALY metrics). 

Canadian Agency for Drugs and Technologies in Health (CADTH) (2019). Grey Matters: a practical tool for searching health-related grey literature. Retrieved from https://www.cadth.ca/resources/finding-evidence/grey-matters . A checklist of N American & international online databases and websites you can use to search for unpublished reports, posters, and policy briefs, on topics including general medicine and nursing, public and mental health, health technology assessment, drug and device regulatory, approvals, warnings, and advisories.

Hempel, S., Xenakis, L., & Danz, M. (2016). Systematic Reviews for Occupational Safety and Health Questions: Resources for Evidence Synthesis. Retrieved 8/15/16 from http://www.rand.org/pubs/research_reports/RR1463.html . NIOSH guidelines for how to carry out a systematic review in the occupational safety and health domain.

A good source for reporting guidelines is the  NLM's  Research Reporting Guidelines and Initiatives .

Grading of Recommendations Assessment, Development and Evaluation (GRADE). (An international group of academics/clinicians working to promote a common approach to grading the quality of evidence and strength of recommendations.) 

Phillips, B., Ball, C., Sackett, D., et al. (2009). Oxford Centre for Evidence Based Medicine: Levels of Evidence. Retrieved 3/20/17 from https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf . (Another commonly used criteria for grading the quality of evidence and strength of recommendations, developed in part by EBM guru David Sackett.) 

Systematic Reviews for Animals & Food  (guidelines including the REFLECT statement for carrying out a systematic review on animal health, animal welfare, food safety, livestock, and agriculture)

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x. (Describes 14 different types of literature and systematic review, useful for thinking at the outset about what sort of literature review you want to do.)

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements . Health information and libraries journal, 36(3), 202–222. doi:10.1111/hir.12276  (An updated look at different types of literature review, expands on the Grant & Booth 2009 article listed above).

Garrard, J. (2007).  Health Sciences Literature Review Made Easy: The Matrix Method  (2nd Ed.).   Sudbury, MA:  Jones & Bartlett Publishers. (Textbook of health sciences literature search methods).

Zilberberg, M. (2012).  Between the lines: Finding the truth in medical literature . Goshen, MA: Evimed Research Press. (Concise book on foundational concepts of evidence-based medicine).

Lang, T. (2009). The Value of Systematic Reviews as Research Activities in Medical Education . In: Lang, T. How to write, publish, & present in the health sciences : a guide for clinicians & laboratory researchers. Philadelphia : American College of Physicians.  (This book chapter has a helpful bibliography on systematic review and meta-analysis methods)

Brown, S., Martin, E., Garcia, T., Winter, M., García, A., Brown, A., Cuevas H.,  & Sumlin, L. (2013). Managing complex research datasets using electronic tools: a meta-analysis exemplar . Computers, Informatics, Nursing: CIN, 31(6), 257-265. doi:10.1097/NXN.0b013e318295e69c. (This article advocates for the programming of electronic fillable forms in Adobe Acrobat Pro to feed data into Excel or SPSS for analysis, and to use cloud based file sharing systems such as Blackboard, RefWorks, or EverNote to facilitate sharing knowledge about the decision-making process and keep data secure. Of particular note are the flowchart describing this process, and their example screening form used for the initial screening of abstracts).

Brown, S., Upchurch, S., & Acton, G. (2003). A framework for developing a coding scheme for meta-analysis . Western Journal Of Nursing Research, 25(2), 205-222. (This article describes the process of how to design a coded data extraction form and codebook, Table 1 is an example of a coded data extraction form that can then be used to program a fillable form in Adobe Acrobat or Microsoft Access).

Elamin, M. B., Flynn, D. N., Bassler, D., Briel, M., Alonso-Coello, P., Karanicolas, P., & ... Montori, V. M. (2009). Choice of data extraction tools for systematic reviews depends on resources and review complexity .  Journal Of Clinical Epidemiology ,  62 (5), 506-510. doi:10.1016/j.jclinepi.2008.10.016  (This article offers advice on how to decide what tools to use to extract data for analytical systematic reviews).

Riegelman R.   Studying a Study and Testing a Test: Reading Evidence-based Health Research , 6th Edition.  Lippincott Williams & Wilkins, 2012. (Textbook of quantitative statistical methods used in health sciences research).

Rathbone, J., Hoffmann, T., & Glasziou, P. (2015). Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Systematic Reviews, 480. doi:10.1186/s13643-015-0067-6

Guyatt, G., Rennie, D., Meade, M., & Cook, D. (2015). Users' guides to the medical literature (3rd ed.). New York: McGraw-Hill Education Medical.  (This is a foundational textbook on evidence-based medicine and of particular use to the reviewer who wants to learn about the different types of published research article e.g. "what is a case report?" and to understand what types of study design best answer what types of clinical question).

Glanville, J., Duffy, S., Mccool, R., & Varley, D. (2014). Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? Journal of the Medical Library Association : JMLA, 102(3), 177–183. https://doi.org/10.3163/1536-5050.102.3.007

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan a web and mobile app for systematic reviews.  Systematic Reviews, 5 : 210, DOI: 10.1186/s13643-016-0384-4. http://rdcu.be/nzDM

Kwon Y, Lemieux M, McTavish J, Wathen N. (2015). Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 103 (4): 184-8. doi: 10.3163/1536-5050.103.4.004. https://www.ncbi.nlm.nih.gov/pubmed/26512216

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. (2016). De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 104 (3):240-3. doi: 10.3163/1536-5050.104.3.014. Erratum in: J Med Libr Assoc. 2017 Jan;105(1):111. https://www.ncbi.nlm.nih.gov/pubmed/27366130

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–46. doi: 10.1016/j.jclinepi.2016.01.021 . PRESS is a guideline with a checklist for librarians to critically appraise the search strategy for a systematic review literature search.

Clark, JM, Sanders, S, Carter, M, Honeyman, D, Cleo, G, Auld, Y, Booth, D, Condron, P, Dalais, C, Bateup, S, Linthwaite, B, May, N, Munn, J, Ramsay, L, Rickett, K, Rutter, C, Smith, A, Sondergeld, P, Wallin, M, Jones, M & Beller, E 2020, 'Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial',  Journal of the Medical Library Association , vol. 108, no. 2, pp. 195-207.

Journal articles describing systematic review methods can be searched for in PubMed using this search string in the PubMed search box: sysrev_methods [sb] . 

Software tools for systematic reviews

• Covidence GW in 2019 has bought a subscription to this Cloud based tool for facilitating screening decisions, used by the Cochrane Collaboration. Register for an account.
• NVIVO for analysis of qualitative research NVIVO is used for coding interview data to identify common themes emerging from interviews with several participants. GW faculty, staff, and students may download NVIVO software.
• RedCAP RedCAP is software that can be used to create survey forms for research or data collection or data extraction. It has very detailed functionality to enable data exchange with Electronic Health Record Systems, and to integrate with study workflow such as scheduling follow up reminders for study participants.
• SRDR tool from AHRQ Free, web-based and has a training environment, tutorials, and example templates of systematic review data extraction forms
• RevMan 5 RevMan 5 is the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews
• Rayyan Free, web-based tool for collecting and screening citations. It has options to screen with multiple people, masking each other.
• GradePro Free, web application to create, manage and share summaries of research evidence (called Evidence Profiles and Summary of Findings Tables) for reviews or guidelines, uses the GRADE criteria to evaluate each paper under review.
• DistillerSR Needs subscription. Create coded data extraction forms from templates.
• EPPI Reviewer Needs subscription. Like DistillerSR, tool for text mining, data clustering, classification and term extraction
• SUMARI Needs subscription. Qualitative data analysis.
• Dedoose Needs subscription. Qualitative data analysis, similar to NVIVO in that it can be used to code interview transcripts, identify word co-occurence, cloud based.
• Meta-analysis software for statistical analysis of data for quantitative reviews SPSS, SAS, and STATA are popular analytical statistical software that include macros for carrying out meta-analysis. Himmelfarb has SPSS on some 3rd floor computers, and GW affiliates may download SAS to your own laptop from the Division of IT website. To perform mathematical analysis of big data sets there are statistical analysis software libraries in the R programming language available through GitHub and RStudio, but this requires advanced knowledge of the R and Python computer languages and data wrangling/cleaning.
• PRISMA 2020 flow diagram generator The PRISMA Statement website has a page listing example flow diagram templates and a link to software for creating PRISMA 2020 flow diagrams using R software.

GW researchers may want to consider using Refworks to manage citations, and GW Box to store the full text PDF's of review articles. You can also use online survey forms such as Qualtrics, RedCAP, or Survey Monkey, to design and create your own coded fillable forms, and export the data to Excel or one of the qualitative analytical software tools listed above.

Forest Plot Generators

• RevMan 5 the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews.
• Meta-Essentials a free set of workbooks designed for Microsoft Excel that, based on your input, automatically produce meta-analyses including Forest Plots. Produced for Erasmus University Rotterdam joint research institute.
• Neyeloff, Fuchs & Moreira Another set of Excel worksheets and instructions to generate a Forest Plot. Published as Neyeloff, J.L., Fuchs, S.C. & Moreira, L.B. Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC Res Notes 5, 52 (2012). https://doi-org.proxygw.wrlc.org/10.1186/1756-0500-5-52
• For R programmers instructions are at https://cran.r-project.org/web/packages/forestplot/vignettes/forestplot.html and you can download the R code package from https://github.com/gforge/forestplot
• << Previous: Home
• Next: Protocol and registration >>

• Last Updated: Mar 21, 2024 11:08 AM
• URL: https://guides.himmelfarb.gwu.edu/systematic_review

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

Literature Review - Finding the Resources

The "literature" for your literature review, reasons for a literature review, types of literature review, are you ready, books are helpful, encyclopedias.

• Search Tools
• Formulating your search statement
• Buliding on what you have found
• Keeping Track
• Academic Reading
• Citing Sources

The type and the number of sources you need for your literature review may vary depending on the purpose of your literature review (see below) and your level of study. Therefore, you have to know very well the purpose and the type of your literature review.

Through a Literature review, you can gain a thorough understanding of the knowledge and ideas established on a topic, so that you can:

• Convey to your reader the current state of research in the field
• Identify in previous research the gap which a new research question can address
• Identify potential hypotheses and research methods for your research
• Give evidence to justify the value and need for your research

To meet different purposes, your literature review can be of various types. It can be:

• A course assignment to show how much you know what others have discovered on a topic, and to analyze and evaluate the literature you read
• A chapter-length review of literature included in your dissertation for you to establish a foundation for your own research topic and to show that your own research is in a particular context of what is known about a subject
• A short literature review near the beginning of a research article to give your reader sufficient background and context to understand your work
• A stand-alone review article to give your reader a comprehensive survey of the major researchers, discoveries, gaps, debates and directions in a field
• A proposal for academic degrees or for funding to show that the proposed research builds upon and plans to fill the gap of existing research 

Before you start your literature search focusing on scholarly writings and original research studies, ask yourself if you are ready. Do you know the topic of your research well enough to get started? If not, try first to get some background information of the topic .

A way to get background information is to read some books or reference materials on your research topic. Books generally give an overview of a topic or contain summary of research on a topic . Locate books on a topic using our CityU LibraryFind .

For example, if you are researching on crisis management, the following two books are good places for you to start with.

The book " Crisis communication: Theory and practice " written by A. J. Zaremba provides an overview of the topic "crisis communication". Another book " The handbook of crisis management " edited by W.T. Coombs and S.J. Holladay contains a summary of research on this topic written by multiple authors. 

Encyclopedias are also good sources for general background information . Below are some suggested general encyclopedias:

• Credo Reference This link opens in a new window
• Britannica Academic This link opens in a new window
• << Previous: Home
• Next: Search Tools >>
• Last Updated: Jan 17, 2024 7:25 PM
• URL: https://libguides.library.cityu.edu.hk/litreview

© City University of Hong Kong | Copyright | Disclaimer

Materials Science and Engineering

• Getting Started
• Dictionaries, Encyclopedias, and Handbooks
• Material Properties
• Technical Reports
• Writing & Citing

Introduction

Gather Your Tools

Determine the Project's Scope

Create the Search Strategy

Determine What Resources to Use

Search, read, refine, repeat.

Saved Searches, Alerts and Feeds

This page focuses on how to do an in-depth literature review for a dissertation, thesis, grant application or lengthy term paper in electrical engineering.  

• For a more general description of what an in-depth literature review is and how it looks, see our guide on " Literature Reviews and Annotated Bibliographies " created by Ed Oetting, history and political science librarian.
• For lower-level engineering undergraduate students who are doing a short term paper, the " How to Research a Topic " page on the " Engineerng Basics" guide may be more applicable.

Library Account Is your library account clear of fines?   If not, you may not be allowed to check out more books nor renew books you already have.  All library notices are sent via email to your "asu.edu" address; if you prefer to receive email at a different address make sure you have forwarded your asu.edu correctly.  Also, make sure that your spam filter allows the library email to come through. 

Illiad (Interlibrary Loan) Account   If you don't already have an ILLiad account, please register for one.  Interlibrary loan services will get you material not available at the ASU Library and also scan or deliver materials from the libraries on the other ASU campuses.

Determine the Project's Scope.

Do you know what you are looking for?  Can you describe your project using one simple sentence or can you phrase the project as a question?  Without a clear idea of the project, you may not be able to determine which are the best resources to search, what terminology should be used in those resources, and if the results are appropriate and sufficient.    

If you're having difficulty getting your project described succinctly, try using a PICO chart to identify the concepts involved:

• P is the popluation, problem, predicament or process
• I is the intervention or improvement
• C is what you'll compare your intervention/improvment to, and
• O is the outcome (or results of the comparison of I and C ) 

For example: 

Your client, the owner of a nuclear power generating facility, has had several less than optimal safety inspections recently.  The inspectors have singled out operator error as a major concern and have required changes in employee training.  But is more training the solution?  The employees complain that the plant's poorly designed control room hampers their ability to respond to non-standard situations.  Could a redesign improve performance and decrease the occurance of unsafe events?   Your client wants more than just your opinion, he wants to see the data to back it up.   So, what can you find in the literature?

Here's one way that the PICO chart could be filled out:     

• P =   nuclear power safety  
• I  = human factors engineering
• C  =  additional training; little or no human factors engineering used  
• O = accident rate or safety inspection comparison

And here are examples of possible search statements:  

• I am looking for ways that human factors engineering can improve safety in the nuclear power industry.
• Is additional training or employing human factors engineering the better method for reducing safety violations in a nuclear power plant? 

Your research will always start with a " P AND I " search; those are the most important pieces of the puzzle.  However, once you have the results from that search, you'll need to know where you want to go with those results; that's when the C and O concepts need to be considered.  

 Also, don't forget --- determine if your project has limits.  For example:

• Are you reviewing the literature only within a specific time frame?
• Are you looking at English-language material only?
• Are you considering research from just the United States or worldwide?
• Are there types of material you won't be covering (trade magazines, patents, technical reports, etc.)?  

Take the simple sentence or question that describes what you are looking for.  What are the concepts in the sentence? Are there synonyms that describe the same concept?   If you filled out a PICO chart, concentrate on the  P (problem) and the  I (intervention) for the concept chart.  

Concept Chart:

Concept 1:   _______  OR _______  OR _______  AND Concept 2:   _______   OR   _______  OR   _______  AND Concept 3:  _______  OR  _______  OR   _______ 

  Example:  

I am looking for ways that human factors engineering can improve safety in the nuclear power industry. 

Concept 1:   nuclear power    OR _ nuclear industry _____   AND Concept 2:   _safety___  OR   _accident prevention____   AND Concept 3: _ human factors engineering ___   

What resources you'll use for your literature review depends on what types of materials you want to find.  

• Background Information The more you know about a topic, the better you'll be able to research it.  You'll be familiar with the terminology, understand the underlining science/technology and be aware of the issues in the field. Most importantly, you'll be able to understand what you've retrieved from your search.  But no matter how much you know before hand you'll likely run across terms and concepts with which you're unfamiliar.    Materials such as encyclopedias, dictionaries and handbooks will not only help you learn about the basics of your topic before you begin your search but they'll also help you understand the terminology used in the documents you found from your literature review.    You'll find these types of resources listed on the Dictionaries and Handbooks pages on this guide.
• Books The large size of books (usually 100-500 pages) allows a topic to be studied braodly, covering many different issues.  Conversely, the large size also allows for a specific aspect of the topic to be covered in great detail.  Because of the time it takes to publish, sci-tech books generally do not contain the most current information. To find print and online books from both the ASU Library as well as in other libraries, see the Books page on this guide.
• Conference Papers Scientists and engineers frequently present new findings at conferences before these findings are written up in journal articles or books.  Not every conference, however, publishes it proceedings.  In some cases, conferences publish only a few of the papers presented but not all.   Many resources that help you find journal articles, may also be used to find conference papers, see the Articles page on this guide.
• Journal and Trade Magazine Articles Articles in journals (also called magazines) are short, usually 5-20 pages in length and cover a specific finding, experiment or project.  Articles in scholary journals are usually written by academics or professional scientists/engineers and are aimed at others at the same level.   Articles in trade journals/magazines are written by the journal staff and report on industry news suchs as sales, mergers, prices, etc.   To find journal and trade magazine articles, use the resources listed on the Articles page on this guide. 
• Patents Patents are grants from governments that gives the inventor certain rights of manufacture.  Patents provide a wealth of information for how a technology is being advanced and by which companies.  It is frequently stated that 80% of the information in patents never appears elsewhere in the literature. 
• To identify patents granted in the U.S. and internationally see the " Searching for Patents " guide.
• To see statistical information for U.S. patents by technology class see the US Patent and Trademark's website.
• Technical Reports Technical reports are part of the "gray literature";  gray literature refers to documents that are not published commercially, hence they are difficult to both identify and find.  Technical reports focus on a specific experiment or research project and are meant to convey the results of the experiment or project back to the funding organization.  In the United States, common sources of technical reports are the government agencies that sponsor research projects.  Reports generated within a private corporation and funded soley by that corporation are seldom ever available to anyone outside of the company.      To find technical reports, use the resources listed on the Technical Reports page on this guide.     

Search, Read, Refine and Repeat

Now it's time to apply your search strategy in the resources you've decided to use.

• Use the Advanced Search feature (or whatever search is set up with the 3 lines of boxes) and enter your search strategy just as you recorded in your search strategy chart.  Don't forget to set your limits.   If the resource only provides a single search box, rearrange your chart from vertical into horizontal so that the search statement looks like this:   (Concept#1 OR synonym) AND (Concept#2 OR synonym) AND (Concept#3 OR synonym) Example: (nuclear power OR nuclear industry) AND (safety OR accident prevention) AND (human factors engineering)
• Examine the results to find the most appropriate items.  Keep your one-sentence project description (and/or your PICO chart) in mind to help you stay on track.
• Export the records/citations you want to keep into a citation manager.
• If there are subjects (may also be called subject headings, index terms, descriptors or controlled vocabulary) assigned to each item, make sure that those also transferred into a citation manager.  If not, add them manually.
• Get the full text of the items 
• Read the full text of the items and look at the subjects assigned to the item and consider:
• Do I have to change (narrow) my topic to something more specific because I'm finding way too much? 
• Do I have to change (broaden) my topic because I can't find enough about it? 
• Is there additional terminology for my topic/concepts that I hadn't included in my original search?
• Redo your search strategy according to what you found in step #6 and rerun the search in the resources again.
• You may need to repeat this cycle several times before you are able to identify the best terminology to use in each resource. 

If there will be several months in between when you search the literature and when you turn in the paper, consider setting up alerts and feeds so that you are notified should new items about your topic appear.   How you set up an alert or feed will vary.  In most cases you'll be required to set up a personal account or profile with the journal or database --- there is no charge for this but you will have to identify yourself and provide an email address.  

For instructions on setting up alerts and feeds, see the " Keep Current " page.

• << Previous: Writing & Citing
• Last updated: Jan 2, 2024 8:27 AM
• URL: https://libguides.asu.edu/mse

The ASU Library acknowledges the twenty-three Native Nations that have inhabited this land for centuries. Arizona State University's four campuses are located in the Salt River Valley on ancestral territories of Indigenous peoples, including the Akimel O’odham (Pima) and Pee Posh (Maricopa) Indian Communities, whose care and keeping of these lands allows us to be here today. ASU Library acknowledges the sovereignty of these nations and seeks to foster an environment of success and possibility for Native American students and patrons. We are advocates for the incorporation of Indigenous knowledge systems and research methodologies within contemporary library practice. ASU Library welcomes members of the Akimel O’odham and Pee Posh, and all Native nations to the Library.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PLoS Comput Biol
• v.9(7); 2013 Jul

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

Literature Reviews

• Getting Started

Selecting a Review Type

Defining the scope of your review, four common types of reviews.

• Developing a Research Question
• Searching the Literature
• Searching Tips
• ChatGPT [beta]
• Documenting your Search
• Using Citation Managers
• Concept Mapping
• Writing the Review
• Further Resources

More Review Types

This article by Sutton & Booth (2019) explores 48 distinct types of Literature Reviews:

Which Review is Right for You?

The  Right Review tool  has questions about your lit review process and plans. It offers a qualitative and quantitative option. At completion, you are given a lit review type recommendation.

You'll want to think about the kind of review you are doing. Is it a selective or comprehensive review? Is the review part of a larger work or a stand-alone work ?

For example, if you're writing the Literature Review section of a journal article, that's a selective review which is part of a larger work. Alternatively, if you're writing a review article, that's a comprehensive review which is a stand-alone work. Thinking about this will help you develop the scope of the review.

This exercise will help define the scope of your Literature Review, setting the boundaries for which literature to include and which to exclude.

A FEW GENERAL CONSIDERATIONS WHEN DEFINING SCOPE

• Which populations to investigate — this can include gender, age, socio-economic status, race, geographic location, etc., if the research area includes humans.
• What years to include — if researching the legalization of medicinal cannabis, you might only look at the previous 20 years; but if researching dolphin mating practices, you might extend many more decades.
• Which subject areas — if researching artificial intelligence, subject areas could be computer science, robotics, or health sciences
• How many sources  — a selective review for a class assignment might only need ten, while a comprehensive review for a dissertation might include hundreds. There is no one right answer.
• There will be many other considerations that are more specific to your topic. 

Most databases will allow you to limit years and subject areas, so look for those tools while searching. See the Searching Tips tab for information on how use these tools.

LITERATURE REVIEW

• Often used as a generic term to describe any type of review
• More precise definition:  Published materials that provide an examination of published literature . Can cover wide range of subjects at various levels of comprehensiveness.
• Identifies gaps in research, explains importance of topic, hypothesizes future work, etc.
• Usually written as part of a larger work like a journal article or dissertation

SCOPING REVIEW

• Conducted to address broad research questions with the goal of understanding the extent of research that has been conducted.
• Provides a preliminary assessment of the potential size and scope of available research literature. It aims to identify the nature and extent of research evidence (usually including ongoing research) 
• Doesn't assess the quality of the literature gathered (i.e. presence of literature on a topic shouldn’t be conflated w/ the quality of that literature)

SYSTEMATIC REVIEW

• Common in the health sciences ( Taubman Health Sciences Library guide to Systematic Reviews )
• Goal: collect all literature that meets specific criteria (methodology, population, treatment, etc.) and then appraise its quality and synthesize it
• Follows strict protocol for literature collection, appraisal and synthesis
• Typically performed by research teams 
• Takes 12-18 months to complete
• Often written as a stand alone work

META-ANALYSIS

• Goes one step further than a systematic review by statistically combining the results of quantitative studies to provide a more precise effect of the results. 
• << Previous: Getting Started
• Next: Developing a Research Question >>
• Last Updated: Feb 29, 2024 10:31 AM
• URL: https://guides.lib.umich.edu/litreview

Types of Literature Review

There are many types of literature review. The choice of a specific type depends on your research approach and design. The following types of literature review are the most popular in business studies:

Narrative literature review , also referred to as traditional literature review, critiques literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps or inconsistencies in a body of knowledge. You need to have a sufficiently focused research question to conduct a narrative literature review

Systematic literature review requires more rigorous and well-defined approach compared to most other types of literature review. Systematic literature review is comprehensive and details the timeframe within which the literature was selected. Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis.

When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn. Meta-analysis is associated with deductive research approach.

Meta-synthesis, on the other hand, is based on non-statistical techniques. This technique integrates, evaluates and interprets findings of multiple qualitative research studies. Meta-synthesis literature review is conducted usually when following inductive research approach.

Scoping literature review , as implied by its name is used to identify the scope or coverage of a body of literature on a given topic. It has been noted that “scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review.” [1] The main difference between systematic and scoping types of literature review is that, systematic literature review is conducted to find answer to more specific research questions, whereas scoping literature review is conducted to explore more general research question.

Argumentative literature review , as the name implies, examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. It should be noted that a potential for bias is a major shortcoming associated with argumentative literature review.

Integrative literature review reviews , critiques, and synthesizes secondary data about research topic in an integrated way such that new frameworks and perspectives on the topic are generated. If your research does not involve primary data collection and data analysis, then using integrative literature review will be your only option.

Theoretical literature review focuses on a pool of theory that has accumulated in regard to an issue, concept, theory, phenomena. Theoretical literature reviews play an instrumental role in establishing what theories already exist, the relationships between them, to what degree existing theories have been investigated, and to develop new hypotheses to be tested.

At the earlier parts of the literature review chapter, you need to specify the type of your literature review your chose and justify your choice. Your choice of a specific type of literature review should be based upon your research area, research problem and research methods.  Also, you can briefly discuss other most popular types of literature review mentioned above, to illustrate your awareness of them.

[1] Munn, A. et. al. (2018) “Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach” BMC Medical Research Methodology

  John Dudovskiy

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Digital Health Device Collection
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives
• Systematic Reviews
• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations

Database Search Tips

• Getting Started with Your Search
• Type of Literature Reviews
• Finding Full-Text
• Getting Help
• Cochrane Library
• Education Full Text
• Johns Hopkins ABX Guide
• Lippincott Advisor
• Lippincott Procedures
• ProQuest Dissertations & Theses Global
• PubMed/MEDLINE (opens new tab) This link opens in a new window
• Sociology Abstracts
• Sociology Source Ultimate
• SportDiscus

Types of Reviews

There are many types of evidence synthesis projects, including systematic reviews as well as others. The review type selection is dependent on the research question, the research team's time and ability, and the parameters of the assignment or journal you are submitting to for publication,

Librarians can help your team determine which review type might be appropriate for your project, what databases to search, and how to manage your results.

Adapted from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Getting Started with Your Search
• Next: Finding Full-Text >>
• Last Updated: Mar 27, 2024 10:45 AM
• URL: https://guides.mclibrary.duke.edu/searchtips
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

• About the Library About Us Library Hours Main Campus Floorplan Library Staff Archives
• Research Help A-Z List of Databases Book a Librarian Book Catalog Journals Research Guides Interlibrary Loan Email a Librarian
• Faculty Resources Request Library Instruction Open Educational Resources Information Literacy Interlibrary Loan Copyright Basics WRT Information Literacy Modules Civic Engagement

Literature Review: Types of Literature Reviews

• Introduction
• Types of Literature Reviews
• Research Questions
• Search the Literature
• Manage and Evaluate References
• Nursing Literature Review Search Tips

5 of the Most Popular Literature Reviews

Methodological

Theoretical

• Critiques and summarizes body of literature,
• Researched from relevant databases with selective materials.
• Requires focused research question.
• Gives comprehensive overview of topic and highlight significant areas of research.
• Criteria for selection not always made open to reader.
• Subject to bias that supports theses.
• Overview of existing evidence pertinent to research question.
• Focuses on very specific empirical questions (cause-and-effect form)
• Uses pre-specified /standardized methods to identify/critically appraise research.
• Rigorous and well-defined approach to searching.
• Details time frame of selection and methods of evaluation/synthesis.

Example of a Systematic Literature Review

• Focuses on examining research throughout period of time and tracing its evolution.
• Places research in a historical context to show familiarity and identify likely future directions.
• Focuses on methods of analysis in previous research.
• Provides understanding of theory, research approaches, data collection, and analysis techniques.
• Helps highlights ethical issues regarding conducting research.

Example of a Methodological Literature Review

• Examines accumulated theories in regard to issues, concepts, or phenomena.
• Establishes existing theories and relationships between theories; develops new hypotheses.
• Reveals inadequacies of current theories and explains need for new/emerging research.

Example of a Theoretical Literature Review

• << Previous: Introduction
• Next: Research Questions >>
• Last Updated: Sep 12, 2023 10:51 AM
• URL: https://library.madonna.edu/literaturereview

Literature Reviews

• "How To" Books
• Examples of Literature Reviews
• Collecting Resources for a Literature Review
• Organizing the Literature Review
• Writing the Literature Review
• Endnote This link opens in a new window
• Evaluating Websites

Organization

Organization of your Literature Review

What is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? What order should you present them?

Just like most academic papers, literature reviews must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper.

Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.

Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).

Conclusions/Recommendations: Discuss what you have drawn from reviewing the literature so far. Where might the discussion proceed?

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario and then three typical ways of organizing the sources into a review:

You've decided to focus your literature review on materials dealing with sperm whales. This is because you've just finished reading Moby Dick, and you wonder if that whale's portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980's. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Chronological

If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.

By publication

Order your sources chronologically by publication if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.

Another way to organize sources chronologically is to examine the sources under a trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Using this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.

Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most: the development of the harpoon or the harpoon technology.

More authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as "evil" in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.

Methodological

A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the "methods" of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Once you've decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

Current Situation: Information necessary to understand the topic or focus of the literature review.

History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.

Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

• << Previous: Collecting Resources for a Literature Review
• Next: Writing the Literature Review >>
• Last Updated: Nov 2, 2021 12:11 PM
• URL: https://guides.library.stonybrook.edu/literature-review
• Request a Class
• Hours & Locations
• Ask a Librarian
• Special Collections
• Library Faculty & Staff

Library Administration: 631.632.7100

• Stony Brook Home
• Campus Maps
• Web Accessibility Information
• Accessibility Barrier Report Form

Comments or Suggestions? | Library Webmaster

Except where otherwise noted, this work by SBU Libraries is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

• University of Texas Libraries
• UT Libraries

Systematic Reviews & Evidence Synthesis Methods

Types of reviews.

• Formulate Question
• Find Existing Reviews & Protocols
• Register a Protocol
• Searching Systematically
• Supplementary Searching
• Managing Results
• Deduplication
• Critical Appraisal
• Glossary of terms
• Librarian Support
• Video tutorials This link opens in a new window
• Systematic Review & Evidence Synthesis Boot Camp

Not sure what type of review you want to conduct?

There are many types of reviews ---  narrative reviews ,  scoping reviews , systematic reviews, integrative reviews, umbrella reviews, rapid reviews and others --- and it's not always straightforward to choose which type of review to conduct. These Review Navigator tools (see below) ask a series of questions to guide you through the various kinds of reviews and to help you determine the best choice for your research needs.

• Which review is right for you? (Univ. of Manitoba)
• What type of review is right for you? (Cornell)
• Review Ready Reckoner - Assessment Tool (RRRsAT)
• A typology of reviews: an analysis of 14 review types and associated methodologies. by Grant & Booth
• Meeting the review family: exploring review types and associated information retrieval requirements | Health Info Libr J, 2019

Reproduced from Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies . Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x

• Last Updated: Feb 27, 2024 12:53 PM
• URL: https://guides.lib.utexas.edu/systematicreviews

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 21 March 2024

Expert review of the science underlying nature-based climate solutions

• B. Buma   ORCID: orcid.org/0000-0003-2402-7737 1 , 2   na1 ,
• D. R. Gordon   ORCID: orcid.org/0000-0001-6398-2345 1 , 3   na1 ,
• K. M. Kleisner 1 ,
• A. Bartuska 1 , 4 ,
• A. Bidlack 5 ,
• R. DeFries   ORCID: orcid.org/0000-0002-3332-4621 6 ,
• P. Ellis   ORCID: orcid.org/0000-0001-7933-8298 7 ,
• P. Friedlingstein   ORCID: orcid.org/0000-0003-3309-4739 8 , 9 ,
• S. Metzger 10   nAff15   nAff16 ,
• G. Morgan 11 ,
• K. Novick   ORCID: orcid.org/0000-0002-8431-0879 12 ,
• J. N. Sanchirico 13 ,
• J. R. Collins   ORCID: orcid.org/0000-0002-5705-9682 1 , 14 ,
• A. J. Eagle   ORCID: orcid.org/0000-0003-0841-2379 1 ,
• R. Fujita 1 ,
• E. Holst 1 ,
• J. M. Lavallee   ORCID: orcid.org/0000-0002-3028-7087 1 ,
• R. N. Lubowski 1   nAff17 ,
• C. Melikov 1   nAff18 ,
• L. A. Moore   ORCID: orcid.org/0000-0003-0239-6080 1   nAff19 ,
• E. E. Oldfield   ORCID: orcid.org/0000-0002-6181-1267 1 ,
• J. Paltseva 1   nAff20 ,
• A. M. Raffeld   ORCID: orcid.org/0000-0002-5036-6460 1 ,
• N. A. Randazzo 1   nAff21   nAff22 ,
• C. Schneider 1 ,
• N. Uludere Aragon 1   nAff23 &
• S. P. Hamburg 1  

Nature Climate Change ( 2024 ) Cite this article

10k Accesses

43 Altmetric

Metrics details

• Climate-change ecology
• Climate-change mitigation
• Environmental impact

Viable nature-based climate solutions (NbCS) are needed to achieve climate goals expressed in international agreements like the Paris Accord. Many NbCS pathways have strong scientific foundations and can deliver meaningful climate benefits but effective mitigation is undermined by pathways with less scientific certainty. Here we couple an extensive literature review with an expert elicitation on 43 pathways and find that at present the most used pathways, such as tropical forest conservation, have a solid scientific basis for mitigation. However, the experts suggested that some pathways, many with carbon credit eligibility and market activity, remain uncertain in terms of their climate mitigation efficacy. Sources of uncertainty include incomplete GHG measurement and accounting. We recommend focusing on resolving those uncertainties before broadly scaling implementation of those pathways in quantitative emission or sequestration mitigation plans. If appropriate, those pathways should be supported for their cobenefits, such as biodiversity and food security.

Similar content being viewed by others

Australian human-induced native forest regeneration carbon offset projects have limited impact on changes in woody vegetation cover and carbon removals

Andrew Macintosh, Don Butler, … Paul Summerfield

Meta-analysis shows the impacts of ecological restoration on greenhouse gas emissions

Tiehu He, Weixin Ding, … Quanfa Zhang

Historical impacts of grazing on carbon stocks and climate mitigation opportunities

Shuai Ren, César Terrer, … Dan Liu

Nature-based climate solutions (NbCS) are conservation, restoration and improved management strategies (pathways) in natural and working ecosystems with the primary motivation to mitigate GHG emissions and remove CO 2 from the atmosphere 1 (similar to ecosystem-based mitigation 2 ). GHG mitigation through ecosystem stewardship is integral to meeting global climate goals, with the greatest benefit coming from near-term maximization of emission reductions, followed by CO 2 removal 3 . Many countries (for example, Indonesia, China and Colombia) use NbCS to demonstrate progress toward national climate commitments.

The scope of NbCS is narrower than that of nature-based solutions (NbS) which include interventions that prioritize non-climate benefits alongside climate (for example, biodiversity, food provisioning and water quality improvement) 4 . In many cases, GHG mitigation is considered a cobenefit that results from NbS actions focused on these other challenges 2 . In contrast, NbCS are broader than natural climate solutions, which are primarily focused on climate mitigation through conservation, restoration and improved land management, generally not moving ecosystems beyond their unmodified structure, function or composition 5 . NbCS may involve moving systems beyond their original function, for example by cultivating macroalgae in water deeper than their natural habitat.

The promise of NbCS has generated a proliferation of interest in using them in GHG mitigation plans 6 , 7 ; 104 of the 168 signatories to the Paris Accord included nature-based actions as part of their mitigation plans 8 . Success in long-term GHG management requires an accurate accounting of inputs and outputs to the atmosphere at scale, so NbCS credits must have robust, comprehensive and transparent scientific underpinnings 9 . Given the urgency of the climate problem, our goal is to identify NbCS pathways with a sufficient scientific foundation to provide broad confidence in their potential GHG mitigation impact, provide resources for confident implementation and identify priority research areas in more uncertain pathways. Evaluating implementation of mitigation projects is beyond our scope; this effort focuses on understanding the underlying science. The purpose is not evaluating any specific carbon crediting protocol or implementation framework but rather the current state of scientific understanding necessary to provide confidence in any NbCS.

In service of this goal, we first investigated nine biomes (boreal forests, coastal marine (salt marsh, mangrove, seagrass and coral reef), freshwater wetlands, grasslands, open ocean (large marine animal and mesopelagic zone biomass, seabed), peatlands, shrublands, temperate forests and tropical forests) and three cultivation types (agroforestry, croplands and macroalgae aquaculture); these were chosen because of their identified potential scale of global impact. In this context, impact is assessed as net GHG mitigation: the CO 2 sequestered or emissions reduced, for example, discounted by understood simultaneous emissions of other GHG (as when N 2 O is released simultaneously with carbon sequestration in cropland soils). From there, we identified 43 NbCS pathways which have been formally implemented (with or without market action) or informally proposed. We estimated the scale of mitigation impact for each pathway on the basis of this literature and, as a proxy measure of NbCS implementation, determined eligibility and activity under existing carbon crediting protocols. Eligibility means that the pathway is addressed by an existing GHG mitigation protocol; market activity means that credits are actively being bought under those eligibility requirements. We considered pathways across a spectrum from protection to improved management to restoration to manipulated systems, but some boundaries were necessary. We excluded primarily abiotically driven pathways (for example, ocean alkalinity enhancement) or where major land use or land-use trade-offs exist (for example, afforestation) 10 , 11 , 12 . Of the 43 pathways, 79% are at present eligible for carbon crediting (sometimes under several methodologies) and at least 65% of those have been implemented (Supplementary Table 1 ). This review was then appraised by 30 independent scholars (at least three per pathway; a complete review synthesis is given in the Supplementary Data ).

Consolidation of a broad body of scientific knowledge, with inherent variance, requires expert judgement. We used an expert elicitation process 13 , 14 , 15 with ten experts to place each proposed NbCS pathway into one of three readiness categories following their own assessment of the scientific literature, categorized by general sources of potential uncertainty: category 1, sufficient scientific basis to support a high-quality carbon accounting system or to support the development of such a system today; category 2, a >25% chance that focused research and reasonable funding would support development of high-quality carbon accounting (that is, move to category 1) within 5 years; or category 3, a <25% chance of development of high-quality carbon accounting within 5 years (for example, due to measurement challenges, unconstrained leakage, external factors which constrain viability).

If an expert ranked a pathway as category 2, they were also asked to rank general research needs to resolve: leakage/displacement (spillover to other areas), measuring, reporting and verification (the ability to quantify all salient stocks and fluxes), basic mechanisms of action (fundamental science), durability (ability to predict or compensate for uncertainty in timescale of effectiveness due to disturbances, climate change, human activity or other factors), geographic uncertainty (place-to-place variation), scaling potential (ability to estimate impact) and setting of a baseline (ability to estimate additionality over non-action; a counterfactual). To avoid biasing towards a particular a priori framework for evaluation of the scientific literature, reviewers could use their own framework for evaluating the NbCS literature about potential climate impact and so could choose to ignore or add relevant categorizations as well. Any pathway in category 1 would not need fundamental research for implementation; research gaps were considered too extensive for useful guidance on reducing uncertainty in category 3 pathways. Estimates of the global scale of likely potential impact (PgCO 2 e yr −1 ) and cobenefits were also collected from expert elicitors. See Methods and Supplementary Information for the survey instrument.

Four pathways with the highest current carbon market activity and high mitigation potential (tropical and temperate forest conservation and reforestation; Table 1 and Supplementary Data ), were consistently rated as high-confidence pathways in the expert elicitation survey. Other NbCS pathways, especially in the forestry sector, were rated relatively strongly by the experts for both confidence in scientific basis and scale of potential impact, with some spread across the experts (upper right quadrant, Fig. 1 ). Conversely, 13 pathways were consistently marked by experts as currently highly uncertain/low confidence (median score across experts: 2.5–3.0) and placed in category 3 (for example, cropland microbial amendments and coral reef restoration; Supplementary Tables 1 and 2 ). For the full review, including crediting protocols currently used, literature estimates of scale and details of sub-pathways, see Supplementary Data .

Pathways in the upper right quadrant have both high confidence in the scientific foundations and the largest potential scale of global impact; pathways in the lower left have the lowest confidence in our present scientific body of knowledge and an estimated smaller potential scale of impact. Designations of carbon credit eligibility under existing protocols and market activity at the present time are noted. Grassland enhanced mineral weathering (EMW) is not shown (mean category rating 2.9) as no scale of impact was estimated. See Supplementary Table 1 for specific pathway data. Bars represent 20th to 80th percentiles of individual estimates, if there was variability in estimates. A small amount of random noise was added to avoid overlap.

The experts assessed 26 pathways as having average confidence scores between 1.5 and 2.4, suggesting the potential for near-term resolution of uncertainties. This categorization arose from either consensus amongst experts on the uncertain potential (for example, boreal forest reforestation consistently rated category 2, with primary concerns about durability) or because experts disagreed, with some ranking category 1 and others category 3 (for example, pasture management). We note that where expert disagreement exists (seen as the spread of responses in Fig. 1 and Supplementary Table 1 ; also see Data availability for link to original data), this suggests caution against overconfidence in statements about these pathways. These results also suggest that confidence may be increased by targeted research on the identified sources of uncertainty (Supplementary Table 3 ).

Sources of uncertainty

Durability and baseline-setting were rated as high sources of uncertainty across all pathways ranked as category 2 by the experts (mean ratings of 3.6 and 3.4 out of 5, respectively; Supplementary Table 3 ). Understanding of mechanisms and geographic spread had the lowest uncertainty ratings (2.1 and 2.3, respectively), showing confidence in the basic science. Different subsets of pathways had different prioritizations, however, suggesting different research needs: forest-centric pathways were most uncertain in their durability and additionality (3.8 and 3.4, respectively), suggesting concerns about long-term climate and disturbance trajectories. Agricultural and grassland systems, however, had higher uncertainty in measurement methods and additionality (3.9 and 3.5 respectively). Although there were concerns about durability from some experts (for example, due to sea-level rise), some coastal blue carbon pathways such as mangrove restoration (mean category ranking: 1.7 (20th to 80th percentile 1.0–2.0)) have higher confidence than others (for example, seagrass restoration: mean category ranking 2.8, 20th to 80th percentile 2.6–3.0)), which are relatively poorly constrained in terms of net radiative forcing potential despite a potentially large carbon impact (seagrass median: 1.60 PgCO 2 e yr −1 ; see Supplementary Data for more scientific literature estimates).

Scale of impact

For those pathways with lower categorization by the expert elicitation (category 2 or 3) at the present time, scale of global impact is a potential heuristic for prioritizing further research. High variability, often two orders of magnitude, was evident in the mean estimated potential PgCO 2 e yr −1 impacts for the different pathways (Fig. 1 and Supplementary Table 2 ) and the review of the literature found even larger ranges produced by individual studies (Supplementary Data ). A probable cause of this wide range was different constraints on the estimated potential, with some studies focusing on potential maximum impact and others on more constrained realizable impacts. Only avoided loss of tropical forest and cropland biochar amendment were consistently estimated as having the likely potential to mitigate >2 PgCO 2 e yr −1 , although biochar was considered more uncertain by experts due to other factors germane to its overall viability as a climate solution, averaging a categorization of 2.2. The next four highest potential impact pathways, ranging from 1.6 to 1.7 PgCO 2 e yr −1 , spanned the spectrum from high readiness (temperate forest restoration) to moderate (cropland conversion from annual to perennial vegetation and grassland restoration) to low (seagrass restoration, with main uncertainties around scale of potential impact and durability).

There was high variability in the elicitors’ estimated potential scale of impact, even in pathways with strong support, such as tropical forest avoided loss (20th to 80th percentile confidence interval: 1–8 PgCO 2 e yr −1 ), again emphasizing the importance of consistent definitions and constraints on how NbCS are measured, evaluated and then used in broad-scale climate change mitigation planning and budgeting. Generally, as pathway readiness decreased (moving from category 1 to 3), the elicitor-estimated estimates of GHG mitigation potential decreased (Supplementary Fig. 1 ). Note that individual studies from the scientific literature may have higher or lower estimates (Supplementary Data ).

Expert elicitation meta-analyses suggest that 6–12 responses are sufficient for a robust and stable quantification of responses 15 . We tested that assumption via a Monte Carlo-based sensitivity assessment. Readiness categorizations by the ten experts were robust to a Monte Carlo simulation test, where further samples were randomly drawn from the observed distribution of responses: mean difference between the original and the boot-strapped data was 0.02 (s.d. = 0.05) with an absolute difference average of 0.06 (s.d. = 0.06). The maximum difference in readiness categorization means across all pathways was 0.20 (s.d. = 0.20) (Supplementary Table 2 ). The full dataset of responses is available online (see ʻData availabilityʼ).

These results highlight opportunities to accelerate implementation of NbCS in well-supported pathways and identify critical research needs in others (Fig. 1 ). We suggest focusing future efforts on resolving identified uncertainties for pathways at the intersection between moderate average readiness (for example, mean categorizations between ~1.5 and 2.0) and high potential impact (for example, median >0.5 PgCO 2 e yr −1 ; Supplementary Table 1 ): agroforestry, improved tropical and temperate forest management, tropical and boreal peatlands avoided loss and peatland restoration. Many, although not all, experts identified durability and baseline/additionality as key concerns to resolve in those systems; research explicitly targeted at those specific uncertainties (Supplementary Table 3 ) could rapidly improve confidence in those pathways.

We recommend a secondary research focus on the lower ranked (mean category 2.0 to 3.0) pathways with estimated potential impacts >1 PgCO 2 e yr −1 (Supplementary Fig. 2 ). For these pathways, explicit, quantitative incorporation into broad-scale GHG management plans will require further focus on systems-level carbon/GHG understandings to inspire confidence at all stages of action and/or identifying locations likely to support durable GHG mitigation, for example ref. 16 . Examples of this group include avoided loss and degradation of boreal forests (for example, fire, pests and pathogens and albedo 16 ) and effective mesopelagic fishery management, which some individual studies estimate would avoid future reductions of the currently sequestered 1.5–2.0 PgC yr −1 (refs. 17 , 18 ). These pathways may turn out to have higher or lower potential than the expert review suggests, on the basis of individual studies (Supplementary Data ) but strong support will require further, independent verification of that potential.

We note that category 3 rankings by expert elicitation do not necessarily imply non-viability but simply that much more research is needed to confidently incorporate actions into quantitative GHG mitigation plans. We found an unsurprising trend of lower readiness categorization with lower pathway familiarity (Supplementary Fig. 3 ). This correlation may result from two, non-exclusive potential causes: (1) lower elicitor expertise in some pathways (inevitable, although the panel was explicitly chosen for global perspectives, connections and diverse specialties) and (2) an actual lack of scientific evidence in the literature, which leads to that self-reported lack of familiarity, a common finding in the literature review (Supplementary Data ). Both explanations suggest a need to better consolidate, develop and disseminate the science in each pathway for global utility and recognition.

Our focus on GHG-related benefits in no way diminishes the substantial conservation, environmental and social cobenefits of these pathways (Supplementary Table 4 ), which often exceed their perceived climate benefits 1 , 19 , 20 , 21 . Where experts found climate impacts to remain highly uncertain but other NbS benefits are clear (for example, biodiversity and water quality; Supplementary Table 4 ), other incentives or financing mechanisms independent of carbon crediting should be pursued. While the goals here directly relate to using NbCS as a reliably quantifiable part of global climate action planning and thus strong GHG-related scientific foundations, non-climate NbS projects may provide climate benefits that are less well constrained (and thus less useful from a GHG budgeting standpoint) but also valuable. Potential trade-offs, if any, between ecosystem services and management actions, such as biodiversity and positive GHG outcomes, should be explored to ensure the best realization of desired goals 2 .

Finally, our focus in this study was on broad-scale NbCS potential in quantitative mitigation planning because of the principal and necessary role of NbCS in overall global warming targets. We recognize the range of project conditions that may increase, or decrease, the rigour of any pathway outside the global-scale focus here. We did not specifically evaluate the large and increasing number of crediting concepts (by pathway: Supplementary Data ), focusing rather on the underlying scientific body of knowledge within those pathways. Some broad pathways may have better defined sub-pathways within them, with a smaller potential scale of impact but potentially lower uncertainty (for example, macroalgae harvest cycling). Poorly enacted NbCS actions and/or crediting methodologies at project scales may result in loss of benefits even from high-ranking pathways 22 , 23 , 24 and attention to implementation should be paramount. Conversely, strong, careful project-scale methodologies may make lower readiness pathways beneficial for a given site.

Viable NbCS are vital to global climate change mitigation but NbCS pathways that lack strong scientific underpinnings threaten global accounting by potentially overestimating future climate benefits and eroding public trust in rigorous natural solutions. Both the review of the scientific literature and the expert elicitation survey identified high potential ready-to-implement pathways (for example, tropical reforestation), reinforcing present use of NbCS in planning.

However, uncertainty remains about the quantifiable GHG mitigation of some active and nascent NbCS pathways. On the basis of the expert elicitation survey and review of the scientific literature, we are concerned that large-scale implementation of less scientifically well-founded NbCS pathways in mitigation plans may undermine net GHG budget planning; those pathways require more study before they can be confidently promoted at broad scales and life-cycle analyses to integrate system-level emissions when calculating totals. The expert elicitation judgements suggest a precautionary approach to scaling lower confidence pathways until the scientific foundations are strengthened, especially for NbCS pathways with insufficient measurement and monitoring 10 , 24 , 25 or poorly understood or measured net GHG mitigation potentials 16 , 26 , 27 , 28 . While the need to implement more NbCS pathways for reducing GHG emissions and removing carbon from the atmosphere is urgent, advancing the implementation of poorly quantified pathways (in relation to their GHG mitigation efficacy) could give the false impression that they can balance ongoing, fossil emissions, thereby undermining overall support for more viable NbCS pathways. Explicitly targeting research to resolve these uncertainties in the baseline science could greatly bolster confidence in the less-established NbCS pathways, benefiting efforts to reduce GHG concentrations 29 .

The results of this study should inform both market-based mechanisms and non-market approaches to NbCS pathway management. Research and action that elucidates and advances pathways to ensure a solid scientific basis will provide confidence in the foundation for successfully implementing NbCS as a core component of global GHG management.

NbCS pathway selection

We synthesized scientific publications for nine biomes (boreal forests, coastal blue carbon, freshwater wetlands, grasslands, open ocean blue carbon, peatlands, shrublands, temperate forests and tropical forests) and three cultivation types (agroforestry, croplands and macroalgae aquaculture) (hereafter, systems) and the different pathways through which they may be able to remove carbon or reduce GHG emissions. Shrublands and grasslands were considered as independent ecosystems; nonetheless, we acknowledge that there is overlap in the numbers presented here because shrublands are often included with grasslands 5 , 30 , 31 , 32 , 33 .

The 12 systems were chosen because they have each been identified as having potential for emissions reductions or carbon removal at globally relevant scales. Within these systems, we identified 43 pathways which either have carbon credit protocols formally established or informally proposed for review (non-carbon associated credits were not evaluated). We obtained data on carbon crediting protocols from international, national and regional organizations and registries, such as Verra, American Carbon Registry, Climate Action Reserve, Gold Standard, Clean Development Mechanism, FAO and Nori. We also obtained data from the Voluntary Registry Offsets Database developed by the Berkeley Carbon Trading Project and Carbon Direct company 34 . While we found evidence of more Chinese carbon crediting protocols, we were not able to review these because of limited publicly available information. To maintain clarity and avoid misrepresentation, we used the language as written in each protocol. A full list of the organizations and registries for each system can be found in the Supplementary Data .

Literature searches and synthesis

We reviewed scientific literature and reviews (for example, IPCC special reports) to identify studies reporting data on carbon stocks, GHG dynamics and sequestration potential of each system. Peer-reviewed studies and meta-analyses were identified on Scopus, Web of Science and Google Scholar using simple queries combining the specific practice or pathway names or synonyms (for example, no-tillage, soil amendments, reduced stocking rates, improved forest management, avoided forest conversion and degradation, avoided mangrove conversion and degradation) and the following search terms: ‘carbon storage’, ‘carbon stocks’, ‘carbon sequestration’, ‘carbon sequestration potential’, ‘additional carbon storage’, ‘carbon dynamics’, ‘areal extent’ or ‘global’.

The full literature review was conducted between January and October 2021. We solicited an independent, external review of the syntheses (obtaining from at least three external reviewers per natural or working system; see p. 2 of the Supplementary Data ) as a second check against missing key papers or misinterpretation of data. The review was generally completed in March 2022. Data from additional relevant citations were added through October 2022 as they were discovered. For a complete list of all literature cited, see pp. 217–249 of the Supplementary Data .

From candidate papers, the papers were considered if their results/data could be applied to the following central questions:

How much carbon is stored (globally) at present in the system (total and on average per hectare) and what is the confidence?

At the global level, is the system a carbon source or sink at this time? What is the business-as-usual projection for its carbon dynamics?

Is it possible, through active management, to either increase net carbon sequestration in the system or prevent carbon emissions from that system? (Note that other GHG emissions and forcings were included here as well.)

What is the range of estimates for how much extra carbon could be sequestered globally?

How much confidence do we have in the present methods to detect any net increases in carbon sequestration in a system or net changes in areal extent of that?

From each paper, quantitative estimates for the above questions were extracted for each pathway, including any descriptive information/metadata necessary to understand the estimate. In addition, information on sample size, sampling scheme, geographic coverage, timeline of study, timeline of projections (if applicable) and specific study contexts (for example, wind-break agroforestry) were recorded.

We also tracked where the literature identified trade-offs between carbon sequestered or CO 2 emissions reduced and emissions of other GHG (for example, N 2 O or methane) for questions three and five above. For example, wetland restoration can result in increased CO 2 uptake from the atmosphere. However, it can also increase methane and N 2 O emissions to the atmosphere. Experts were asked to consider the uncertainty in assessing net GHG mitigation as they categorized the NbCS pathways.

Inclusion of each pathway in mitigation protocols and the specific carbon registries involved were also identified. These results are reported (grouped or individually as appropriate) in the Supplementary Data , organized by the central questions and including textual information for interpretation. The data and protocol summaries for each of the 12 systems were reviewed by at least three scientists each and accordingly revised.

These summaries were provided to the expert elicitation group as optional background information.

Unit conversions

Since this synthesis draws on literature from several sources that use different methods and units, all carbon measurements were standardized to the International System of Units (SI units). When referring to total stocks for each system, numbers are reported in SI units of elemental carbon (that is, PgC). When referring to mitigation potential, elemental carbon was converted to CO 2 by multiplying by 3.67. Differences in methodology, such as soil sampling depth, make it difficult to standardize across studies. Where applicable, the specific measurement used to develop each stock estimate is reported.

Expert elicitation process

To assess conclusions brought about by the initial review process described above, we conducted an expert elicitation survey to consolidate and add further, independent assessments to the original literature review. The expert elicitation survey design followed best practice recommendations 14 , with a focus on participant selection, explicitly defining uncertainty, minimizing cognitive and overconfidence biases and clarity of focus. Research on expert elicitation suggests that 6–12 responses are sufficient for a stable quantification of responses 15 . We identified >40 potential experts via a broad survey of leading academics, science-oriented NGO and government agency publications and products. These individuals have published on several NbCS pathways or could represent larger research efforts that spanned the NbCS under consideration. Careful attention was paid to the gender and sectoral breakdown of respondents to ensure equitable representation. Of the invitees, ten completed the full elicitation effort. Experts were offered compensation for their time.

Implementation of the expert elicitation process followed the IDEA protocol 15 . Briefly, after a short introductory interview, the survey was sent to the participants. Results were anonymized and standardized (methods below) and a meeting held with the entire group to discuss the initial results and calibrate understanding of questions. The purpose of this meeting was not to develop consensus on a singular answer but to discuss and ensure that all questions are being considered in the same way (for example, clarifying any potentially confusing language, discussing any questions that emerged as part of the process). The experts then revisited their initial rankings to provide final, anonymous rankings which were compiled in the same way. These final rankings are the results presented here and may be the same or different from the initial rankings, which were discarded.

Survey questions

The expert elicitation survey comprised five questions for each pathway. The data were collected via Google Forms and collated anonymously at the level of pathways, with each respondent contributing one datapoint for each pathway. The experts reported their familiarity (or the familiarity of the organization whose work they were representing) with the pathway and other cobenefits for the pathways.

The initial question ranked the NbCS pathway by category, from one to three.

Category 1 was defined as a pathway with sufficient scientific knowledge to support a high-quality carbon accounting system today (for example, meets the scientific criteria identified in the WWF-EDF-Oeko Institut and ICAO TAB) or to support the development of such a system today. The intended interpretation is that sufficient science is available for quantifying and verifying net GHG mitigation. Note that experts were not required to reference any given ‘high-quality’ crediting framework, which were provided only as examples. In other words, the evaluation was not intended to rank a given framework (for example, ref. 35 ) but rather expert confidence in the fundamental scientific understandings that underpin potential for carbon accounting overall. To this end, no categorization of uncertainty was required (reviewers could skip categorizations they felt were not necessary) and space was available to fill in new categories by individual reviewers (if they felt a category was missing or needed). Uncertainties at this category 1 level are deemed ‘acceptable’, for example, not precluding accounting now, although more research may further substantiate high-quality credits.

Category 2 pathways have a good chance (>25%) that with more research and within the next 5 years, the pathway could be developed into a high-quality pathway for carbon accounting and as a nature-based climate solution pathway. For these pathways, further understanding is needed for factors such as baseline processes, long-term stability, unconstrained fluxes, possible leakage or other before labelling as category 1 but the expert is confident that information can be developed, in 5 years or less, with more work. The >25% chance threshold and 5-year timeframe were determined a priori to reflect and identify pathways that experts identified as having the potential to meet the Paris Accord 2030 goal. Other thresholds (for example, longer timeframes) could have been chosen, which would impact the relative distribution of pathways in categories 2 and 3 (for example, a longer timeframe allowed could move some pathways from category 3 into category 2, for some reviewers). We emphasize that category 3 pathways do not necessarily mean non-valuable approaches but longer timeframes required for research than the one set here.

Category 3 responses denoted pathways that the expert thought had little chance (<25%) that with more research and within the next 5 years, this pathway could be developed into a suitable pathway for managing as a natural solutions pathway, either because present evidence already suggests GHG reduction is not likely to be viable, co-emissions or other biophysical feedbacks may offset those gains or because understanding of key factors is lacking and unlikely to be developed within the next 5 years. Notably, the last does not mean that the NbCS pathway is not valid or viable in the long-term, simply that physical and biological understandings are probably not established enough to enable scientific rigorous and valid NbCS activity in the near term.

The second question asked the experts to identify research gaps associated with those that they ranked as category 2 pathways to determine focal areas for further research. The experts were asked to rank concerns about durability (ability to predict or compensate for uncertainty in timescale of effectiveness due to disturbances, climate change, human activity or other factors), geographic uncertainty (place-to-place variation), leakage or displacement (spillover of activities to other areas), measuring, reporting and verification (MRV, referring to the ability to quantify all salient stocks and fluxes to fully assess climate impacts), basic mechanisms of action (fundamental science), scaling potential (ability to estimate potential growth) and setting of a baseline (ability to reasonably quantify additionality over non-action, a counterfactual). Respondents could also enter a different category if desired. For complete definitions of these categories, see the survey instrument ( Supplementary Information ). This question was not asked if the expert ranked the pathway as category 1, as those were deemed acceptable, or for category 3, respecting the substantial uncertainty in that rating. Note that responses were individual and so the same NbCS pathway could receive (for example) several individual category 1 rankings, which would indicate reasonable confidence from those experts, and several category 2 rankings from others, which would indicate that those reviewers have lingering concerns about the scientific basis, along with their rankings of the remaining key uncertainties in those pathways. These are important considerations, as they reflect the diversity of opinions and research priorities; individual responses are publicly available (anonymized: https://doi.org/10.5281/zenodo.7859146 ).

The third question involved quantification of the potential for moving from category 2 to 1 explicitly. Following ref. 14 , the respondents first reported the lowest plausible value for the potential likelihood of movement (representing the lower end of a 95% confidence interval), then the upper likelihood and then their best guess for the median/most likely probability. They were also asked for the odds that their chosen interval contained the true value, which was used to scale responses to standard 80% credible intervals and limit overconfidence bias 13 , 15 . This question was not asked if the expert ranked the pathway as category 3, respecting the substantial uncertainty in that rating.

The fourth question involved the scale of potential impact from the NbCS, given the range of uncertainties associated with effectiveness, area of applicability and other factors. The question followed the same pattern as the third, first asking about lowest, then highest, then best estimate for potential scale of impact (in PgCO 2 e yr −1 ). Experts were again asked to express their confidence in their own range, which was used to scale to a standard 80% credible interval. This estimate represents a consolidation of the best-available science by the reviewers. For a complete review including individual studies and their respective findings, see the Supplementary Data . This question was not asked if the expert ranked the pathway as category 3, respecting the substantial uncertainty in that rating.

Final results

After collection of the final survey responses, results were anonymized and compiled by pathway. For overall visualization and discussion purposes, responses were combined into a mean and 20th to 80th percentile range. The strength of the expert elicitation process lies in the collection of several independent assessments. Those different responses represent real differences in data interpretation and synthesis ascribed by experts. This can have meaningful impacts on decision-making by different individuals and organizations (for example, those that are more optimistic or pessimistic about any given pathway). Therefore, individual anonymous responses were retained by pathway to show the diversity of responses for any given pathway. The experts surveyed, despite their broad range of expertise, ranked themselves as less familiar with category 3 pathways than category 1 or 2 (linear regression, P  < 0.001, F  = 59.6 2, 394 ); this could be because of a lack of appropriate experts—although they represented all principal fields—or simply because the data are limited in those areas.

Sensitivity

To check for robustness against sample size variation, we conducted a Monte Carlo sensitivity analysis of the data on each pathway to generate responses of a further ten hypothetical experts. Briefly, the extra samples were randomly drawn from the observed category ranking mean and standard deviations for each individual pathway and appended to the original list; values <1 or >3 were truncated to those values. This analysis resulted in only minor differences in the mean categorization across all pathways: the mean difference between the original and the boot-strapped data was 0.02 (s.d. = 0.05) with an absolute difference average of 0.06 (s.d. = 0.06). The maximum difference in means across all pathways was 0.20 (s.d. = 0.20) (Supplementary Table 2 ). The results suggest that the response values are stable to additional responses.

All processing was done in R 36 , with packages including fmsb 37 and forcats 38 .

Data availability

Anonymized expert elicitation responses are available on Zenodo 39 : https://doi.org/10.5281/zenodo.7859146 .

Code availability

R code for analysis available on Zenodo 39 : https://doi.org/10.5281/zenodo.7859146 .

Novick, K. A. et al. Informing nature‐based climate solutions for the United States with the best‐available science. Glob. Change Biol. 28 , 3778–3794 (2022).

Article   Google Scholar  

Cohen-Shacham, E., Walters, G., Janzen, C. & Maginnis, S. (eds) Nature-based Solutions to Address Global Societal Challenges (IUCN, 2016).

IPCC Climate Change 2021: The Physical Science Basis (eds Masson-Delmotte, V. et al.) (Cambridge Univ. Press, 2021).

Seddon, N. et al. Understanding the value and limits of nature-based solutions to climate change and other global challenges. Philos. Trans. R. Soc. B 375 , 20190120 (2020).

Griscom, B. W. et al. Natural climate solutions. Proc. Natl Acad. Sci. USA 114 , 11645–11650 (2017).

Article   CAS   PubMed   PubMed Central   ADS   Google Scholar  

Blaufelder, C., Levy, C., Mannion, P. & Pinner, D. A. Blueprint for Scaling Voluntary Carbon Markets to Meet the Climate Challenge (McKinsey & Company, 2021).

Arcusa, S. & Sprenkle-Hyppolite, S. Snapshot of the carbon dioxide removal certification and standards ecosystem (2021–2022). Clim. Policy 22 , 1319–1332 (2022).

Seddon, N. et al. Global recognition of the importance of nature-based solutions to the impacts of climate change Glob. Sustain. 3 , pe15 (2020).

Anderegg, W. R. Gambling with the climate: how risky of a bet are natural climate solutions? AGU Adv. 2 , e2021AV000490 (2021).

Article   ADS   Google Scholar  

Gattuso, J. P. et al. Ocean solutions to address climate change and its effects on marine ecosystems. Front. Mar. Sci. 5 , p337 (2018).

Bach, L. T., Gill, S. J., Rickaby, R. E., Gore, S. & Renforth, P. CO 2 removal with enhanced weathering and ocean alkalinity enhancement: potential risks and co-benefits for marine pelagic ecosystems. Front. Clim. 1 , 7 (2019).

Doelman, J. C. et al. Afforestation for climate change mitigation: potentials, risks and trade‐offs. Glob. Change Biol. 26 , 1576–1591 (2019).

Speirs-Bridge, A. et al. Reducing overconfidence in the interval judgments of experts. Risk Anal. 30 , 512–523 (2010).

Article   PubMed   Google Scholar  

Morgan, M. G. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc. Natl Acad. Sci. USA 111 , 7176–7184 (2014).

Hemming, V., Burgman, M. A., Hanea, A. M., McBride, M. F. & Wintle, B. C. A practical guide to structured expert elicitation using the IDEA protocol. Methods Ecol. Evol. 9 , 169–180 (2018).

Anderegg, W. R. et al. Climate-driven risks to the climate mitigation potential of forests. Science 368 , eaaz7005 (2020).

Article   CAS   PubMed   Google Scholar  

Boyd, P. W., Claustre, H., Levy, M., Siegel, D. A. & Weber, T. Multi-faceted particle pumps drive carbon sequestration in the ocean. Nature 568 , 327–335 (2019).

Article   CAS   PubMed   ADS   Google Scholar  

Saba, G. K. et al. Toward a better understanding of fish-based contribution to ocean carbon flux. Limnol. Oceanogr. 66 , 1639–1664 (2021).

Article   CAS   ADS   Google Scholar  

Seddon, N., Turner, B., Berry, P., Chausson, A. & Girardin, C. A. Grounding nature-based climate solutions in sound biodiversity science. Nat. Clim. Change 9 , 84–87 (2019).

Soto-Navarro, C. et al. Mapping co-benefits for carbon storage and biodiversity to inform conservation policy and action. Philos. Trans. R. Soc. B 375 , 20190128 (2020).

Article   CAS   Google Scholar  

Schulte, I., Eggers, J., Nielsen, J. Ø. & Fuss, S. What influences the implementation of natural climate solutions? A systematic map and review of the evidence. Environ. Res. Lett. 17 , p013002 (2022).

West, T. A., Börner, J., Sills, E. O. & Kontoleon, A. Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon. Proc. Natl Acad. Sci. USA 117 , 24188–24194 (2020).

Di Sacco, A. et al. Ten golden rules for reforestation to optimize carbon sequestration, biodiversity recovery and livelihood benefits. Glob. Change Biol. 27 , 1328–1348 (2021).

López-Vallejo, M. in Towards an Emissions Trading System in Mexico: Rationale, Design and Connections with the Global Climate Agenda (ed. Lucatello, S.) 191–221 (Springer, 2022)

Oldfield, E. E. et al. Realizing the potential of agricultural soil carbon sequestration requires more effective accounting. Science 375 , 1222–1225 (2022).

Burkholz, C., Garcias-Bonet, N. & Duarte, C. M. Warming enhances carbon dioxide and methane fluxes from Red Sea seagrass ( Halophila stipulacea ) sediments. Biogeosciences 17 , 1717–1730 (2020).

Guenet, B. et al. Can N 2 O emissions offset the benefits from soil organic carbon storage? Glob. Change Biol. 27 , 237–256 (2021).

Rosentreter, J. A., Al‐Haj, A. N., Fulweiler, R. W. & Williamson, P. Methane and nitrous oxide emissions complicate coastal blue carbon assessments. Glob. Biogeochem. Cycles 35 , pe2020GB006858 (2021).

Schwartzman, S. et al. Environmental integrity of emissions reductions depends on scale and systemic changes, not sector of origin. Environ. Res. Lett. 16 , p091001 (2021).

Crop and Livestock Products Database (FAO, 2022); https://www.fao.org/faostat/en/#data/QCL

Fargione, J. E. et al. Natural climate solutions for the United States. Sci. Adv. 4 , eaat1869 (2018).

Article   PubMed   PubMed Central   ADS   Google Scholar  

Meyer, S. E. Is climate change mitigation the best use of desert shrublands? Nat. Resour. Environ. Issues 17 , 2 (2011).

Google Scholar  

Lorenz, K. & Lal, R. Carbon Sequestration in Agricultural Ecosystems (Springer Cham, 2018).

Haya, B., So, I. & Elias, M. The Voluntary Registry Offsets Database (Univ. California, 2021); https://gspp.berkeley.edu/faculty-and-impact/centers/cepp/projects/berkeley-carbon-trading-project/offsets-database

Core Carbon Principles; CCP Attributes; Assessment Framework for Programs; and Assessment Procedure (ICVCM, 2023); https://icvcm.org/the-core-carbon-principles/

R Core Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2022).

Nakazawa, M. fmsb: Functions for medical statistics book with some demographic data. R package version 0.7.4 https://CRAN.R-project.org/package=fmsb (2022).

Wickham, H. forcats: Tools for working with categorical variables (factors). R package version 0.5.2 https://CRAN.R-project.org/package=forcats (2022)

Buma, B. Nature-based climate solutions: expert elicitation data and analysis code. Zenodo https://doi.org/10.5281/zenodo.7859146 (2023).

Download references

Acknowledgements

This research was supported through gifts to the Environmental Defense Fund from the Bezos Earth Fund, King Philanthropies and Arcadia, a charitable fund of L. Rausing and P. Baldwin. We thank J. Rudek for help assembling the review and 30 experts who reviewed some or all of those data and protocol summaries (Supplementary Data ). S.M. was supported by a cooperative agreement between the National Science Foundation and Battelle that sponsors the National Ecological Observatory Network programme.

Author information

Present address: Department of Atmospheric and Oceanic Sciences, University of Wisconsin-Madison, Madison, WI, USA

Present address: AtmoFacts, Longmont, CO, USA

R. N. Lubowski

Present address: Lombard Odier Investment Managers, New York, NY, USA

Present address: Ecological Carbon Offset Partners LLC, dba EP Carbon, Minneapolis, MN, USA

L. A. Moore

Present address: , San Francisco, CA, USA

J. Paltseva

Present address: ART, Arlington, VA, USA

N. A. Randazzo

Present address: NASA/GSFC, Greenbelt, MD, USA

Present address: University of Maryland, College Park, MD, USA

N. Uludere Aragon

Present address: Numerical Terradynamic Simulation Group, University of Montana, Missoula, MT, USA

These authors contributed equally: B. Buma, D. R. Gordon.

Authors and Affiliations

Environmental Defense Fund, New York, NY, USA

B. Buma, D. R. Gordon, K. M. Kleisner, A. Bartuska, J. R. Collins, A. J. Eagle, R. Fujita, E. Holst, J. M. Lavallee, R. N. Lubowski, C. Melikov, L. A. Moore, E. E. Oldfield, J. Paltseva, A. M. Raffeld, N. A. Randazzo, C. Schneider, N. Uludere Aragon & S. P. Hamburg

Department of Integrative Biology, University of Colorado, Denver, CO, USA

Department of Biology, University of Florida, Gainesville, FL, USA

D. R. Gordon

Resources for the Future, Washington, DC, USA

A. Bartuska

International Arctic Research Center, University of Alaska, Fairbanks, AK, USA

Department of Ecology Evolution and Environmental Biology and the Climate School, Columbia University, New York, NY, USA

The Nature Conservancy, Arlington, VA, USA

Faculty of Environment, Science and Economy, University of Exeter, Exeter, UK

P. Friedlingstein

Laboratoire de Météorologie Dynamique/Institut Pierre-Simon Laplace, CNRS, Ecole Normale Supérieure/Université PSL, Sorbonne Université, Ecole Polytechnique, Palaiseau, France

National Ecological Observatory Network, Battelle, Boulder, CO, USA

Department of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA

O’Neill School of Public and Environmental Affairs, Indiana University, Bloomington, IN, USA

Department of Environmental Science and Policy, University of California, Davis, CA, USA

J. N. Sanchirico

Department of Marine Chemistry & Geochemistry, Woods Hole Oceanographic Institution, Woods Hole, MA, USA

J. R. Collins

You can also search for this author in PubMed   Google Scholar

Contributions

D.R.G. and B.B. conceived of and executed the study design. D.R.G., K.M.K., J.R.C., A.J.E., R.F., E.H., J.M.L., R.N.L., C.M., L.A.M., E.E.O., J.P., A.M.R., N.A.R., C.S. and N.U.A. coordinated and conducted the literature review. G.M. and B.B. primarily designed the survey. A. Bartuska, A. Bidlack, B.B., J.N.S., K.N., P.E., P.F., R.D. and S.M. contributed to the elicitation. B.B. conducted the analysis and coding. S.P.H. coordinated funding. B.B. and D.R.G. were primary writers; all authors were invited to contribute to the initial drafting.

Corresponding author

Correspondence to B. Buma .

Ethics declarations

Competing interests.

The authors declare no competing interests. In the interest of full transparency, we note that while B.B., D.R.G., K.M.K., A.B., J.R.C., A.J.E., R.F., E.H., J.M.L., R.N.L., C.M., L.A.M., E.E.O., J.P., A.M.R., N.A.R., C.S., N.U.A., S.P.H. and P.E. are employed by organizations that have taken positions on specific NbCS frameworks or carbon crediting pathways (not the focus of this work), none have financial or other competing interest in any of the pathways and all relied on independent science in their contributions to the work.

Peer review

Peer review information.

Nature Climate Change thanks Camila Donatti, Connor Nolan and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary Tables 1–4, Figs. 1–3 and survey instrument.

Supplementary Data

Literature review and list of reviewers.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Buma, B., Gordon, D.R., Kleisner, K.M. et al. Expert review of the science underlying nature-based climate solutions. Nat. Clim. Chang. (2024). https://doi.org/10.1038/s41558-024-01960-0

Download citation

Received : 24 April 2023

Accepted : 20 February 2024

Published : 21 March 2024

DOI : https://doi.org/10.1038/s41558-024-01960-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.