what are the limitations of literature review

About the LSE Impact Blog
Comments Policy
Popular Posts
Recent Posts
Subscribe to the Impact Blog
Write for us
LSE comment

Neal Haddaway

October 19th, 2020, 8 common problems with literature reviews and how to fix them.

3 comments | 315 shares

Estimated reading time: 5 minutes

Literature reviews are an integral part of the process and communication of scientific research. Whilst systematic reviews have become regarded as the highest standard of evidence synthesis, many literature reviews fall short of these standards and may end up presenting biased or incorrect conclusions. In this post, Neal Haddaway highlights 8 common problems with literature review methods, provides examples for each and provides practical solutions for ways to mitigate them.

Enjoying this blogpost? 📨 Sign up to our mailing list and receive all the latest LSE Impact Blog news direct to your inbox.

Researchers regularly review the literature – it’s an integral part of day-to-day research: finding relevant research, reading and digesting the main findings, summarising across papers, and making conclusions about the evidence base as a whole. However, there is a fundamental difference between brief, narrative approaches to summarising a selection of studies and attempting to reliably and comprehensively summarise an evidence base to support decision-making in policy and practice.

So-called ‘evidence-informed decision-making’ (EIDM) relies on rigorous systematic approaches to synthesising the evidence. Systematic review has become the highest standard of evidence synthesis and is well established in the pipeline from research to practice in the field of health . Systematic reviews must include a suite of specifically designed methods for the conduct and reporting of all synthesis activities (planning, searching, screening, appraising, extracting data, qualitative/quantitative/mixed methods synthesis, writing; e.g. see the Cochrane Handbook ). The method has been widely adapted into other fields, including environment (the Collaboration for Environmental Evidence ) and social policy (the Campbell Collaboration ).

what are the limitations of literature review

Despite the growing interest in systematic reviews, traditional approaches to reviewing the literature continue to persist in contemporary publications across disciplines. These reviews, some of which are incorrectly referred to as ‘systematic’ reviews, may be susceptible to bias and as a result, may end up providing incorrect conclusions. This is of particular concern when reviews address key policy- and practice- relevant questions, such as the ongoing COVID-19 pandemic or climate change.

These limitations with traditional literature review approaches could be improved relatively easily with a few key procedures; some of them not prohibitively costly in terms of skill, time or resources.

In our recent paper in Nature Ecology and Evolution , we highlight 8 common problems with traditional literature review methods, provide examples for each from the field of environmental management and ecology, and provide practical solutions for ways to mitigate them.

There is a lack of awareness and appreciation of the methods needed to ensure systematic reviews are as free from bias and as reliable as possible: demonstrated by recent, flawed, high-profile reviews. We call on review authors to conduct more rigorous reviews, on editors and peer-reviewers to gate-keep more strictly, and the community of methodologists to better support the broader research community. Only by working together can we build and maintain a strong system of rigorous, evidence-informed decision-making in conservation and environmental management.

Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below

Image credit: Jaeyoung Geoffrey Kang via unsplash

About the author

Neal Haddaway is a Senior Research Fellow at the Stockholm Environment Institute, a Humboldt Research Fellow at the Mercator Research Institute on Global Commons and Climate Change, and a Research Associate at the Africa Centre for Evidence. He researches evidence synthesis methodology and conducts systematic reviews and maps in the field of sustainability and environmental science. His main research interests focus on improving the transparency, efficiency and reliability of evidence synthesis as a methodology and supporting evidence synthesis in resource constrained contexts. He co-founded and coordinates the Evidence Synthesis Hackathon (www.eshackathon.org) and is the leader of the Collaboration for Environmental Evidence centre at SEI. @nealhaddaway

Why is mission creep a problem and not a legitimate response to an unexpected finding in the literature? Surely the crucial points are that the review’s scope is stated clearly and implemented rigorously, not when the scope was finalised.

Pingback: Quick, but not dirty – Can rapid evidence reviews reliably inform policy? | Impact of Social Sciences

#9. Most of them are terribly boring. Which is why I teach students how to make them engaging…and useful.

“But I’m not ready!” Common barriers to writing and how to overcome them

November 16th, 2020.

“Remember a condition of academic writing is that we expose ourselves to critique” – 15 steps to revising journal articles

January 18th, 2017.

A simple guide to ethical co-authorship

March 29th, 2021.

How common is academic plagiarism?

February 8th, 2024.

Visit our sister blog LSE Review of Books

Eight problems with literature reviews and how to fix them

Affiliations.

1 Mercator Research Institute on Climate Change and Global Commons, Berlin, Germany. [email protected].
2 Stockholm Environment Institute, Stockholm, Sweden. [email protected].
3 Africa Centre for Evidence, University of Johannesburg, Johannesburg, South Africa. [email protected].
4 College of Medicine and Health, Exeter University, Exeter, UK.
5 Department of Zoology, University of Cambridge, Cambridge, UK.
6 School of Biological Sciences, University of East Anglia, Norwich, UK.
7 Department of Biological Sciences, Royal Holloway University of London, Egham, UK.
8 Stockholm Environment Institute, Stockholm, Sweden.
9 Department of Zoology, University of Oxford, Oxford, UK.
10 Collaboration for Environmental Evidence, UK Centre, School of Natural Sciences, Bangor University, Bangor, UK.
11 Liljus ltd, London, UK.
12 Department of Forest Sciences, University of Helsinki, Helsinki, Finland.
13 Evidence Synthesis Lab, School of Natural and Environmental Sciences, University of Newcastle, Newcastle-upon-Tyne, UK.
PMID: 33046871
DOI: 10.1038/s41559-020-01295-x

Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to synthesizing evidence across studies; they rely on a suite of evidence-based methods aimed at maximizing rigour and minimizing susceptibility to bias. Despite the increasing popularity of systematic reviews in the environmental field, evidence synthesis methods continue to be poorly applied in practice, resulting in the publication of syntheses that are highly susceptible to bias. Recognizing the constraints that researchers can sometimes feel when attempting to plan, conduct and publish rigorous and comprehensive evidence syntheses, we aim here to identify major pitfalls in the conduct and reporting of systematic reviews, making use of recent examples from across the field. Adopting a 'critical friend' role in supporting would-be systematic reviews and avoiding individual responses to police use of the 'systematic review' label, we go on to identify methodological solutions to mitigate these pitfalls. We then highlight existing support available to avoid these issues and call on the entire community, including systematic review specialists, to work towards better evidence syntheses for better evidence and better decisions.

Environment
Research Design
Systematic Reviews as Topic*

University of Texas Libraries

Literature Reviews

What is a literature review?
Steps in the Literature Review Process
Define your research question
Determine inclusion and exclusion criteria
Choose databases and search
Review Results
Synthesize Results
Analyze Results
Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field.

A literature review should:

Provide a comprehensive and updated review of the literature;
Explain why this review has taken place;
Articulate a position or hypothesis;
Acknowledge and account for conflicting and corroborating points of view

From S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

Demonstrate how a study fills a gap in research
Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

Organizes or describes a topic
Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
It may be too focused. Some niche studies may miss the bigger picture.
It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

October 26, 2022 recording
Last Updated: Oct 26, 2022 2:49 PM
URL: https://guides.lib.utexas.edu/literaturereviews

Advantages and disadvantages of literature review

This comprehensive article explores some of the advantages and disadvantages of literature review in research. Reviewing relevant literature is a key area in research, and indeed, it is a research activity in itself. It helps researchers investigate a particular topic in detail. However, it has some limitations as well.

What is literature review?

In order to understand the advantages and disadvantages of literature review, it is important to understand what a literature review is and how it differs from other methods of research. According to Jones and Gratton (2009) a literature review essentially consists of critically reading, evaluating, and organising existing literature on a topic to assess the state of knowledge in the area. It is sometimes called critical review.

A literature review is a select analysis of existing research which is relevant to a researcher’s selected topic, showing how it relates to their investigation. It explains and justifies how their investigation may help answer some of the questions or gaps in the chosen area of study (University of Reading, 2022).

A literature review is a term used in the field of research to describe a systematic and methodical investigation of the relevant literature on a particular topic. In other words, it is an analysis of existing research on a topic in order to identify any relevant studies and draw conclusions about the topic.

A literature review is not the same as a bibliography or a database search. Rather than simply listing references to sources of information, a literature review involves critically evaluating and summarizing existing research on a topic. As such, it is a much more detailed and complex process than simply searching databases and websites, and it requires a lot of effort and skills.

Advantages of literature review

Information synthesis

A literature review is a very thorough and methodical exercise. It can be used to synthesize information and draw conclusions about a particular topic. Through a careful evaluation and critical summarization, researchers can draw a clear and comprehensive picture of the chosen topic.

Familiarity with the current knowledge

According to the University of Illinois (2022), literature reviews allow researchers to gain familiarity with the existing knowledge in their selected field, as well as the boundaries and limitations of that field.

Creation of new body of knowledge

One of the key advantages of literature review is that it creates new body of knowledge. Through careful evaluation and critical summarisation, researchers can create a new body of knowledge and enrich the field of study.

Answers to a range of questions

Literature reviews help researchers analyse the existing body of knowledge to determine the answers to a range of questions concerning a particular subject.

Disadvantages of literature review

Time consuming

As a literature review involves collecting and evaluating research and summarizing the findings, it requires a significant amount of time. To conduct a comprehensive review, researchers need to read many different articles and analyse a lot of data. This means that their review will take a long time to complete.

Lack of quality sources

Researchers are expected to use a wide variety of sources of information to present a comprehensive review. However, it may sometimes be challenging for them to identify the quality sources because of the availability of huge numbers in their chosen field. It may also happen because of the lack of past empirical work, particularly if the selected topic is an unpopular one.

Descriptive writing

One of the major disadvantages of literature review is that instead of critical appreciation, some researchers end up developing reviews that are mostly descriptive. Their reviews are often more like summaries of the work of other writers and lack in criticality. It is worth noting that they must go beyond describing the literature.

Key features of literature review

Clear organisation

A literature review is typically a very critical and thorough process. Universities usually recommend students a particular structure to develop their reviews. Like all other academic writings, a review starts with an introduction and ends with a conclusion. Between the beginning and the end, researchers present the main body of the review containing the critical discussion of sources.

No obvious bias

A key feature of a literature review is that it should be very unbiased and objective. However, it should be mentioned that researchers may sometimes be influenced by their own opinions of the world.

Proper citation

One of the key features of literature review is that it must be properly cited. Researchers should include all the sources that they have used for information. They must do citations and provide a reference list by the end in line with a recognized referencing system such as Harvard.

To conclude this article, it can be said that a literature review is a type of research that seeks to examine and summarise existing research on a particular topic. It is an essential part of a dissertation/thesis. However, it is not an easy thing to handle by an inexperienced person. It also requires a lot of time and patience.

Hope you like this ‘Advantages and disadvantages of literature review’. Please share this with others to support our research work.

Author: M Rahman

M Rahman writes extensively online and offline with an emphasis on business management, marketing, and tourism. He is a lecturer in Management and Marketing. He holds an MSc in Tourism & Hospitality from the University of Sunderland. Also, graduated from Leeds Metropolitan University with a BA in Business & Management Studies and completed a DTLLS (Diploma in Teaching in the Life-Long Learning Sector) from London South Bank University.

How to be a good team player, competitive advantage for tourist destinations, advantages and disadvantages of snowball sampling.

Eight common problems with science literature reviews and how to fix them

Research Fellow, Africa Centre for Evidence, University of Johannesburg

Disclosure statement

Neal Robert Haddaway works for the Stockholm Environment Institute and the Mercator Research Institute on Global Commons and Climate Change. He receives funding from the Alexander von Humboldt Foundation, Mistra, Formas, and Vinnova. He is also an honorary Research Associate at the Africa Centre for Evidence at the University of Johannesburg.

University of Johannesburg provides support as an endorsing partner of The Conversation AFRICA.

View all partners

A pile of books, with ribbons sticking out to denote bookmarks

Researchers regularly review the literature that’s generated by others in their field. This is an integral part of day-to-day research: finding relevant research, reading and digesting the main findings, summarising across papers, and making conclusions about the evidence base as a whole.

However, there is a fundamental difference between brief, narrative approaches to summarising a selection of studies and attempting to reliably, comprehensively summarise an evidence base to support decision-making in policy and practice.

So-called “evidence-informed decision-making” relies on rigorous systematic approaches to synthesising the evidence. Systematic review has become the highest standard of evidence synthesis. It is well established in the pipeline from research to practice in several fields including health , the environment and social policy . Rigorous systematic reviews are vital for decision-making because they help to provide the strongest evidence that a policy is likely to work (or not). They also help to avoid expensive or dangerous mistakes in the choice of policies.

But systematic review has not yet entirely replaced traditional methods of literature review. These traditional reviews may be susceptible to bias and so may end up providing incorrect conclusions. This is especially worrying when reviews address key policy and practice questions.

The good news is that the limitations of traditional literature review approaches could be improved relatively easily with a few key procedures. Some of these are not prohibitively costly in terms of skill, time or resources. That’s particularly important in African contexts, where resource constraints are a daily reality, but should not compromise the continent’s need for rigorous, systematic and transparent evidence to inform policy.

In our recent paper in Nature Ecology and Evolution , we highlighted eight common problems with traditional literature review methods. We gave examples for each problem, drawing from the field of environmental management and ecology. Finally, we outlined practical solutions.

These are the eight problems we identified in our paper .

First, traditional literature reviews can lack relevance. This is because limited stakeholder engagement can lead to a review that is of limited practical use to decision-makers.

Second, reviews that don’t publish their methods in an a priori (meaning that it is published before the review work begins) protocol may suffer from mission creep. In our paper we give the example of a 2019 review that initially stated it was looking at all population trends among insects. Instead, it ended up focusing only on studies that showed insect population declines. This could have been prevented by publishing and sticking to methods outlined in a protocol.

Third, a lack of transparency and replicability in the review methods may mean that the review cannot be replicated . Replicability is a central tenet of the scientific method.

Selection bias is another common problem. Here, the studies that are included in a literature review are not representative of the evidence base. A lack of comprehensiveness, stemming from an inappropriate search method, can also mean that reviews end up with the wrong evidence for the question at hand.

Traditional reviews may also exclude grey literature . This is defined as any document

produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers, i.e., where publishing is not the primary activity of the producing body.

It includes organisational reports and unpublished theses or other studies . Traditional reviews may also fail to test for evidence of publication bias; both these issues can result in incorrect or misleading conclusions. Another common error is to treat all evidence as equally valid. The reality is that some research studies are more valid than others. This needs to be accounted for in the synthesis.

Inappropriate synthesis is another common issue. This involves methods like vote-counting, which refers to tallying studies based on their statistical significance. Finally, a lack of consistency and error checking (as would happen when a reviewer works alone) can introduce errors and biases if a single reviewer makes decisions without consensus .

All of these common problems can be solved, though. Here’s how.

Stakeholders can be identified, mapped and contacted for feedback and inclusion without the need for extensive budgets. Best-practice guidelines for this process already exist .

Researchers can carefully design and publish an a priori protocol that outlines planned methods for searching, screening, data extraction, critical appraisal and synthesis in detail. Organisations like the Collaboration for Environmental Evidence have existing protocols from which people can draw.

Researchers also need to be explicit and use high-quality guidance and standards for review conduct and reporting . Several such standards already exist .

Another useful approach is to carefully design a search strategy with an info specialist; to trial the search strategy against a benchmark list; and to use multiple bibliographic databases, languages and sources of grey literature. Researchers should then publish their search methods in an a priori protocol for peer review.

Researchers should consider carefully planning and trialling a critical appraisal tool before starting the process in full, learning from existing robust critical appraisal tools . Critical appraisal is the carefully planned assessment of all possible risks of bias and possible confounders in a research study. Researchers should select their synthesis method carefully, based on the data analysed. Vote-counting should never be used instead of meta-analysis. Formal methods for narrative synthesis should be used to summarise and describe the evidence base.

Finally, at least two reviewers should screen a subset of the evidence base to ensure consistency and shared understanding of the methods before proceeding. Ideally, reviewers should conduct all decisions separately and then consolidate.

Collaboration

Collaboration is crucial to address the problems with traditional review processes. Authors need to conduct more rigorous reviews. Editors and peer reviewers need to gate-keep more strictly. The community of methodologists needs to better support the broader research community.

Working together, the academic and research community can build and maintain a strong system of rigorous, evidence-informed decision-making in conservation and environmental management – and, ultimately, in other disciplines.

Systematic reviews
Evidence based policy
Academic research

Events and Communications Coordinator

Assistant Editor - 1 year cadetship

Executive Dean, Faculty of Health

Lecturer/Senior Lecturer, Earth System Science (School of Science)

Sydney Horizon Educators (Identified)

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
Explore content
About the journal
Publish with us
Sign up for alerts
Perspective
Published: 12 October 2020

Eight problems with literature reviews and how to fix them

Neal R. Haddaway ORCID: orcid.org/0000-0003-3902-2234 1 , 2 , 3 ,
Alison Bethel 4 ,
Lynn V. Dicks 5 , 6 ,
Julia Koricheva ORCID: orcid.org/0000-0002-9033-0171 7 ,
Biljana Macura ORCID: orcid.org/0000-0002-4253-1390 2 ,
Gillian Petrokofsky 8 ,
Andrew S. Pullin 9 ,
Sini Savilaakso ORCID: orcid.org/0000-0002-8514-8105 10 , 11 &
Gavin B. Stewart ORCID: orcid.org/0000-0001-5684-1544 12

Nature Ecology & Evolution volume 4 , pages 1582–1589 ( 2020 ) Cite this article

12k Accesses

88 Citations

386 Altmetric

Metrics details

Conservation biology
Environmental impact

An Author Correction to this article was published on 19 October 2020

This article has been updated

Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to synthesizing evidence across studies; they rely on a suite of evidence-based methods aimed at maximizing rigour and minimizing susceptibility to bias. Despite the increasing popularity of systematic reviews in the environmental field, evidence synthesis methods continue to be poorly applied in practice, resulting in the publication of syntheses that are highly susceptible to bias. Recognizing the constraints that researchers can sometimes feel when attempting to plan, conduct and publish rigorous and comprehensive evidence syntheses, we aim here to identify major pitfalls in the conduct and reporting of systematic reviews, making use of recent examples from across the field. Adopting a ‘critical friend’ role in supporting would-be systematic reviews and avoiding individual responses to police use of the ‘systematic review’ label, we go on to identify methodological solutions to mitigate these pitfalls. We then highlight existing support available to avoid these issues and call on the entire community, including systematic review specialists, to work towards better evidence syntheses for better evidence and better decisions.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

Purchase on Springer Link
Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health

Insights from a cross-sector review on how to conceptualise the quality of use of research evidence

The past, present and future of Registered Reports

Change history, 19 october 2020.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Grant, M. J. & Booth, A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr. J. 26 , 91–108 (2009).

PubMed Google Scholar

Haddaway, N. R. & Macura, B. The role of reporting standards in producing robust literature reviews. Nat. Clim. Change 8 , 444–447 (2018).

Google Scholar

Pullin, A. S. & Knight, T. M. Science informing policy–a health warning for the environment. Environ. Evid. 1 , 15 (2012).

Haddaway, N., Woodcock, P., Macura, B. & Collins, A. Making literature reviews more reliable through application of lessons from systematic reviews. Conserv. Biol. 29 , 1596–1605 (2015).

CAS PubMed Google Scholar

Pullin, A., Frampton, G., Livoreil, B. & Petrokofsky, G. Guidelines and Standards for Evidence Synthesis in Environmental Management (Collaboration for Environmental Evidence, 2018).

White, H. The twenty-first century experimenting society: the four waves of the evidence revolution. Palgrave Commun. 5 , 47 (2019).

O’Leary, B. C. et al. The reliability of evidence review methodology in environmental science and conservation. Environ. Sci. Policy 64 , 75–82 (2016).

Woodcock, P., Pullin, A. S. & Kaiser, M. J. Evaluating and improving the reliability of evidence syntheses in conservation and environmental science: a methodology. Biol. Conserv. 176 , 54–62 (2014).

Campbell Systematic Reviews: Policies and Guidelines (Campbell Collaboration, 2014).

Higgins, J. P. et al. Cochrane Handbook for Systematic Reviews of Interventions (John Wiley & Sons, 2019).

Shea, B. J. et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358 , j4008 (2017).

PubMed PubMed Central Google Scholar

Haddaway, N. R., Land, M. & Macura, B. “A little learning is a dangerous thing”: a call for better understanding of the term ‘systematic review’. Environ. Int. 99 , 356–360 (2017).

Freeman, R. E. Strategic Management: A Stakeholder Approach (Cambridge Univ. Press, 2010).

Haddaway, N. R. et al. A framework for stakeholder engagement during systematic reviews and maps in environmental management. Environ. Evid. 6 , 11 (2017).

Land, M., Macura, B., Bernes, C. & Johansson, S. A five-step approach for stakeholder engagement in prioritisation and planning of environmental evidence syntheses. Environ. Evid. 6 , 25 (2017).

Oliver, S. & Dickson, K. Policy-relevant systematic reviews to strengthen health systems: models and mechanisms to support their production. Evid. Policy 12 , 235–259 (2016).

Savilaakso, S. et al. Systematic review of effects on biodiversity from oil palm production. Environ. Evid. 3 , 4 (2014).

Savilaakso, S., Laumonier, Y., Guariguata, M. R. & Nasi, R. Does production of oil palm, soybean, or jatropha change biodiversity and ecosystem functions in tropical forests. Environ. Evid. 2 , 17 (2013).

Haddaway, N. R. & Crowe, S. Experiences and lessons in stakeholder engagement in environmental evidence synthesis: a truly special series. Environ. Evid. 7 , 11 (2018).

Sánchez-Bayo, F. & Wyckhuys, K. A. Worldwide decline of the entomofauna: a review of its drivers. Biol. Conserv. 232 , 8–27 (2019).

Agarwala, M. & Ginsberg, J. R. Untangling outcomes of de jure and de facto community-based management of natural resources. Conserv. Biol. 31 , 1232–1246 (2017).

Gurevitch, J., Curtis, P. S. & Jones, M. H. Meta-analysis in ecology. Adv. Ecol. Res. 32 , 199–247 (2001).

CAS Google Scholar

Haddaway, N. R., Macura, B., Whaley, P. & Pullin, A. S. ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ. Evid. 7 , 7 (2018).

Lwasa, S. et al. A meta-analysis of urban and peri-urban agriculture and forestry in mediating climate change. Curr. Opin. Environ. Sustain. 13 , 68–73 (2015).

Pacifici, M. et al. Species’ traits influenced their response to recent climate change. Nat. Clim. Change 7 , 205–208 (2017).

Owen-Smith, N. Ramifying effects of the risk of predation on African multi-predator, multi-prey large-mammal assemblages and the conservation implications. Biol. Conserv. 232 , 51–58 (2019).

Prugh, L. R. et al. Designing studies of predation risk for improved inference in carnivore-ungulate systems. Biol. Conserv. 232 , 194–207 (2019).

Li, Y. et al. Effects of biochar application in forest ecosystems on soil properties and greenhouse gas emissions: a review. J. Soil Sediment. 18 , 546–563 (2018).

Moher, D., Liberati, A., Tetzlaff, J. & Altman, D. G., The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6 , e1000097 (2009).

Bernes, C. et al. What is the influence of a reduction of planktivorous and benthivorous fish on water quality in temperate eutrophic lakes? A systematic review. Environ. Evid. 4 , 7 (2015).

McDonagh, M., Peterson, K., Raina, P., Chang, S. & Shekelle, P. Avoiding bias in selecting studies. Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet] (Agency for Healthcare Research and Quality, 2013).

Burivalova, Z., Hua, F., Koh, L. P., Garcia, C. & Putz, F. A critical comparison of conventional, certified, and community management of tropical forests for timber in terms of environmental, economic, and social variables. Conserv. Lett. 10 , 4–14 (2017).

Min-Venditti, A. A., Moore, G. W. & Fleischman, F. What policies improve forest cover? A systematic review of research from Mesoamerica. Glob. Environ. Change 47 , 21–27 (2017).

Bramer, W. M., Giustini, D. & Kramer, B. M. R. Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Syst. Rev. 5 , 39 (2016).

Bramer, W. M., Giustini, D., Kramer, B. M. R. & Anderson, P. F. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Syst. Rev. 2 , 115 (2013).

Gusenbauer, M. & Haddaway, N. R. Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods 11 , 181–217 (2020).

Livoreil, B. et al. Systematic searching for environmental evidence using multiple tools and sources. Environ. Evid. 6 , 23 (2017).

Mlinarić, A., Horvat, M. & Šupak Smolčić, V. Dealing with the positive publication bias: why you should really publish your negative results. Biochem. Med. 27 , 447–452 (2017).

Lin, L. & Chu, H. Quantifying publication bias in meta‐analysis. Biometrics 74 , 785–794 (2018).

Haddaway, N. R. & Bayliss, H. R. Shades of grey: two forms of grey literature important for reviews in conservation. Biol. Conserv. 191 , 827–829 (2015).

Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36 , 1–48 (2010).

Bilotta, G. S., Milner, A. M. & Boyd, I. On the use of systematic reviews to inform environmental policies. Environ. Sci. Policy 42 , 67–77 (2014).

Englund, G., Sarnelle, O. & Cooper, S. D. The importance of data‐selection criteria: meta‐analyses of stream predation experiments. Ecology 80 , 1132–1141 (1999).

Burivalova, Z., Şekercioğlu, Ç. H. & Koh, L. P. Thresholds of logging intensity to maintain tropical forest biodiversity. Curr. Biol. 24 , 1893–1898 (2014).

Bicknell, J. E., Struebig, M. J., Edwards, D. P. & Davies, Z. G. Improved timber harvest techniques maintain biodiversity in tropical forests. Curr. Biol. 24 , R1119–R1120 (2014).

Damette, O. & Delacote, P. Unsustainable timber harvesting, deforestation and the role of certification. Ecol. Econ. 70 , 1211–1219 (2011).

Blomley, T. et al. Seeing the wood for the trees: an assessment of the impact of participatory forest management on forest condition in Tanzania. Oryx 42 , 380–391 (2008).

Haddaway, N. R. et al. How does tillage intensity affect soil organic carbon? A systematic review. Environ. Evid. 6 , 30 (2017).

Higgins, J. P. et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343 , d5928 (2011).

Stewart, G. Meta-analysis in applied ecology. Biol. Lett. 6 , 78–81 (2010).

Koricheva, J. & Gurevitch, J. Uses and misuses of meta‐analysis in plant ecology. J. Ecol. 102 , 828–844 (2014).

Vetter, D., Ruecker, G. & Storch, I. Meta‐analysis: a need for well‐defined usage in ecology and conservation biology. Ecosphere 4 , 1–24 (2013).

Stewart, G. B. & Schmid, C. H. Lessons from meta-analysis in ecology and evolution: the need for trans-disciplinary evidence synthesis methodologies. Res. Synth. Methods 6 , 109–110 (2015).

Macura, B. et al. Systematic reviews of qualitative evidence for environmental policy and management: an overview of different methodological options. Environ. Evid. 8 , 24 (2019).

Koricheva, J. & Gurevitch, J. in Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J. et al.) Ch. 1 (Princeton Scholarship Online, 2013).

Britt, M., Haworth, S. E., Johnson, J. B., Martchenko, D. & Shafer, A. B. The importance of non-academic coauthors in bridging the conservation genetics gap. Biol. Conserv. 218 , 118–123 (2018).

Graham, L., Gaulton, R., Gerard, F. & Staley, J. T. The influence of hedgerow structural condition on wildlife habitat provision in farmed landscapes. Biol. Conserv. 220 , 122–131 (2018).

Delaquis, E., de Haan, S. & Wyckhuys, K. A. On-farm diversity offsets environmental pressures in tropical agro-ecosystems: a synthetic review for cassava-based systems. Agric. Ecosyst. Environ. 251 , 226–235 (2018).

Popay, J. et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: A Product from the ESRC Methods Programme Version 1 (Lancaster Univ., 2006).

Pullin, A. S. et al. Human well-being impacts of terrestrial protected areas. Environ. Evid. 2 , 19 (2013).

Waffenschmidt, S., Knelangen, M., Sieben, W., Bühn, S. & Pieper, D. Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. BMC Med. Res. Methodol. 19 , 132 (2019).

Rallo, A. & García-Arberas, L. Differences in abiotic water conditions between fluvial reaches and crayfish fauna in some northern rivers of the Iberian Peninsula. Aquat. Living Resour. 15 , 119–128 (2002).

Glasziou, P. & Chalmers, I. Research waste is still a scandal—an essay by Paul Glasziou and Iain Chalmers. BMJ 363 , k4645 (2018).

Haddaway, N. R. Open Synthesis: on the need for evidence synthesis to embrace Open Science. Environ. Evid. 7 , 26 (2018).

Download references

Acknowledgements

We thank C. Shortall from Rothamstead Research for useful discussions on the topic.

Author information

Authors and affiliations.

Mercator Research Institute on Climate Change and Global Commons, Berlin, Germany

Neal R. Haddaway

Stockholm Environment Institute, Stockholm, Sweden

Neal R. Haddaway & Biljana Macura

Africa Centre for Evidence, University of Johannesburg, Johannesburg, South Africa

College of Medicine and Health, Exeter University, Exeter, UK

Alison Bethel

Department of Zoology, University of Cambridge, Cambridge, UK

Lynn V. Dicks

School of Biological Sciences, University of East Anglia, Norwich, UK

Department of Biological Sciences, Royal Holloway University of London, Egham, UK

Julia Koricheva

Department of Zoology, University of Oxford, Oxford, UK

Gillian Petrokofsky

Collaboration for Environmental Evidence, UK Centre, School of Natural Sciences, Bangor University, Bangor, UK

Andrew S. Pullin

Liljus ltd, London, UK

Sini Savilaakso

Department of Forest Sciences, University of Helsinki, Helsinki, Finland

Evidence Synthesis Lab, School of Natural and Environmental Sciences, University of Newcastle, Newcastle-upon-Tyne, UK

Gavin B. Stewart

You can also search for this author in PubMed Google Scholar

Contributions

N.R.H. developed the manuscript idea and a first draft. All authors contributed to examples and edited the text. All authors have read and approve of the final submission.

Corresponding author

Correspondence to Neal R. Haddaway .

Ethics declarations

Competing interests.

S.S. is a co-founder of Liljus ltd, a firm that provides research services in sustainable finance as well as forest conservation and management. The other authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary table.

Examples of literature reviews and common problems identified.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Haddaway, N.R., Bethel, A., Dicks, L.V. et al. Eight problems with literature reviews and how to fix them. Nat Ecol Evol 4 , 1582–1589 (2020). https://doi.org/10.1038/s41559-020-01295-x

Download citation

Received : 24 March 2020

Accepted : 31 July 2020

Published : 12 October 2020

Issue Date : December 2020

DOI : https://doi.org/10.1038/s41559-020-01295-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Facing the storm: developing corporate adaptation and resilience action plans amid climate uncertainty.

Katharina Hennes
David Bendig
Andreas Löschel

npj Climate Action (2024)

A review of the necessity of a multi-layer land-use planning

Hashem Dadashpoor
Leyla Ghasempour

Landscape and Ecological Engineering (2024)

Synthesizing the relationships between environmental DNA concentration and freshwater macrophyte abundance: a systematic review and meta-analysis

Toshiaki S. Jo

Hydrobiologia (2024)

A Systematic Review of the Effects of Multi-purpose Forest Management Practices on the Breeding Success of Forest Birds

João M. Cordeiro Pereira
Grzegorz Mikusiński
Ilse Storch

Current Forestry Reports (2024)

Parasitism in viviparous vertebrates: an overview

Juan J. Palacios-Marquez
Palestina Guevara-Fiore

Parasitology Research (2024)

Quick links

Explore articles by subject
Guide to authors
Editorial policies

Sign up for the Nature Briefing: Anthropocene newsletter — what matters in anthropocene research, free to your inbox weekly.

Limitations of the literature – A guide to a sticky problem

One of the more difficult tasks set for a Masters and especially PhD student is to identify limitations in the literature under review. Postgraduate students are required not only to “tell” (repeat what an author/s say) but to critically engage the literature being reviewed. This work typically appears in a separate chapter in the thesis or dissertation under the title Literature Review. To help you, here are 10 typical examples often used to identify limitations in a particular body of literature—with a caveat—using education examples.

1. The sample size is too small and therefore not representative of the phenomenon being studied This is fine but be aware of the fact that a single case study (e.g. the leadership practices of a non-traditional principal) is completely acceptable in a qualitative study. Here depth makes up for spread (representivity) for the purpose of the study is different e.g. an ethnographic case study of one principal.

2. The context is limiting and therefore not representative of all This too is acceptable e.g. a study of subject competency levels of 10 science teachers in rural Limpopo province. However, always specific WHAT it is about the context that is limiting e.g. nothing about science teachers in urban areas. A common mistake is to say ‘that study was done in America’ but that is not helpful in itself. What is it about the American context that might not be applicable in the Southern African context?

3. The study is dated as in out-of-date e.g. 1985 This is important as a criticism except in the case of historical studies or in reference to a classical piece of work that set the standard for understanding a particular problem. For example, Michael Apple’s 1979 Ideology and Curriculum might be dated but it is the landmark criticism in the politics of curriculum and often deserves referencing as a launching pad for more recent studies.

4. The period of observation was too short This is valuable especially in the qualitative study of classrooms where the researcher visits a history class of 10 teachers once and then jumps to make major findings and conclusions from a 40-minute lesson per teacher. This is a common problem with qualitative research in education in South Africa—the lack of extended engagement in the field.

5. The theoretical framework or theory is limiting A researcher might be using a particular theory to explain events e.g. a behaviorist account of learner discipline where you believe that a constructivist account offers more insight into why learners misbehave in classrooms. The onus is still on you to explain why the rival theory is ‘better’ than the one used.

6. The methodological approach is inappropriate for the question posed A study uses a self-reporting questionnaire completed by teachers to determine their competency levels in mathematics teaching. You could argue that direct observation of actual teaching is a much more direct measure of teaching competency in mathematics since teachers might overestimate their own levels of competency in the subject (by the way, there is research to back you up with such a claim)

7. The research question or instrument(s) is biased A study that asks ‘why teachers are incompetent’ or ‘the students are lazy’ already assumes the fact ahead of the inquiry itself. The research question can and should be posed in an open-ended manner to allow for more than one outcome. In the controversial SU study on Coloured women’s cognitive abilities and health styles it was found that the measuring instrument used was found in other studies to be flawed. And the study of code-switching in a Grade 6 language classroom would clearly show bias if the researcher was competent in English alone—unless, of course, this deficiency is remedied in the study design.

8. The study does not differentiate between subjects or contexts A report might give the results of a study for academic performance in Grade 12 economics in the National Senior Certificate, which is fine, but does not differentiate between the results of the former white schools and those of black schools thereby concealing variable performance and, for the sake of reform, where exactly added support might be needed.

9. The literature reviewed does not cover the subject at all or from a particular perspective The conclusion that ‘there is no research on topic X’ is a common one among students and often wrong. The fact that you have not yet found literature on topic X does not mean that such references do not exist. However, there are instances in which the case can and should be made that topic X appears to be understudied given the literature reviewed. Hint: always check with your supervisor and other experts whether the ‘not yet researched’ claim is justified. More plausible are studies which critique a dominant perspective on a subject and offer a new lens on the same issue (see point #5) above.

10. The study is based largely on opinion and uses primarily secondary sources This is a very valid criticism of many articles posing as research studies where the evidence is often anecdotal (not ‘thick descriptions’ as anthropologists like to point out) and drawing on other opinion pieces rather than solid research on the topic.

REMEMBER: THESE NOTES ARE NOT ABOUT HOW TO IDENTIFY LIMITATIONS IN LITERATURE REVIEWED FOR YOUR STUDY. IT IS NOT ABOUT LISTING THE LIMITATIONS OF YOUR STUDY IN THE COURSE OF WRITING UP A RESEARCH PROPOSAL OR THE THESIS ITSELF, EVEN THOUGH THIS COULD HELP YOU DEFEND SELF-IDENTIFIED LIMITATIONS OF YOUR RESEARCH.

The upsides to a pandemic: from jobs to teaching, it’s not all bad news

Let’s face facts, the 2020 school year is lost. so what to do, you may also like, corrupted: a study of chronic dysfunction in south..., learning under lockdown – 100 student voices, ten elements of a good literature review, ten rules for turning a dissertation into a....

This was very helpful to me.

Truly helpful. So much valuable information on this site that’s worth spreading across all social media platforms.

First group of successful Future Professors celebrated

2024 – chin yidan visiting global fellow, harvard…, power to truth with jj tabane, 2023 – elected fellow of the american academy…, 2023 – south african education research association honours, 2023 – lifetime achievement award, national science and…, limitations of the literature – a guide to…, ten rules for turning a dissertation into a…, the future of leadership book, corrupted: a study of chronic dysfunction in south…, what happens to your rights in a pandemic, trading places, a different take, my south africa, 8 myths about teachers: what parents often get…, the future of uwc and uct’s leadership hangs…, if time-wasting were in the curriculum, schools would…, profit and greed plague private education, but don’t….

Books published in 2022 & 2023
Recent Research
Presentations
Speaking Requests

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process.

What is the purpose of literature review?
a. Habitat Loss and Species Extinction:
b. Range Shifts and Phenological Changes:
c. Ocean Acidification and Coral Reefs:
d. Adaptive Strategies and Conservation Efforts:
How to write a good literature review
Choose a Topic and Define the Research Question:
Decide on the Scope of Your Review:
Select Databases for Searches:
Conduct Searches and Keep Track:
Review the Literature:
Organize and Write Your Literature Review:
Frequently asked questions

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2

Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge.
Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field.
Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research.
Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered.
Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research.
Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature.

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic.

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies:

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements.

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources.

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems.

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning.

How to write a good literature review

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements.

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review.

Opening Statement: Begin with a general statement about the broader topic and its significance in the field.
Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address.
Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research.
Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic.
Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review.

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria.

Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement.
Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic.
Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature.
Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps.
Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review.

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research.

Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective.
Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field.
Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice.
Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field.
Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community.

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1

Choose a Topic and Define the Research Question:

Select a topic that is relevant to your field of study.
Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore?

Decide on the Scope of Your Review:

Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview?
Consider the geographical scope. Is your review global, or are you focusing on a specific region?
Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude?

Select Databases for Searches:

Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar.
Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic.

Conduct Searches and Keep Track:

Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques.
Record and document your search strategy for transparency and replicability.
Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references.

Review the Literature:

Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies.
Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research.
Summarize key findings and arguments from each source. Compare and contrast different perspectives.
Identify areas where there is a consensus in the literature and where there are conflicting opinions.
Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research?

Organize and Write Your Literature Review:

Literature review outline should be based on themes, chronological order, or methodological approaches.
Write a clear and coherent narrative that synthesizes the information gathered.
Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.).
Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research.

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses.

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic.

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods.

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers. Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved. Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic. Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings. Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject. It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review: Introduction: Provide an overview of the topic. Define the scope and purpose of the literature review. State the research question or objective. Body: Organize the literature by themes, concepts, or chronology. Critically analyze and evaluate each source. Discuss the strengths and weaknesses of the studies. Highlight any methodological limitations or biases. Identify patterns, connections, or contradictions in the existing research. Conclusion: Summarize the key points discussed in the literature review. Highlight the research gap. Address the research question or objective stated in the introduction. Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows:

References

Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234.
Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis.
Cantero, C. (2019). How to write a literature review.  San José State University Writing Center .

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples , measuring academic success: definition & strategies for excellence, what is academic writing: tips for students, why traditional editorial process needs an upgrade.

Library Homepage

Literature Review: The What, Why and How-to Guide: Literature Reviews?

Literature Reviews?
Strategies to Finding Sources
Keeping up with Research!
Evaluating Sources & Literature Reviews
Organizing for Writing
Writing Literature Review
Other Academic Writings

What is a Literature Review?

So, what is a literature review .

"A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available or a set of summaries." - Quote from Taylor, D. (n.d)."The Literature Review: A Few Tips on Conducting it".

Citation: "The Literature Review: A Few Tips on Conducting it"

What kinds of literature reviews are written?

Each field has a particular way to do reviews for academic research literature. In the social sciences and humanities the most common are:

Narrative Reviews: The purpose of this type of review is to describe the current state of the research on a specific research topic and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weaknesses, and gaps are identified. The review ends with a conclusion section that summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.
Book review essays/ Historiographical review essays : A type of literature review typical in History and related fields, e.g., Latin American studies. For example, the Latin American Research Review explains that the purpose of this type of review is to “(1) to familiarize readers with the subject, approach, arguments, and conclusions found in a group of books whose common focus is a historical period; a country or region within Latin America; or a practice, development, or issue of interest to specialists and others; (2) to locate these books within current scholarship, critical methodologies, and approaches; and (3) to probe the relation of these new books to previous work on the subject, especially canonical texts. Unlike individual book reviews, the cluster reviews found in LARR seek to address the state of the field or discipline and not solely the works at issue.” - LARR

What are the Goals of Creating a Literature Review?

To develop a theory or evaluate an existing theory
To summarize the historical or existing state of a research topic
Identify a problem in a field of research
Baumeister, R.F. & Leary, M.R. (1997). "Writing narrative literature reviews," Review of General Psychology , 1(3), 311-320.

When do you need to write a Literature Review?

When writing a prospectus or a thesis/dissertation
When writing a research paper
When writing a grant proposal

In all these cases you need to dedicate a chapter in these works to showcase what has been written about your research topic and to point out how your own research will shed new light into a body of scholarship.

Where I can find examples of Literature Reviews?

Note: In the humanities, even if they don't use the term "literature review", they may have a dedicated chapter that reviewed the "critical bibliography" or they incorporated that review in the introduction or first chapter of the dissertation, book, or article.

UCSB electronic theses and dissertations In partnership with the Graduate Division, the UC Santa Barbara Library is making available theses and dissertations produced by UCSB students. Currently included in ADRL are theses and dissertations that were originally filed electronically, starting in 2011. In future phases of ADRL, all theses and dissertations created by UCSB students may be digitized and made available.

Where to Find Standalone Literature Reviews

Literature reviews are also written as standalone articles as a way to survey a particular research topic in-depth. This type of literature review looks at a topic from a historical perspective to see how the understanding of the topic has changed over time.

Find e-Journals for Standalone Literature Reviews The best way to get familiar with and to learn how to write literature reviews is by reading them. You can use our Journal Search option to find journals that specialize in publishing literature reviews from major disciplines like anthropology, sociology, etc. Usually these titles are called, "Annual Review of [discipline name] OR [Discipline name] Review. This option works best if you know the title of the publication you are looking for. Below are some examples of these journals! more... less... Journal Search can be found by hovering over the link for Research on the library website.

Social Sciences

Annual Review of Anthropology
Annual Review of Political Science
Annual Review of Sociology
Ethnic Studies Review

Hard science and health sciences:

Annual Review of Biomedical Data Science
Annual Review of Materials Science
Systematic Review From journal site: "The journal Systematic Reviews encompasses all aspects of the design, conduct, and reporting of systematic reviews" in the health sciences.
<< Previous: Overview
Next: Strategies to Finding Sources >>
Last Updated: Mar 5, 2024 11:44 AM
URL: https://guides.library.ucsb.edu/litreview

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Cover of Handbook of eHealth Evaluation: An Evidence-based Approach

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

formulating the research question(s) and objective(s),
searching the extant literature,
screening for inclusion,
assessing the quality of primary studies,
extracting data, and
analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
Analyzing data using quantitative or qualitative methods.
Presenting results in summary of findings tables.
Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed independently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guidelines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel .aisnet.org/cgi/viewcontent .cgi?article =1221&context =ecis2011 .
Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
PDF version of this title (4.5M)
Disable Glossary Links

In this Page

Introduction
Overview of the Literature Review Process and Steps
Types of Review Articles and Brief Illustrations
Concluding Remarks

Related information

PMC PubMed Central citations
PubMed Links to PubMed

Recent Activity

Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Dissertations - Skills Guide

Where to start
Research Proposal
Ethics Form
Primary Research

Literature Review

Methodology
Downloadable Resources
Further Reading

What is it?

Literature reviews involve collecting information from literature that is already available, similar to a long essay. It is a written argument that builds a case from previous research (Machi and McEvoy, 2012). Every dissertation should include a literature review, but a dissertation as a whole can be a literature review. In this section we discuss literature reviews for the whole dissertation.

What are the benefits of a literature review?

There are advantages and disadvantages to any approach. The advantages of conducting a literature review include accessibility, deeper understanding of your chosen topic, identifying experts and current research within that area, and answering key questions about current research. The disadvantages might include not providing new information on the subject and, depending on the subject area, you may have to include information that is out of date.

How do I write it?

A literature review is often split into chapters, you can choose if these chapters have titles that represent the information within them, or call them chapter 1, chapter 2, ect. A regular format for a literature review is:

Introduction (including methodology)

This particular example is split into 6 sections, however it may be more or less depending on your topic.

Literature Reviews Further Reading

<< Previous: Primary Research
Next: Methodology >>
Last Updated: Oct 18, 2023 9:32 AM
URL: https://libguides.derby.ac.uk/c.php?g=690330

Research Limitations

It is for sure that your research will have some limitations and it is normal. However, it is critically important for you to be striving to minimize the range of scope of limitations throughout the research process. Also, you need to provide the acknowledgement of your research limitations in conclusions chapter honestly.

It is always better to identify and acknowledge shortcomings of your work, rather than to leave them pointed out to your by your dissertation assessor. While discussing your research limitations, don’t just provide the list and description of shortcomings of your work. It is also important for you to explain how these limitations have impacted your research findings.

Your research may have multiple limitations, but you need to discuss only those limitations that directly relate to your research problems. For example, if conducting a meta-analysis of the secondary data has not been stated as your research objective, no need to mention it as your research limitation.

Research limitations in a typical dissertation may relate to the following points:

1. Formulation of research aims and objectives . You might have formulated research aims and objectives too broadly. You can specify in which ways the formulation of research aims and objectives could be narrowed so that the level of focus of the study could be increased.

2. Implementation of data collection method . Because you do not have an extensive experience in primary data collection (otherwise you would not be reading this book), there is a great chance that the nature of implementation of data collection method is flawed.

3. Sample size. Sample size depends on the nature of the research problem. If sample size is too small, statistical tests would not be able to identify significant relationships within data set. You can state that basing your study in larger sample size could have generated more accurate results. The importance of sample size is greater in quantitative studies compared to qualitative studies.

4. Lack of previous studies in the research area . Literature review is an important part of any research, because it helps to identify the scope of works that have been done so far in research area. Literature review findings are used as the foundation for the researcher to be built upon to achieve her research objectives.

However, there may be little, if any, prior research on your topic if you have focused on the most contemporary and evolving research problem or too narrow research problem. For example, if you have chosen to explore the role of Bitcoins as the future currency, you may not be able to find tons of scholarly paper addressing the research problem, because Bitcoins are only a recent phenomenon.

5. Scope of discussions . You can include this point as a limitation of your research regardless of the choice of the research area. Because (most likely) you don’t have many years of experience of conducing researches and producing academic papers of such a large size individually, the scope and depth of discussions in your paper is compromised in many levels compared to the works of experienced scholars.

You can discuss certain points from your research limitations as the suggestion for further research at conclusions chapter of your dissertation.

My e-book, The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline. John Dudovskiy

Service placement in fog–cloud computing environments: a comprehensive literature review

Published: 02 May 2024

Cite this article

Fatemeh Sarkohaki 1 &
Mohsen Sharifi 1

70 Accesses

Explore all metrics

With the rapid expansion of the Internet of Things and the surge in the volume of data exchanged in it, cloud computing became more significant. To face the challenges of the cloud, the idea of fog computing was formed. The heterogeneity of nodes, distribution, and limitation of their resources in fog computing in turn led to the formation of the service placement problem. In service placement, we are looking for the mapping of the requested services to the available nodes so that a set of Quality-of-Service objectives are satisfied. Since the problem is NP-hard, various methods have been proposed to solve it, each of which has its advantages and shortcomings. In this survey paper, while reviewing the most prominent state-of-the-art service placement methods by presenting a taxonomy based on their optimization strategy, the advantages, disadvantages, and applications of each category of methods are discussed. Consequently, recommendations for future works are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

A scalable and flexible platform for service placement in multi-fog and multi-cloud environments

Evaluation of fog application placement algorithms: a survey

Resource Management Approaches in Fog Computing: a Comprehensive Review

Data availability.

This manuscript is a review article, and no data or materials have been used.

Mobile Edge Computing.

Qays MO et al (2023) Key communication technologies, applications, protocols and future guides for IoT-assisted smart grid systems: a review. Energy Rep 9:2440–2452

Article Google Scholar

Moudgil V et al (2023) Integration of IoT in building energy infrastructure: a critical review on challenges and solutions. Renew Sustain Energy Rev 174:113121

Sharma VK et al (2022) An optimization-based machine learning technique for smart home security using 5G. Comput Electr Eng 104:108434

Philip SJ, Luu TJ, Carte T (2023) There’s No place like home: Understanding users’ intentions toward securing internet-of-things (IoT) smart home networks. Comput Hum Behav 139:107551

Khanpara P et al (2023) A context-aware internet of things-driven security scheme for smart homes. Secur Priv 6(1):e269

Zaminkar M, Sarkohaki F, Fotohi R (2021) A method based on encryption and node rating for securing the RPL protocol communications in the IoT ecosystem. Int J Commun Syst 34(3):e4693

Salehi-Amiri A et al (2022) Designing an effective two-stage, sustainable, and IoT based waste management system. Renew Sustain Energy Rev 157:112031

Salman MY, Hasar H (2023) Review on environmental aspects in smart city concept: water, waste, air pollution and transportation smart applications using IoT techniques. Sustain Cities Soc 94:104567

Hashemi-Amiri O et al (2023) An allocation-routing optimization model for integrated solid waste management. Exp Syst Appl 227:120364

Sridhar K et al (2023) A modular IOT sensing platform using hybrid learning ability for air quality prediction. Meas Sens 25:100609

Barthwal A (2023) A Markov chain–based IoT system for monitoring and analysis of urban air quality. Environ Monit Assess 195(1):235

Article MathSciNet Google Scholar

Kumar M et al (2023) Quality assessment and monitoring of river water using IoT infrastructure. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2023.3238123

Kumar P et al (2023) A blockchain-orchestrated deep learning approach for secure data transmission in IoT-enabled healthcare system. J Parallel Distrib Comput 172:69–83

Krishnamoorthy S, Dua A, Gupta S (2023) Role of emerging technologies in future IoT-driven healthcare 4.0 technologies: a survey, current challenges and future directions. J Ambient Intell Humaniz Comput 14(1):361–407

Rejeb A et al (2023) The Internet of Things (IoT) in healthcare: Taking stock and moving forward. Internet of Things 22:100721

Ahmed ST, Kumar V, Kim J (2023) AITel: eHealth augmented intelligence based telemedicine resource recommendation framework for iot devices in smart cities. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2023.3243784

Cheikhrouhou O et al (2023) A lightweight blockchain and fog-enabled secure remote patient monitoring system. Internet of Things 22:100691

Khan AA et al (2023) The collaborative role of blockchain, artificial intelligence, and industrial internet of things in digitalization of small and medium-size enterprises. Sci Rep 13(1):1656

Rahman A et al (2023) Towards a blockchain-SDN-based secure architecture for cloud computing in smart industrial IoT. Digit Commun Netw 9(2):411–421

Huang J et al (2023) AoI-aware energy control and computation offloading for industrial IoT. Futur Gener Comput Syst 139:29–37

Karakaya A, Ulu A, Akleylek S (2022) GOALALERT: a novel real-time technical team alert approach using machine learning on an IoT-based system in sports. Microprocess Microsyst 93:104606

Liu L (2021) Construction of youth public sports service system based on embedded system and wireless IoT. Microprocess Microsyst 83:103984

Prajapati D et al (2022) Blockchain and IoT embedded sustainable virtual closed-loop supply chain in E-commerce towards the circular economy. Comput Ind Eng 172:108530

Kulkarni PM et al (2022) IOT data fusion framework for e-commerce. Meas Sens 24:100507

Boursianis AD et al (2022) Internet of things (IoT) and agricultural unmanned aerial vehicles (UAVs) in smart farming: a comprehensive review. Internet of Things 18:100187

Zeng H et al (2023) An IoT and Blockchain-based approach for the smart water management system in agriculture. Expert Syst 40(4):e12892

McCaig M, Rezania D, Dara R (2023) Framing the response to IoT in agriculture: a discourse analysis. Agric Syst 204:103557

Krishankumar R, Ecer F (2023) Selection of IoT service provider for sustainable transport using q-rung orthopair fuzzy CRADIS and unknown weights. Appl Soft Comput 132:109870

Jiang H et al (2023) RETRACTED ARTICLE: creating a ubiquitous learning environment using IoT in transportation. Soft Comput 27(2):1213–1213

Wu X et al (2023) A digital decision approach for scheduling process planning of shared bikes under internet of things environment. Appl Soft Comput 133:109934

Kuo Y-H, Leung JM, Yan Y (2023) Public transport for smart cities: recent innovations and future challenges. Eur J Oper Res 306(3):1001–1026

Quy VK et al (2022) Smart healthcare IoT applications based on fog computing: architecture, applications and challenges. Complex Intell Syst 8(5):3805–3815

Peixoto M et al (2023) FogJam: a fog service for detecting traffic congestion in a continuous data stream VANET. Ad Hoc Netw 140:103046

Tavousi F, Azizi S, Ghaderzadeh A (2022) A fuzzy approach for optimal placement of IoT applications in fog–cloud computing. Clust Comput 25:1–18

Sabuj SR et al (2022) Delay optimization in mobile edge computing: cognitive UAV-assisted eMBB and mMTC services. IEEE Trans Cognit Commun Netw 8(2):1019–1033

Kang H et al (2023) Cooperative UAV resource allocation and task offloading in hierarchical aerial computing systems: a MAPPO based approach. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2023.3240173

He Y et al (2022) Trajectory optimization and channel allocation for delay sensitive secure transmission in UAV-relayed VANETs. IEEE Trans Veh Technol 71(4):4512–4517

Sadeghi-Niaraki A (2023) Internet of thing (IoT) review of review: bibliometric overview since its foundation. Futur Gener Comput Syst. https://doi.org/10.1016/j.future.2023.01.016

Rahimikhanghah A et al (2022) Resource scheduling methods in cloud and fog computing environments: a systematic literature review. Clust Comput. https://doi.org/10.1007/s10586-021-03467-1

Bonomi, F., et al. Fog computing and its role in the internet of things . in Proceedings of the first edition of the MCC workshop on Mobile cloud computing . 2012.

Kumar D, Annam S (2022) Fog Computing Applications with Decentralized Computing Infrastructure—Systematic Review . in PROCEEDINGS OF THE SEVENTH INTERNATIONAL CONFERENCE ON MATHEMATICS AND COMPUTING: ICMC 2021 . 2022. Springer.

Songhorabadi M et al (2023) Fog computing approaches in IoT-enabled smart cities. J Netw Comput Appl 211:103557

Sethi V, Pal S (2023) FedDOVe: a federated deep Q-learning-based offloading for vehicular fog computing. Futur Gener Comput Syst 141:96–105

Hazra A et al (2023) Fog computing for next-generation internet of things: fundamental, state-of-the-art and research challenges. Comput Sci Rev 48:100549

Singh S, Vidyarthi D (2023) An integrated approach of ml-metaheuristics for secure service placement in fog–cloud ecosystem. Internet of Things 22:100817

Singh S, Vidyarthi D (2022) QoS-Aware Service Placement for Fog Integrated Cloud Using Modified Neuro-Fuzzy Approach . in Soft Computing and Its Engineering Applications: 4th INTERNATIONAL CONFERENCE, icSoftComp 2022, Changa, Anand, India, December 9–10, 2022, Proceedings . 2023. Springer.

Teng M et al. (2020) Priority based service placement strategy in heterogeneous mobile edge computing . in Algorithms and Architectures for Parallel Processing: 20th INTERNATIONAL CONFERENCE, ICA3PP 2020, New York City, NY, USA, October 2–4, 2020, Proceedings, Part I 20 . 2020. Springer.

Zare M, Sola YE, Hasanpour H (2023) Towards distributed and autonomous IoT service placement in fog computing using asynchronous advantage actor-critic algorithm. J King Saud Univ Comput Inf Sci 35(1):368–381

Google Scholar

Das R, Inuwa MM (2023) A review on fog computing: issues, characteristics, challenges, and potential applications. Telematics and Informatics Reports, p. 100049.

Salaht FA, Desprez F, Lebre A (2020) An overview of service placement problem in fog and edge computing. ACM Comput Surv (CSUR) 53(3):1–35

Matoušek J, Gärtner B (2007) Understanding and using linear programming. Springer, Berlin

Kuhn HW, Tucker AW (2013) Nonlinear programming. Traces and emergence of nonlinear programming. Springer, Berlin, pp 247–258

Vielma JP (2015) Mixed integer linear programming formulation techniques. SIAM Rev 57(1):3–57

Velasquez K et al (2017) Service placement for latency reduction in the internet of things. Ann Telecommun 72:105–115

Tinini RI et al. (2017) Optimal placement of virtualized BBU processing in hybrid cloud-fog RAN over TWDM-PON . in GLOBECOM 2017–2017 IEEE GLOBAL COMMUNICATIONS CONFERENCE . IEEE.

Gong Y (2020) Optimal edge server and service placement in mobile edge computing . in 2020 IEEE 9th JOINT INTERNATIONAL INFORMATION TECHNOLOGY AND ARTIFICIAL INTELLIGENCE CONFERENCE (ITAIC) . IEEE.

Kim W-S, Chung S-H (2018) User-participatory fog computing architecture and its management schemes for improving feasibility. IEEE Access 6:20262–20278

Yala L, Frangoudis PA, Ksentini A (2018) Latency and availability driven VNF placement in a MEC-NFV environment . in 2018 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM) . IEEE.

Daneshfar N et al. (2018) Service allocation in a mobile fog infrastructure under availability and qos constraints . in 2018 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM) . IEEE.

Donassolo B, et al. (2019) Fog based framework for IoT service provisioning . in 2019 16th IEEE ANNUAL CONSUMER COMMUNICATIONS & NETWORKING CONFERENCE (CCNC) . IEEE.

Chen M et al (2013) Markov approximation for combinatorial network optimization. IEEE Trans Inf Theory 59(10):6301–6327

Yu R, Xue G, Zhang X (2018) Application provisioning in fog computing-enabled internet-of-things: A network perspective . in IEEE INFOCOM 2018-IEEE CONFERENCE ON COMPUTER COMMUNICATIONS . IEEE.

Ouyang T, Zhou Z, Chen X (2018) Follow me at the edge: mobility-aware dynamic service placement for mobile edge computing. IEEE J Sel Areas Commun 36(10):2333–2345

Ning Z et al (2020) Distributed and dynamic service placement in pervasive edge computing networks. IEEE Trans Parallel Distrib Syst 32(6):1277–1292

Jokar E, Mosleh M, Kheyrandish M (2022) Discovering community structure in social networks based on the synergy of label propagation and simulated annealing. Multimed Tools Appl 81(15):21449–21470

Mirjalili S, Mirjalili S (2019) Genetic algorithm. Evolutionary Algorithms and Neural Networks: Theory and Applications, p. 43–55.

Wang D, Tan D, Liu L (2018) Particle swarm optimization algorithm: an overview. Soft Comput 22:387–408

Blum C (2005) Ant colony optimization: Introduction and recent trends. Phys Life Rev 2(4):353–373

Jokar E, Mosleh M, Kheyrandish M (2022) GWBM: an algorithm based on grey wolf optimization and balanced modularity for community discovery in social networks. J Supercomput 78(5):7354–7377

Hoseiny F et al. (2021) PGA: a priority-aware genetic algorithm for task scheduling in heterogeneous fog–cloud computing . in IEEE INFOCOM 2021-IEEE CONFERENCE ON COMPUTER COMMUNICATIONS WORKSHOPS (INFOCOM WKSHPS) . IEEE.

Sarrafzade N, Entezari-Maleki R, Sousa L (2022) A genetic-based approach for service placement in fog computing. J Supercomput 78(8):10854–10875

Maia AM et al. (2020) Dynamic service placement and load distribution in edge computing . in 2020 16TH INTERNATIONAL CONFERENCE ON NETWORK AND SERVICE MANAGEMENT (CNSM) . IEEE.

Khosroabadi F, Fotouhi-Ghazvini F, Fotouhi H (2021) Scatter: service placement in real-time fog-assisted iot networks. J Sens Actuator Netw 10(2):26

Eyckerman R et al (2020) Requirements for distributed task placement in the fog. Internet of Things 12:100237

Souza VB et al (2018) Towards a proper service placement in combined Fog-to-Cloud (F2C) architectures. Futur Gener Comput Syst 87:1–15

Apat HK et al. (2021) A Nature-Inspired-Based Multi-objective Service Placement in Fog Computing Environment , in Intelligent Systems: Proceedings of ICMIB 2020 . Springer. p. 293–304.

Ma R (2021) Edge server placement for service offloading in internet of things. Secur Commun Netw 2021:1–16

Hu Y et al (2022) An energy-aware service placement strategy using hybrid meta-heuristic algorithm in iot environments. Clust Comput 26:1–7

Natesha B, Guddeti RMR (2021) Adopting elitism-based Genetic Algorithm for minimizing multi-objective problems of IoT service placement in fog computing environment. J Netw Comput Appl 178:102972

Natesha B, Guddeti RMR (2022) Meta-heuristic based hybrid service placement strategies for two-level fog computing architecture. J Netw Syst Manage 30(3):47

Guerrero C, Lera I, Juiz C (2019) Evaluation and efficiency comparison of evolutionary algorithms for service placement optimization in fog architectures. Futur Gener Comput Syst 97:131–144

Shahryari O-K et al (2021) Energy and task completion time trade-off for task offloading in fog-enabled IoT networks. Pervasive Mob Comput 74:101395

Apat HK et al (2024) A hybrid meta-heuristic algorithm for multi-objective IoT service placement in fog computing environments. Decis Anal J 10:100379

Azizi S et al (2024) DCSP: a delay and cost-aware service placement and load distribution algorithm for IoT-based fog networks. Comput Commun 215:9–20

Jordan MI, Mitchell TM (2015) Machine learning: Trends, perspectives, and prospects. Science 349(6245):255–260

Jokar E, Mosleh M, Kheyrandish M (2022) Overlapping community detection in complex networks using fuzzy theory, balanced link density, and label propagation. Expert Syst 39(5):e12921

Quadri C, Ceselli A, Rossi GP (2023) Multi-user edge service orchestration based on deep reinforcement learning. Comput Commun 203:30–47

Hao H et al (2023) Computing offloading with fairness guarantee: a deep reinforcement learning method. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2023.3255229

Liu T et al (2022) Deep reinforcement learning based approach for online service placement and computation resource allocation in edge computing. IEEE Trans Mob Comput. https://doi.org/10.1109/TMC.2022.3148254

Zhan W et al (2020) Deep-reinforcement-learning-based offloading scheduling for vehicular edge computing. IEEE Internet Things J 7(6):5449–5465

Talpur A, Gurusamy M (2021) DRLD-SP: a deep-reinforcement-learning-based dynamic service placement in edge-enabled internet of vehicles. IEEE Internet Things J 9(8):6239–6251

Lv P et al (2022) Edge computing task offloading for environmental perception of autonomous vehicles in 6G networks. IEEE Trans Netw Sci Eng. https://doi.org/10.1109/TNSE.2022.3211193

Ibn-Khedher H et al (2022) Next-generation edge computing assisted autonomous driving based artificial intelligence algorithms. IEEE Access 10:53987–54001

Zhou Z et al (2019) Reliable task offloading for vehicular fog computing under information asymmetry and information uncertainty. IEEE Trans Veh Technol 68(9):8322–8335

Nsouli A, El-Hajj W, Mourad A (2023) Reinforcement learning based scheme for on-demand vehicular fog formation. Veh Commun 40:100571

Wei D et al (2022) Privacy-aware multiagent deep reinforcement learning for task offloading in VANET. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2022.3202196

Sharma A, Thangaraj V (2024) Intelligent service placement algorithm based on DDQN and prioritized experience replay in IoT-Fog computing environment. Internet of Things 25:101112

Tian Z et al (2019) Evaluating reputation management schemes of internet of vehicles based on evolutionary game theory. IEEE Trans Veh Technol 68(6):5971–5980

Sun Z et al (2023) BARGAIN-MATCH: a game theoretical approach for resource allocation and task offloading in vehicular edge computing networks. IEEE Trans Mob Comput. https://doi.org/10.1109/TMC.2023.3239339

Chen Y et al (2022) Qoe-aware decentralized task offloading and resource allocation for end-edge-cloud systems: a game-theoretical approach. IEEE Trans Mob Comput 23(1):769–784

Kayal P, Liebeherr J (2019) Distributed service placement in fog computing: An iterative combinatorial auction approach . in 2019 IEEE 39th INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS (ICDCS) . IEEE.

Sharma A, Thangaraj V (2022) DMAP: a decentralized matching game theory based optimized internet of things application placement in fog computing environment. Concurr Comput Pract Exp 34(23):e7189

Shi D et al (2020) Mean field game guided deep reinforcement learning for task placement in cooperative multiaccess edge computing. IEEE Internet Things J 7(10):9330–9340

Aloqaily MB, Kantarci, Mouftah HT (2017) Fairness-aware game theoretic approach for service management in vehicular clouds . in 2017 IEEE 86th VEHICULAR TECHNOLOGY CONFERENCE (VTC-Fall) . IEEE.

Zafari F et al (2020) Let’s share: a game-theoretic framework for resource sharing in mobile edge clouds. IEEE Trans Netw Serv Manage 18(2):2107–2122

Xiao Z et al (2019) Vehicular task offloading via heat-aware MEC cooperation using game-theoretic method. IEEE Internet Things J 7(3):2038–2052

Shabir B et al (2022) On collective intellect for task offloading in vehicular fog paradigm. IEEE Access 10:101445–101457

Krogh A (2008) What are artificial neural networks? Nat Biotechnol 26(2):195–197

Wu Z et al (2020) A comprehensive survey on graph neural networks. IEEE trans Neural Netw Learn Syst 32(1):4–24

Li Y, Liang S, Jiang Y (2023) Path reliability-based graph attention networks. Neural Netw 159:153–160

Veličković P (2023) Everything is connected: Graph neural networks. Curr Opin Struct Biol 79:102538

Zhong X and He Y (2021) A Cybertwin-Driven Task Offloading Scheme Based on Deep Reinforcement Learning and Graph Attention Networks . in 2021 13th INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS AND SIGNAL PROCESSING (WCSP) . IEEE.

Wu T et al. (2021) A Scalable Computation Offloading Scheme for MEC Based on Graph Neural Networks . in 2021 IEEE Globecom Workshops (GC Wkshps) . IEEE.

Eyckerman R et al. (2022) Application placement in fog environments using multi-objective reinforcement learning with maximum reward formulation . in NOMS 2022–2022 IEEE/IFIP network operations and management symposium . IEEE.

Zhang J et al. (2022) Fine-grained service offloading in B5G/6G collaborative edge computing based on graph neural networks . in ICC 2022-IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS . IEEE.

He Y et al (2022) A DDPG hybrid of graph attention network and action branching for multi-scale end-edge-cloud vehicular orchestrated task offloading. IEEE Wirel Commun. https://doi.org/10.1109/MWC.019.2100718

Tang Z et al. (2020) Dependent task offloading for multiple jobs in edge computing . in 2020 29th INTERNATIONAL CONFERENCE ON COMPUTER COMMUNICATIONS AND NETWORKS (ICCCN) . IEEE.

Sun Z, Mo Y, Yu C (2021) Graph reinforcement learning based task offloading for multi-access edge computing. IEEE Internet Things J 10(4):3138–3150

Liu B (2024) Hybrid fuzzy neural network for joint task offloading in the internet of vehicles. J Grid Comput 22(1):10

Tong S et al (2022) Joint task offloading and resource allocation for fog-based intelligent transportation systems: a uav-enabled multi-hop collaboration paradigm. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2022.3163804

Sarkar I et al (2021) Dynamic task placement for deadline-aware IoT applications in federated fog networks. IEEE Internet Things J 9(2):1469–1478

Ayoubi M, Ramezanpour M, Khorsand R (2021) An autonomous IoT service placement methodology in fog computing. Softw Pract Exp 51(5):1097–1120

Cao T et al (2024) Walking on two legs: joint service placement and computation configuration for provisioning containerized services at edges. Comput Netw 239:110144

Download references

The authors did not receive support from any organization for the submitted work.

Author information

Authors and affiliations.

School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran

Fatemeh Sarkohaki & Mohsen Sharifi

You can also search for this author in PubMed Google Scholar

Contributions

Both authors in all stages of work including conceptualization, investigation, methodology, writing original draft, designing figures and tables, writing review, and editing have contributed equally.

Corresponding author

Correspondence to Mohsen Sharifi .

Ethics declarations

Competing interest.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

Hereby, We Fatemeh Sarkohaki and Mohsen Sharifi consciously assure that for the manuscript “Service Placement in Fog–cloud Computing Environments: A Comprehensive Literature Review” the following is fulfilled: 1) This material is the original work of the authors, which has not been previously published elsewhere. 2) The paper is not currently being considered for publication elsewhere. 3) The paper reflects the authors’ own research and analysis in a truthful and complete manner. 4) The paper properly credits the meaningful contributions of co-authors and co-researchers. 5) All sources used are properly disclosed (correct citation). 6) All authors have been personally and actively involved in substantial work leading to the paper and will take public responsibility for its content.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Sarkohaki, F., Sharifi, M. Service placement in fog–cloud computing environments: a comprehensive literature review. J Supercomput (2024). https://doi.org/10.1007/s11227-024-06151-4

Download citation

Accepted : 14 April 2024

Published : 02 May 2024

DOI : https://doi.org/10.1007/s11227-024-06151-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Internet of things
Cloud computing
Fog computing
Service placement
Find a journal
Publish with us
Track your research

COMMENTS

8 common problems with literature reviews and how to fix them
In our recent paper in Nature Ecology and Evolution, we highlight 8 common problems with traditional literature review methods, provide examples for each from the field of environmental management and ecology, and provide practical solutions for ways to mitigate them. Problem. Solution. Lack of relevance - limited stakeholder engagement can ...
Eight problems with literature reviews and how to fix them
Environment. Policy*. Research Design. Systematic Reviews as Topic*. Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to ...
What is a literature review?
A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important ...
Advantages and disadvantages of literature review
Disadvantages of literature review . Time consuming . As a literature review involves collecting and evaluating research and summarizing the findings, it requires a significant amount of time. To conduct a comprehensive review, researchers need to read many different articles and analyse a lot of data. This means that their review will take a ...
Literature review as a research methodology: An ...
A literature review can broadly be described as a more or less systematic way of collecting and synthesizing previous research (Baumeister & Leary, 1997; Tranfield, Denyer, & Smart, 2003). ... the limitations of the search method, and the quality of the search process, and they often lack details on how the analysis was conducted. Second, in ...
Eight problems with literature reviews and how to fix them
Gillian Petrok ofsky, Andrew S. Pullin9, Sini Savilaakso 10,11 and Gavin B. Stewart 12. T raditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions ...
Eight common problems with science literature reviews and how to fix them
First, traditional literature reviews can lack relevance. This is because limited stakeholder engagement can lead to a review that is of limited practical use to decision-makers. Second, reviews ...
Systematic reviews: Brief overview of methods, limitations, and
CONCLUSION. Siddaway 16 noted that, "The best reviews synthesize studies to draw broad theoretical conclusions about what the literature means, linking theory to evidence and evidence to theory" (p. 747). To that end, high quality systematic reviews are explicit, rigorous, and reproducible. It is these three criteria that should guide authors seeking to write a systematic review or editors ...
Eight problems with literature reviews and how to fix them
The aims of literature reviews range from providing a primer for the uninitiated to summarizing the evidence for decision making 1. Traditional approaches to literature reviews are susceptible to ...
Writing the Literature Review: Common Mistakes and Best Practices
The literature review is an essential component of academic research writing, providing a comprehensive overview of existing research and informing the development of new studies. ... But this time, a solution to the original study's limitation is offered in boldfaced font as well as support for that solution using the findings of yet another ...
How to Write a Literature Review
Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.
The Literature Review: A Foundation for High-Quality Medical Education
The literature review is a vital part of medical education research and should occur throughout the research process to help researchers design a strong study and effectively communicate study results and importance. To achieve these goals, researchers are advised to plan and execute the literature review carefully. ...
Limitations of the literature
This work typically appears in a separate chapter in the thesis or dissertation under the title Literature Review. To help you, here are 10 typical examples often used to identify limitations in a particular body of literature—with a caveat—using education examples. 1. The sample size is too small and therefore not representative of the ...
Writing a literature review
A formal literature review is an evidence-based, in-depth analysis of a subject. There are many reasons for writing one and these will influence the length and style of your review, but in essence a literature review is a critical appraisal of the current collective knowledge on a subject. Rather than just being an exhaustive list of all that ...
Ten Simple Rules for Writing a Literature Review
Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications .For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively .Given such mountains of papers, scientists cannot be expected to examine in detail every ...
What is a Literature Review? How to Write It (with Examples)
A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship ...
Literature Reviews?
Most literature reviews are embedded in articles, books, and dissertations. In most research articles, there are set as a specific section, usually titled, "literature review", so they are hard to miss.But, sometimes, they are part of the narrative of the introduction of a book or article. This section is easily recognized since the author is engaging with other academics and experts by ...
Chapter 9 Methods for Literature Reviews
Literature reviews can take two major forms. The most prevalent one is the "literature review" or "background" section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses (Sylvester, Tate, & Johnstone, 2013).
(PDF) Literature Reviews: What are the Challenges, and ...
The literature review is a key part of a dissertation, academic or journal paper, yet it is also one that often proves challenging. ... on the relevant industry and again given the limitations of ...
Literature Review
Literature reviews involve collecting information from literature that is already available, similar to a long essay. It is a written argument that builds a case from previous research (Machi and McEvoy, 2012). Every dissertation should include a literature review, but a dissertation as a whole can be a literature review.
Research Limitations
Research limitations in a typical dissertation may relate to the following points: 1. Formulation of research aims and objectives. You might have formulated research aims and objectives too broadly. You can specify in which ways the formulation of research aims and objectives could be narrowed so that the level of focus of the study could be ...
PDF Thesis Write UP Margin adjusted
5.0 Limitations The limitations of a literature review of this nature is the complete reliance on previously published research and the availability of these studies using the method outlined in the search methodology (section 3.2) and the appropriateness of these studies with the criteria of the selection/exclusion procedure (section 3.5).
Full article: A Systematic Review of the Limitations and Associated
Abstract. This systematic review explores the limitations and opportunities associated with ChatGPT's application across various fields. Following a rigorous screening process of 485 studies identified through searches in Scopus, Web of Science, ERIC, and IEEE Xplore databases, 33 high-quality empirical studies were selected for analysis.
Sustainable tourist behavior: A systematic literature review and
The objectives of this systematic literature review of sustainable tourist behavior were as follows: (1) to clarify and redefine sustainable tourist behavior; (2) to elicit key themes emerging from this body of research and establish the main theoretical perspectives adopted by scholars during this process; and (3) to suggest potential ...
Narrative Reviews: Flexible, Rigorous, and Practical
A critical review is a narrative synthesis of literature that brings an interpretative lens: the review is shaped by a theory, a critical point of view, or perspectives from other domains to inform the literature analysis. Critical reviews involve an interpretative process that combines the reviewer's theoretical premise with existing theories ...
Brain Sciences
The systematic literature review and meta-analysis by Gabriele et al. confirmed that elevated 5-HT blood levels are the most reliable and frequently replicated biomarker for ASD, ... In addressing the limitations of this review on hyperserotonemia in ASD, firstly, it is crucial to recognize the inherent limitations of relying on existing ...
Service placement in fog-cloud computing environments: a ...
With the rapid expansion of the Internet of Things and the surge in the volume of data exchanged in it, cloud computing became more significant. To face the challenges of the cloud, the idea of fog computing was formed. The heterogeneity of nodes, distribution, and limitation of their resources in fog computing in turn led to the formation of the service placement problem. In service placement ...
Establishing the limitations of sustainable construction in developing
Consequently, the final framework presented a holistic approach to surmounting the established limitations and aided recommendations for future studies. Thus, setting a background for developing strategies to overcome the limitations and further attain sustainable development (SD). ... a systematic literature review using PRISMA Ogunmakinde ...
The influence of professional identity and occupational well-being on
Literature review The concept of teacher retention first appeared in Billingsley's (1991) study on the reasons for leaving special school teachers, and later, Tett and Meyer (1993) pointed out intention to stay is the behavior of teachers who are satisfied with their identity and willing to maintain it and decide to remain in the organization ...

Neal Haddaway

About the author

Leave a Comment Cancel reply

Related Posts

“But I’m not ready!” Common barriers to writing and how to overcome them

“Remember a condition of academic writing is that we expose ourselves to critique” – 15 steps to revising journal articles

A simple guide to ethical co-authorship

How common is academic plagiarism?

Eight problems with literature reviews and how to fix them

Literature Reviews

What is a Literature Review?

Purpose of a Literature Review

Limitations of a Literature Review

Author: M Rahman

Related Posts

Eight common problems with science literature reviews and how to fix them

Disclosure statement

Collaboration

Events and Communications Coordinator

Assistant Editor - 1 year cadetship

Executive Dean, Faculty of Health

Lecturer/Senior Lecturer, Earth System Science (School of Science)

Sydney Horizon Educators (Identified)

Eight problems with literature reviews and how to fix them

Access options

Similar content being viewed by others

Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health

Insights from a cross-sector review on how to conceptualise the quality of use of research evidence

The past, present and future of Registered Reports

Acknowledgements

Author information

Contributions

Corresponding author

Ethics declarations

Additional information

Supplementary information

Rights and permissions

About this article

Share this article

This article is cited by

A review of the necessity of a multi-layer land-use planning

Synthesizing the relationships between environmental DNA concentration and freshwater macrophyte abundance: a systematic review and meta-analysis

A Systematic Review of the Effects of Multi-purpose Forest Management Practices on the Breeding Success of Forest Birds

Parasitism in viviparous vertebrates: an overview

Quick links

Limitations of the literature – A guide to a sticky problem

The upsides to a pandemic: from jobs to teaching, it’s not all bad news

Leave a Comment Cancel Reply

First group of successful Future Professors celebrated

What is a Literature Review? How to Write It (with Examples)

Table of Contents

What is a literature review?

What is the purpose of literature review?

Literature review example

Literature Review on Climate Change Impacts on Biodiversity:

a. Habitat Loss and Species Extinction:

b. Range Shifts and Phenological Changes:

c. Ocean Acidification and Coral Reefs:

d. Adaptive Strategies and Conservation Efforts:

How to write a good literature review

Conducting a literature review

Choose a Topic and Define the Research Question:

Decide on the Scope of Your Review:

Select Databases for Searches:

Conduct Searches and Keep Track:

Review the Literature:

Organize and Write Your Literature Review:

Frequently asked questions

Related Reads:

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Literature Review: The What, Why and How-to Guide: Literature Reviews?

What is a Literature Review?

What kinds of literature reviews are written?

What are the Goals of Creating a Literature Review?

When do you need to write a Literature Review?

Where to Find Standalone Literature Reviews

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

9.1. Introduction

9.2. Overview of the Literature Review Process and Steps

9.3. Types of Review Articles and Brief Illustrations