literature reviews evaluate

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 9 April 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

• 5. Critically Analyze and Evaluate
• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles
• Searching Article Databases
• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References

Critically analyze and evaluate

Tip: read and annotate pdfs.

• 6. Synthesize
• 7. Write a Literature Review

Ask yourself questions like these about each book or article you include:

• What is the research question?
• What is the primary methodology used?
• How was the data gathered?
• How is the data presented?
• What are the main conclusions?
• Are these conclusions reasonable?
• What theories are used to support the researcher's conclusions?

Take notes on the articles as you read them and identify any themes or concepts that may apply to your research question.

This sample template (below) may also be useful for critically reading and organizing your articles. Or you can use this online form and email yourself a copy .

• Sample Template for Critical Analysis of the Literature

Opening an article in PDF format in Acrobat Reader will allow you to use "sticky notes" and "highlighting" to make notes on the article without printing it out. Make sure to save the edited file so you don't lose your notes!

Some Citation Managers like Mendeley also have highlighting and annotation features.Here's a screen capture of a pdf in Mendeley with highlighting, notes, and various colors:

Screen capture from a UO Librarian's Mendeley Desktop app

• Learn more about citation management software in the previous step: 4. Manage Your References
• << Previous: 4. Manage Your References
• Next: 6. Synthesize >>
• Last Updated: Jan 10, 2024 4:46 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

Literature Review - what is a Literature Review, why it is important and how it is done

What are literature reviews, goals of literature reviews, types of literature reviews, about this guide/licence.

• Strategies to Find Sources
• Evaluating Literature Reviews and Sources
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings
• Useful Resources

Help is Just a Click Away

Search our FAQ Knowledge base, ask a question, chat, send comments...

Go to LibAnswers

 What is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries. " - Quote from Taylor, D. (n.d) "The literature review: A few tips on conducting it"

Source NC State University Libraries. This video is published under a Creative Commons 3.0 BY-NC-SA US license.

What are the goals of creating a Literature Review?

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

- Baumeister, R.F. & Leary, M.R. (1997). "Writing narrative literature reviews," Review of General Psychology , 1(3), 311-320.

When do you need to write a Literature Review?

• When writing a prospectus or a thesis/dissertation
• When writing a research paper
• When writing a grant proposal

In all these cases you need to dedicate a chapter in these works to showcase what have been written about your research topic and to point out how your own research will shed a new light into these body of scholarship.

Literature reviews are also written as standalone articles as a way to survey a particular research topic in-depth. This type of literature reviews look at a topic from a historical perspective to see how the understanding of the topic have change through time.

What kinds of literature reviews are written?

• Narrative Review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.
• Book review essays/ Historiographical review essays : This is a type of review that focus on a small set of research books on a particular topic " to locate these books within current scholarship, critical methodologies, and approaches" in the field. - LARR
• Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L.K. (2013). Research in Communication Sciences and Disorders . San Diego, CA: Plural Publishing.
• Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M.C. & Ilardi, S.S. (2003). Handbook of Research Methods in Clinical Psychology . Malden, MA: Blackwell Pub.
• Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). "Qualitative meta-synthesis: A question of dialoguing with texts," Journal of Advanced Nursing , 53(3), 311-318.

Guide adapted from "Literature Review" , a guide developed by Marisol Ramos used under CC BY 4.0 /modified from original.

• Next: Strategies to Find Sources >>
• Last Updated: Apr 10, 2024 3:27 PM
• URL: https://lit.libguides.com/Literature-Review

The Library, Technological University of the Shannon: Midwest

• Library Homepage

Literature Review: The What, Why and How-to Guide: Evaluating Sources & Literature Reviews

• Literature Reviews?
• Strategies to Finding Sources
• Keeping up with Research!
• Evaluating Sources & Literature Reviews
• Organizing for Writing
• Writing Literature Review
• Other Academic Writings

Evaluating Literature Reviews and Sources

• Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

Criteria to evaluate sources:

• Authority : Who is the author? What are the author's credentials and areas of expertise? Is he or she affiliated with a university?
• Usefulness : How this source related to your topic? How current or relevant it is to your topic?
• Reliability : Does the information comes from a reliable, trusted source such as an academic journal?
• Critically Analyzing Information Sources: Critical Appraisal and Analysis (Cornell University Library) Ten things to look for when you evaluate an information source.

Reading Critically

Reading critically (summary from how to read academic texts critically).

• Who is the author? What is his/her standing in the field?
• What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, critique or clarify?
• Note the experts in the field: are there specific names/labs that are frequently cited?
• Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
• Note conflicting theories, methodologies, and results. Are there any assumptions being made by most/some researchers?
• Theories: have they evolved over time?
• Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?
• How to Read Academic Texts Critically Excellent document about how best to read critically academic articles and other texts.
• How to Read an Academic Article This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.
• << Previous: Keeping up with Research!
• Next: Organizing for Writing >>
• Last Updated: Mar 5, 2024 11:44 AM
• URL: https://guides.library.ucsb.edu/litreview

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 9, 2024 1:19 PM
• URL: https://libguides.usc.edu/writingguide

How To Write An A-Grade Literature Review

3 straightforward steps (with examples) + free template.

By: Derek Jansen (MBA) | Expert Reviewed By: Dr. Eunice Rautenbach | October 2019

Quality research is about building onto the existing work of others , “standing on the shoulders of giants”, as Newton put it. The literature review chapter of your dissertation, thesis or research project is where you synthesise this prior work and lay the theoretical foundation for your own research.

Long story short, this chapter is a pretty big deal, which is why you want to make sure you get it right . In this post, I’ll show you exactly how to write a literature review in three straightforward steps, so you can conquer this vital chapter (the smart way).

Overview: The Literature Review Process

• Understanding the “ why “
• Finding the relevant literature
• Cataloguing and synthesising the information
• Outlining & writing up your literature review
• Example of a literature review

But first, the “why”…

Before we unpack how to write the literature review chapter, we’ve got to look at the why . To put it bluntly, if you don’t understand the function and purpose of the literature review process, there’s no way you can pull it off well. So, what exactly is the purpose of the literature review?

Well, there are (at least) four core functions:

• For you to gain an understanding (and demonstrate this understanding) of where the research is at currently, what the key arguments and disagreements are.
• For you to identify the gap(s) in the literature and then use this as justification for your own research topic.
• To help you build a conceptual framework for empirical testing (if applicable to your research topic).
• To inform your methodological choices and help you source tried and tested questionnaires (for interviews ) and measurement instruments (for surveys ).

Most students understand the first point but don’t give any thought to the rest. To get the most from the literature review process, you must keep all four points front of mind as you review the literature (more on this shortly), or you’ll land up with a wonky foundation.

Okay – with the why out the way, let’s move on to the how . As mentioned above, writing your literature review is a process, which I’ll break down into three steps:

• Finding the most suitable literature
• Understanding , distilling and organising the literature
• Planning and writing up your literature review chapter

Importantly, you must complete steps one and two before you start writing up your chapter. I know it’s very tempting, but don’t try to kill two birds with one stone and write as you read. You’ll invariably end up wasting huge amounts of time re-writing and re-shaping, or you’ll just land up with a disjointed, hard-to-digest mess . Instead, you need to read first and distil the information, then plan and execute the writing.

Step 1: Find the relevant literature

Naturally, the first step in the literature review journey is to hunt down the existing research that’s relevant to your topic. While you probably already have a decent base of this from your research proposal , you need to expand on this substantially in the dissertation or thesis itself.

Essentially, you need to be looking for any existing literature that potentially helps you answer your research question (or develop it, if that’s not yet pinned down). There are numerous ways to find relevant literature, but I’ll cover my top four tactics here. I’d suggest combining all four methods to ensure that nothing slips past you:

Method 1 – Google Scholar Scrubbing

Google’s academic search engine, Google Scholar , is a great starting point as it provides a good high-level view of the relevant journal articles for whatever keyword you throw at it. Most valuably, it tells you how many times each article has been cited, which gives you an idea of how credible (or at least, popular) it is. Some articles will be free to access, while others will require an account, which brings us to the next method.

Method 2 – University Database Scrounging

Generally, universities provide students with access to an online library, which provides access to many (but not all) of the major journals.

So, if you find an article using Google Scholar that requires paid access (which is quite likely), search for that article in your university’s database – if it’s listed there, you’ll have access. Note that, generally, the search engine capabilities of these databases are poor, so make sure you search for the exact article name, or you might not find it.

Method 3 – Journal Article Snowballing

At the end of every academic journal article, you’ll find a list of references. As with any academic writing, these references are the building blocks of the article, so if the article is relevant to your topic, there’s a good chance a portion of the referenced works will be too. Do a quick scan of the titles and see what seems relevant, then search for the relevant ones in your university’s database.

Method 4 – Dissertation Scavenging

Similar to Method 3 above, you can leverage other students’ dissertations. All you have to do is skim through literature review chapters of existing dissertations related to your topic and you’ll find a gold mine of potential literature. Usually, your university will provide you with access to previous students’ dissertations, but you can also find a much larger selection in the following databases:

• Open Access Theses & Dissertations
• Stanford SearchWorks

Keep in mind that dissertations and theses are not as academically sound as published, peer-reviewed journal articles (because they’re written by students, not professionals), so be sure to check the credibility of any sources you find using this method. You can do this by assessing the citation count of any given article in Google Scholar. If you need help with assessing the credibility of any article, or with finding relevant research in general, you can chat with one of our Research Specialists .

Alright – with a good base of literature firmly under your belt, it’s time to move onto the next step.

Need a helping hand?

Step 2: Log, catalogue and synthesise

Once you’ve built a little treasure trove of articles, it’s time to get reading and start digesting the information – what does it all mean?

While I present steps one and two (hunting and digesting) as sequential, in reality, it’s more of a back-and-forth tango – you’ll read a little , then have an idea, spot a new citation, or a new potential variable, and then go back to searching for articles. This is perfectly natural – through the reading process, your thoughts will develop , new avenues might crop up, and directional adjustments might arise. This is, after all, one of the main purposes of the literature review process (i.e. to familiarise yourself with the current state of research in your field).

As you’re working through your treasure chest, it’s essential that you simultaneously start organising the information. There are three aspects to this:

• Logging reference information
• Building an organised catalogue
• Distilling and synthesising the information

I’ll discuss each of these below:

2.1 – Log the reference information

As you read each article, you should add it to your reference management software. I usually recommend Mendeley for this purpose (see the Mendeley 101 video below), but you can use whichever software you’re comfortable with. Most importantly, make sure you load EVERY article you read into your reference manager, even if it doesn’t seem very relevant at the time.

2.2 – Build an organised catalogue

In the beginning, you might feel confident that you can remember who said what, where, and what their main arguments were. Trust me, you won’t. If you do a thorough review of the relevant literature (as you must!), you’re going to read many, many articles, and it’s simply impossible to remember who said what, when, and in what context . Also, without the bird’s eye view that a catalogue provides, you’ll miss connections between various articles, and have no view of how the research developed over time. Simply put, it’s essential to build your own catalogue of the literature.

I would suggest using Excel to build your catalogue, as it allows you to run filters, colour code and sort – all very useful when your list grows large (which it will). How you lay your spreadsheet out is up to you, but I’d suggest you have the following columns (at minimum):

• Author, date, title – Start with three columns containing this core information. This will make it easy for you to search for titles with certain words, order research by date, or group by author.
• Categories or keywords – You can either create multiple columns, one for each category/theme and then tick the relevant categories, or you can have one column with keywords.
• Key arguments/points – Use this column to succinctly convey the essence of the article, the key arguments and implications thereof for your research.
• Context – Note the socioeconomic context in which the research was undertaken. For example, US-based, respondents aged 25-35, lower- income, etc. This will be useful for making an argument about gaps in the research.
• Methodology – Note which methodology was used and why. Also, note any issues you feel arise due to the methodology. Again, you can use this to make an argument about gaps in the research.
• Quotations – Note down any quoteworthy lines you feel might be useful later.
• Notes – Make notes about anything not already covered. For example, linkages to or disagreements with other theories, questions raised but unanswered, shortcomings or limitations, and so forth.

If you’d like, you can try out our free catalog template here (see screenshot below).

2.3 – Digest and synthesise

Most importantly, as you work through the literature and build your catalogue, you need to synthesise all the information in your own mind – how does it all fit together? Look for links between the various articles and try to develop a bigger picture view of the state of the research. Some important questions to ask yourself are:

• What answers does the existing research provide to my own research questions ?
• Which points do the researchers agree (and disagree) on?
• How has the research developed over time?
• Where do the gaps in the current research lie?

To help you develop a big-picture view and synthesise all the information, you might find mind mapping software such as Freemind useful. Alternatively, if you’re a fan of physical note-taking, investing in a large whiteboard might work for you.

Step 3: Outline and write it up!

Once you’re satisfied that you have digested and distilled all the relevant literature in your mind, it’s time to put pen to paper (or rather, fingers to keyboard). There are two steps here – outlining and writing:

3.1 – Draw up your outline

Having spent so much time reading, it might be tempting to just start writing up without a clear structure in mind. However, it’s critically important to decide on your structure and develop a detailed outline before you write anything. Your literature review chapter needs to present a clear, logical and an easy to follow narrative – and that requires some planning. Don’t try to wing it!

Naturally, you won’t always follow the plan to the letter, but without a detailed outline, you’re more than likely going to end up with a disjointed pile of waffle , and then you’re going to spend a far greater amount of time re-writing, hacking and patching. The adage, “measure twice, cut once” is very suitable here.

In terms of structure, the first decision you’ll have to make is whether you’ll lay out your review thematically (into themes) or chronologically (by date/period). The right choice depends on your topic, research objectives and research questions, which we discuss in this article .

Once that’s decided, you need to draw up an outline of your entire chapter in bullet point format. Try to get as detailed as possible, so that you know exactly what you’ll cover where, how each section will connect to the next, and how your entire argument will develop throughout the chapter. Also, at this stage, it’s a good idea to allocate rough word count limits for each section, so that you can identify word count problems before you’ve spent weeks or months writing!

PS – check out our free literature review chapter template…

3.2 – Get writing

With a detailed outline at your side, it’s time to start writing up (finally!). At this stage, it’s common to feel a bit of writer’s block and find yourself procrastinating under the pressure of finally having to put something on paper. To help with this, remember that the objective of the first draft is not perfection – it’s simply to get your thoughts out of your head and onto paper, after which you can refine them. The structure might change a little, the word count allocations might shift and shuffle, and you might add or remove a section – that’s all okay. Don’t worry about all this on your first draft – just get your thoughts down on paper.

Once you’ve got a full first draft (however rough it may be), step away from it for a day or two (longer if you can) and then come back at it with fresh eyes. Pay particular attention to the flow and narrative – does it fall fit together and flow from one section to another smoothly? Now’s the time to try to improve the linkage from each section to the next, tighten up the writing to be more concise, trim down word count and sand it down into a more digestible read.

Once you’ve done that, give your writing to a friend or colleague who is not a subject matter expert and ask them if they understand the overall discussion. The best way to assess this is to ask them to explain the chapter back to you. This technique will give you a strong indication of which points were clearly communicated and which weren’t. If you’re working with Grad Coach, this is a good time to have your Research Specialist review your chapter.

Finally, tighten it up and send it off to your supervisor for comment. Some might argue that you should be sending your work to your supervisor sooner than this (indeed your university might formally require this), but in my experience, supervisors are extremely short on time (and often patience), so, the more refined your chapter is, the less time they’ll waste on addressing basic issues (which you know about already) and the more time they’ll spend on valuable feedback that will increase your mark-earning potential.

Literature Review Example

In the video below, we unpack an actual literature review so that you can see how all the core components come together in reality.

Let’s Recap

In this post, we’ve covered how to research and write up a high-quality literature review chapter. Let’s do a quick recap of the key takeaways:

• It is essential to understand the WHY of the literature review before you read or write anything. Make sure you understand the 4 core functions of the process.
• The first step is to hunt down the relevant literature . You can do this using Google Scholar, your university database, the snowballing technique and by reviewing other dissertations and theses.
• Next, you need to log all the articles in your reference manager , build your own catalogue of literature and synthesise all the research.
• Following that, you need to develop a detailed outline of your entire chapter – the more detail the better. Don’t start writing without a clear outline (on paper, not in your head!)
• Write up your first draft in rough form – don’t aim for perfection. Remember, done beats perfect.
• Refine your second draft and get a layman’s perspective on it . Then tighten it up and submit it to your supervisor.

Psst… there’s more!

This post is an extract from our bestselling Udemy Course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

38 Comments

Thank you very much. This page is an eye opener and easy to comprehend.

This is awesome!

I wish I come across GradCoach earlier enough.

But all the same I’ll make use of this opportunity to the fullest.

Thank you for this good job.

Keep it up!

You’re welcome, Yinka. Thank you for the kind words. All the best writing your literature review.

Thank you for a very useful literature review session. Although I am doing most of the steps…it being my first masters an Mphil is a self study and one not sure you are on the right track. I have an amazing supervisor but one also knows they are super busy. So not wanting to bother on the minutae. Thank you.

You’re most welcome, Renee. Good luck with your literature review 🙂

This has been really helpful. Will make full use of it. 🙂

Thank you Gradcoach.

Really agreed. Admirable effort

thank you for this beautiful well explained recap.

Thank you so much for your guide of video and other instructions for the dissertation writing.

It is instrumental. It encouraged me to write a dissertation now.

Thank you the video was great – from someone that knows nothing thankyou

an amazing and very constructive way of presetting a topic, very useful, thanks for the effort,

It is timely

It is very good video of guidance for writing a research proposal and a dissertation. Since I have been watching and reading instructions, I have started my research proposal to write. I appreciate to Mr Jansen hugely.

I learn a lot from your videos. Very comprehensive and detailed.

Thank you for sharing your knowledge. As a research student, you learn better with your learning tips in research

I was really stuck in reading and gathering information but after watching these things are cleared thanks, it is so helpful.

Really helpful, Thank you for the effort in showing such information

This is super helpful thank you very much.

Thank you for this whole literature writing review.You have simplified the process.

I’m so glad I found GradCoach. Excellent information, Clear explanation, and Easy to follow, Many thanks Derek!

You’re welcome, Maithe. Good luck writing your literature review 🙂

Thank you Coach, you have greatly enriched and improved my knowledge

Great piece, so enriching and it is going to help me a great lot in my project and thesis, thanks so much

This is THE BEST site for ANYONE doing a masters or doctorate! Thank you for the sound advice and templates. You rock!

Thanks, Stephanie 🙂

This is mind blowing, the detailed explanation and simplicity is perfect.

I am doing two papers on my final year thesis, and I must stay I feel very confident to face both headlong after reading this article.

thank you so much.

if anyone is to get a paper done on time and in the best way possible, GRADCOACH is certainly the go to area!

This is very good video which is well explained with detailed explanation

Thank you excellent piece of work and great mentoring

Thanks, it was useful

Thank you very much. the video and the information were very helpful.

Good morning scholar. I’m delighted coming to know you even before the commencement of my dissertation which hopefully is expected in not more than six months from now. I would love to engage my study under your guidance from the beginning to the end. I love to know how to do good job

Thank you so much Derek for such useful information on writing up a good literature review. I am at a stage where I need to start writing my one. My proposal was accepted late last year but I honestly did not know where to start

Like the name of your YouTube implies you are GRAD (great,resource person, about dissertation). In short you are smart enough in coaching research work.

This is a very well thought out webpage. Very informative and a great read.

Very timely.

I appreciate.

Very comprehensive and eye opener for me as beginner in postgraduate study. Well explained and easy to understand. Appreciate and good reference in guiding me in my research journey. Thank you

Thank you. I requested to download the free literature review template, however, your website wouldn’t allow me to complete the request or complete a download. May I request that you email me the free template? Thank you.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Literature Reviews

• Step 1: Understanding Literature Reviews
• Step 2: Gathering Information
• Step 3: Organizing your Information (Intellectually and Physically)

Analyzing and Evaluating the Literature

Evaluating your sources, analysing your sources.

• Step 5: Writing Your Literature Review
• Other Resources

How to evaluate a source

Consider the more obvious elements of the paper:.

• Is its title clear? Does it accurately reflect the content of the paper?
• Is the abstract well-structured (providing an accurate, albeit brief, description of the purpose, method, theoretical background of the research, as well as its results or conclusions?)
• What does their bibliography look like? For example, if most of their references are quite old despite being a newer paper, you should see if they provide an explanation for that in the paper itself. If not, you may want to consider why they do not have any newer sources informing their research.
• Is the journal its published in prestigious and reputable, or does the journal stand to gain something from publishing this paper? You may need to consider the biases of not only the author, but the publisher!

Evaluate the content:

• Look for identifiable gaps in their method, as well as potential problems with their interpretation of the data.
• Look for any obvious manipulations of the data.
• Do they themselves identify any biases or limitations, or do you notice any that they haven’t identified? 

You are not just looking at what they are saying, but also at what they have NOT said. If they didn’t identify a clear gap or bias, why not? What does that say about the rest of the paper? If at all possible, you may want to see if you can identify where the funding for their study came from if you’re noticing these gaps, in case it is possible to spot a conflict of interest.

How do you analyze your sources?

It can be daunting to logically analyze the argument. If they are only showing one side and not addressing the topic from multiple perspectives, you may want to consider why, and if you feel they do a fair job of trying to present a holistic argument – if they don’t bring up conflicting information and demonstrate how they argument works against it, why not? You can also look for key red flags like:

• logical fallacies: mistakes in your reasoning that undermine the logic of the argument – usually identified due to a lack of evidence
• slippery slopes: a conclusion based on the idea that if one thing happens, it will trigger a series of other small steps leading to a drastic conclusion
• post hoc ergo propter hoc: a conclusion that says that if one event occurs after another, that it was the first event that caused the second due to chronology rather than evidence
• circular arguments: these are arguments that simply restate their premises rather than providing proof
• moral equivalence: this compares minor actions with major atrocities and concludes that both are equally immoral
• ad hominem: these are arguments that attack the character of the person making the argument rather than the argument itself

This is just a sample of the types or red flags that occur in academic writing. For more examples or further explanation, consult Purdue Owl’s academic writing guide, “ Logic in Argumentative Writing .”

• << Previous: Step 3: Organizing your Information (Intellectually and Physically)
• Next: Step 5: Writing Your Literature Review >>
• Last Updated: Feb 6, 2024 5:46 PM
• URL: https://yorkvilleu.libguides.com/LiteratureReviews

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

• October 26, 2022 recording
• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Literature reviews

• Introduction
• Conducting your search
• Store and organise the literature

Evaluate the information you have found

Critique the literature.

• Different subject areas
• Find literature reviews

When conducting your searches you may find many references that will not be suitable to use in your literature review.

• Skim through the resource - a quick read through the table of contents, the introductory paragraph or the abstract should indicate whether you need to read further or whether you can immediately discard the result.
• Evaluate the quality and reliability of the references you find - our page on evaluating information outlines what you need to consider when evaluating the books, journal articles, news and websites you find to ensure they are suitable for use in your literature review.

Critiquing the literature involves looking at the strength and weaknesses of the paper and evaluating the statements made by the author/s.

Books and resources on reading critically

• CASP Checklists Critical appraisal tools designed to be used when reading research. Includes tools for Qualitative studies, Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies and Clinical Prediction Rule.
• How to read critically - business and management From Postgraduate research in business - the aim of this chapter is to show you how to become a critical reader of typical academic literature in business and management.
• Learning to read critically in language and literacy Aims to develop skills of critical analysis and research design. It presents a series of examples of `best practice' in language and literacy education research.
• Critical appraisal in health sciences See tools for critically appraising health science research.

• << Previous: Store and organise the literature
• Next: Different subject areas >>
• Last Updated: Dec 15, 2023 12:09 PM
• URL: https://guides.library.uq.edu.au/research-techniques/literature-reviews

Conducting a Literature Review

• Getting Started
• Define your Research Question
• Finding Sources
• Evaluating Sources
• Organizing the Review
• Cite and Manage your Sources

Introduction

The process of evaluating sources can take place when you first encounter a source, when you're reading it over, and as you incorporate it into your project. In general, some level of evaluation should take place at all of these stages, with different goals for each. The last evaluation will be discussed in "Organizing the Review," but we'll go over the first two here.

The overall purpose of evaluating sources is to make sure that your review has the most relevant, accurate, and unbiased literature in the field, so that you can determine what has already been learned about your topic and where further research may be needed.

Additional Resources

• Evaluate Sources by Kansas State University Libraries
• How to Evaluate Any Source by Skyline College Library

Evaluating Sources During the Initial Search Process

When you first encounter a potential source, you'll want to know very quickly whether it is worth reading in detail and considering for your literature review. To avoid wasting time on unhelpful sources as much as possible, it's generally best to run each article, book, or other resource you find through a quick checklist, using information you can find by skimming through the summary and introduction.

The two most common forms of early source evaluation are the "Big 5 Criteria" and the "CRAAP Test." These cover the same most significant variables for evaluation, and which one to use comes down to preference.

Big 5 Criteria:

The most important criteria for evaluating a potential resource are:

• Currency : When the source was published
• Coverage/Relevance : How closely related the source is to your topic and research question
• Authority : Who wrote the source and whether they are likely to be credible on the subject
• Accuracy : Whether the information is accurate or not (this will be more heavily evaluated further on in the research process, but you should skim through quickly for any obviously inaccurate information as a means of disqualifying the source)
• Objectivity/Purpose : Whether or not the source presents a biased point of view or agenda

A good rule of thumb for  Currency  is that medical, scientific, and technology resources should be published within the last 5 years to prevent the information from being out-of-date; for less time-sensitive topics like history or the humanities, resources published within the last 5-10 years are often acceptable.

CRAAP Test:

A helpful mnemonic to remember the evaluation criteria, CRAAP is an acronym for:

Helpful questions for initial evaluation:

• When was this source published?
• Is this source relevant to your topic?
• What are the author or authors' qualifications?
• Is the resource scholarly/peer-reviewed?
• Are sources cited to support the author's claims?
• Does the website or journal the source comes from have a bias to their reporting?
• Do you notice biased or emotional language in the summary or introduction?
• Do you notice spelling or grammatical errors in a quick examination of the source?
• Writing a Research Paper: Evaluate Sources by Kansas State University

Evaluating Sources During the Reading Process

Once a resource has passed the initial evaluation, you are ready to begin reading through it to more carefully determine if it belongs in your project. In addition to the questions posed above, which are always relevant to evaluating sources, you should look at your potential sources of literature with an eye to the following questions:

1. Is there any bias visible in the work?

You already began this process in the previous step and hopefully eliminated the most obviously unreliable sources, but as you read it is always important to keep an eye out for potential blind spots the author might have based on their own perspective. Bias is not inherently disqualifying -- a biased article may still have accurate information -- but it is essential to know if a bias exists and be aware of how it might impact how the information was gathered, evaluated, or delivered.

Peer-reviewed sources tend to be less likely to have this risk, because multiple editors had to go through the resource looking for mistakes or biases. They need to meet a much higher academic threshold.

2. How was the research conducted? Are there any strengths or weaknesses in its methodology?

It is important to understand how the study in your source was administered; a significant part of the literature review will be about potential gaps in the current research, so you need to understand how the existing research was done.

3. How does the author justify their conclusions?

Either through the results of their own research or by citing external evidence, an article, book, or other type of resource should provide proof of its claims. In the initial research process you checked to make sure that there was evidence supporting the author's assertions; now it is time to take a look at that evidence and see if you find it compelling, or if you think it doesn't justify the conclusions drawn by the article.

4. What similarities do these articles share?

Grouping your literature review by categories based on subtopic, findings, or chronology can be an extremely helpful way to organize your work. Take notes while you're reading of common themes and results to make planning the review easier.

5. Where does this research differ from the other sources?

While very close to the previous question, this one emphasizes what unique information, methodology, or insights a particular source brings to the overall understanding of the topic. What  new  knowledge is being brought to the table by this source that would justify it appearing in your literature review?

6. Does this source leave any unanswered questions or opportunities for further research?

Most scholarly journal articles will include a section near the end of the article addressing limits of their study and opportunities for further research. Examine these closely and see where other sources you have fill in the gaps, and where perhaps additional research needs to be done to gain a more complete understanding of your topic.

As you go through this process, you might find yourself eliminating certain sources that no longer seem like they fit with your project's goals, or getting inspired to search for additional sources based on new information you've found. This is a normal part of the research process, and additional searching may be necessary to fill in any gaps left after evaluating the sources you already have.

• << Previous: Finding Sources
• Next: Organizing the Review >>
• Last Updated: Dec 19, 2022 1:17 PM
• URL: https://libraryservices.acphs.edu/lit_review

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 
• How to write a good literature review 
• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

• Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 
• Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 
• Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 
• Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 
• Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 
• Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

How to write a good literature review

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. 

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• Life Sciences Papers: 9 Tips for Authors Writing in Biological Sciences
• What is an Argumentative Essay? How to Write It (With Examples)

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, what is hedging in academic writing  , how to use ai to enhance your college..., ai + human expertise – a paradigm shift..., how to use paperpal to generate emails &..., ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., do plagiarism checkers detect ai content, word choice problems: how to use the right..., how to avoid plagiarism when using generative ai..., what are journal guidelines on using generative ai....

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: Jan 17, 2024 10:05 AM
• URL: https://libguides.northwestern.edu/literaturereviews

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

CPS Online Library Research Guide (UNH Manchester Library): Create a Literature Review

• Home & Table of Contents
• Different Types of Information
• The Savvy Information Consumer
• Primary & Secondary Sources
• Periodical Literature
• Peer Review
• Research Glossary
• Business Research Databases
• Understanding the Assignment
• Preliminary Considerations
• 7 Steps to Completing a Research Assignment
• Define the Topic
• Find & Evaluate Your Sources
• Research Integrity & Citing Your Sources
• Searching for Information
• Evaluating Information
• How to Read an Academic Journal Article This link opens in a new window
• Evaluating Social Media Sources
• Writing Your Research Paper
• Create a Literature Review
• Summarize an Article
• How to Write an Abstract
• How to Write a Book Review
• How to Do an Annotated Bibliography
• Finding Professional Organizations
• Find Key Journals in Your Field of Study
• Find Peer-Reviewed Articles
• Write a Research Paper Proposal
• Research a Company This link opens in a new window
• Citing Your Sources
• Avoiding Plagiarism
• NHCUC Libraries
• Other Types of Research
• Academic Libraries in NH
• Government Documents
• Using Google for Academic Research
• Information Literacy
• Developing Effective Library Research Assignments
• Using Permalinks
• Primary Source Websites

What is a literature review?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine.

More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc ., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews are likely to contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Attribution

"Create a Literature Review" is derivative of Literature Reviews by The Writing Center, University of North Carolina at Chapel Hill licensed Creative Commons Attribution-NonCommercial-NoDerivs 4.0 License .

• << Previous: Other Types of Research Assignments
• Next: Summarize an Article >>
• Last Updated: Apr 9, 2024 2:30 PM
• URL: https://libraryguides.unh.edu/CPSonlineLibraryResearch

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• Library Guides
• Literature Reviews
• Writing the Review

Literature Reviews: Writing the Review

Outline of review sections.

Your Literature Review should not be a summary and evaluation of each article, one after the other. Your sources should be integrated together to create a narrative on your topic.

Consider the following ways to organize your review:

• By themes, variables, or issues
• By varying perspectives regarding a topic of controversy
• Chronologically, to show how the topic and research have developed over time

Use an outline to organize your sources and ideas in a logical sequence. Identify main points and subpoints, and consider the flow of your review. Outlines can be revised as your ideas develop. They help guide your readers through your ideas and show the hierarchy of your thoughts. What do your readers need to understand first? Where might certain studies fit most naturally? These are the kinds of questions that an outline can clarify.

An example outline for a Literature Review might look like this:

Introduction

• Background information on the topic & definitions
• Purpose of the literature review
• Scope and limitations of the review (what is included /excluded)
• Historical background 
• Overview of the existing research on the topic
• Principle question being asked
• Organization of the literature into categories or themes
• Evaluation of the strengths and weaknesses of each study
• Combining the findings from multiple sources to identify patterns and trends
• Insight into the relationship between your central topic and a larger area of study
• Development of a new research question or hypothesis
• Summary of the key points and findings in the literature
• Discussion of gaps in the existing knowledge
• Implications for future research

Strategies for Writing

Annotated bibliography.

An annotated bibliography collects short descriptions of each source in one place. After you have read each source carefully, set aside some time to write a brief summary. Your summary might be simply informative (e.g. identify the main argument/hypothesis, methods, major findings, and/or conclusions), or it might be evaluative (e.g. state why the source is interesting or useful for your review, or why it is not).

This method is more narrative than the Literature Matrix talked about on the Documenting Your Search page.

Taking the time to write short informative and/or evaluative summaries of your sources while you are researching can help you transition into the drafting stage later on. By making a record of your sources’ contents and your reactions to them, you make it less likely that you will need to go back and re-read many sources while drafting, and you might also start to gain a clearer idea of the overarching shape of your review.

READ EXTANT LIT REVIEWS CLOSELY

As you conduct your research, you will likely read many sources that model the same kind of literature review that you are researching and writing. While your original intent in reading those sources is likely to learn from the studies’ content (e.g. their results and discussion), it will benefit you to re-read these articles rhetorically.

Reading rhetorically means paying attention to how a text is written—how it has been structured, how it presents its claims and analyses, how it employs transitional words and phrases to move from one idea to the next. You might also pay attention to an author’s stylistic choices, like the use of first-person pronouns, active and passive voice, or technical terminology.

See  Finding Example Literature Reviews on the Developing a Research Question page for tips on finding reviews relevant to your topic.

MIND-MAPPING

Creating a mind-map is a form of brainstorming that lets you visualize how your ideas function and relate. Draw the diagram freehand or download software that lets you easily manipulate and group text, images, and shapes ( Coggle ,  FreeMind , MindMaple ).

Write down a central idea, then identify associated concepts, features, or questions around that idea. Make lines attaching various ideas, or arrows to signify directional relationships. Use different shapes, sizes, or colors to indicate commonalities, sequences, or relative importance.

This drafting technique allows you to generate ideas while thinking visually about how they function together. As you follow lines of thought, you can see which ideas can be connected, where certain pathways lead, and what the scope of your project might be. By drawing out a mind-map you may be able to see what elements of your review are underdeveloped and will benefit from more focused attention.

USE VISUALIZATION TOOLS

Attribution.

Thanks to Librarian Jamie Niehof at the University of Michigan for providing permission to reuse and remix this Literature Reviews guide.

Avoiding Bias

Reporting bias.

This occurs when you are summarizing the literature in an unbalanced, inconsistent or distorted way . 

Ways to avoid:

• look for literature that supports multiple perspectives, viewpoints or theories 
• ask multiple people to review your writing for bias
• Last Updated: Apr 9, 2024 3:50 PM
• URL: https://info.library.okstate.edu/literaturereviews

• Open access
• Published: 10 April 2024

Development of an index system for the scientific literacy of medical staff: a modified Delphi study in China

• Shuyu Liang 2   na1 ,
• Ziyan Zhai 2   na1 ,
• Xingmiao Feng 2 ,
• Xiaozhi Sun 1 ,
• Jingxuan Jiao 1 ,
• Yuan Gao 1   na2 &
• Kai Meng   ORCID: orcid.org/0000-0003-1467-7904 2 , 3   na2  

BMC Medical Education volume  24 , Article number:  397 ( 2024 ) Cite this article

Metrics details

Scientific research activity in hospitals is important for promoting the development of clinical medicine, and the scientific literacy of medical staff plays an important role in improving the quality and competitiveness of hospital research. To date, no index system applicable to the scientific literacy of medical staff in China has been developed that can effectively evaluate and guide scientific literacy. This study aimed to establish an index system for the scientific literacy of medical staff in China and provide a reference for improving the evaluation of this system.

In this study, a preliminary indicator pool for the scientific literacy of medical staff was constructed through the nominal group technique ( n  = 16) with medical staff. Then, two rounds of Delphi expert consultation surveys ( n  = 20) were conducted with clinicians, and the indicators were screened, revised and supplemented using the boundary value method and expert opinions. Next, the hierarchical analysis method was utilized to determine the weights of the indicators and ultimately establish a scientific literacy indicator system for medical staff.

Following expert opinion, the index system for the scientific literacy of medical staff featuring 2 first-level indicators, 9 second-level indicators, and 38 third-level indicators was ultimately established, and the weights of the indicators were calculated. The two first-level indicators were research literacy and research ability, and the second-level indicators were research attitude (0.375), ability to identify problems (0.2038), basic literacy (0.1250), ability to implement projects (0.0843), research output capacity (0.0747), professional capacity (0.0735), data-processing capacity (0.0239), thesis-writing skills (0.0217), and ability to use literature (0.0181).

Conclusions

This study constructed a comprehensive scientific literacy index system that can assess medical staff's scientific literacy and serve as a reference for evaluating and improving their scientific literacy.

Peer Review reports

Due to the accelerated aging of the population and the growing global demand for healthcare in the wake of epidemics, there is an urgent need for medicine to provide greater support and protection. Medical scientific research is a critical element in promoting medical science and technological innovation, as well as improving clinical diagnosis and treatment techniques. It is the main driving force for the development of healthcare [ 1 ].

Medical personnel are highly compatible with clinical research. Due to their close interaction with patients, medical staff are better equipped to identify pertinent clinical research issues and actually implement clinical research projects [ 2 ]. Countries have created favorable conditions for the research and development of medical personnel by providing financial support, developing policies, and offering training courses [ 3 , 4 ]. However, some clinical studies have shown that the ability of most medical staff does not match current health needs and cannot meet the challenges posed by the twenty-first century [ 5 ]. It is clear that highly skilled professionals with scientific literacy are essential for national and social development [ 6 ]. Given the importance of scientific research in countries and hospitals, it is crucial to determine the level of scientific research literacy that medical personnel should possess and how to train them to acquire the necessary scientific research skills. These issues have significant practical implications.

Scientific literacy refers to an individual's ability to engage in science-related activities [ 7 ]. Some scholars suggest that the scientific literacy of medical personnel encompasses the fundamental qualities required for scientific research work, encompassing three facets: academic moral accomplishment, scientific research theory accomplishment, and scientific research ability accomplishment [ 8 ]. The existing research has focused primarily on the research capabilities of medical staff. According to Rillero, problem-solving skills, critical thinking, communication skills, and the ability to interpret data are the four core components of scientific literacy [ 9 ]. The ability to perform scientific research in nursing encompasses a range of abilities, including identifying problems, conducting literature reviews, designing and conducting scientific research, practicing scientific research, processing data, and writing papers [ 10 ]. Moule and Goodman proposed a framework of skills that research-literate nurses should possess, such as critical thinking capacity, analytical skills, searching skills, research critique skills, the ability to read and critically appraise research, and an awareness of ethical issues [ 11 ]. Several researchers have developed self-evaluation questionnaires to assess young researchers' scientific research and innovative abilities in the context of university-affiliated hospitals (UHAs) [ 12 ]. The relevant indicators include sensitivity to problems, sensitivity to cutting-edge knowledge, critical thinking, and other aspects. While these indicators cover many factors, they do not consider the issue of scientific research integrity in the medical field. The lack of detailed and targeted indicators, such as clinical resource collection ability and interdisciplinary cooperation ability, hinders the effective measurement of the current status of scientific literacy among medical staff [ 12 ]. In conclusion, the current research on the evaluation indicators of scientific literacy among medical personnel is incomplete, overlooking crucial humanistic characteristics, attitudes, and other moral literacy factors. Therefore, there is an urgent need to establish a comprehensive and systematic evaluation index to effectively assess the scientific literacy of medical staff.

Therefore, this study utilized a literature search and nominal group technique to screen the initial evaluation index and subsequently constructed an evaluation index system for medical staff's scientific research literacy utilizing the Delphi method. This index system would serve as a valuable tool for hospital managers, aiding them in the selection, evaluation, and training of scientific research talent. Additionally, this approach would enable medical personnel to identify their own areas of weakness and implement targeted improvement strategies.

Patient and public involvement

Patients and the public were not involved in this research.

Study design and participants

In this study, an initial evaluation index system was developed through a literature review and nominal group technique. Subsequently, a more comprehensive and scientific index system was constructed by combining qualitative and quantitative analysis utilizing the Delphi method to consult with experts. Finally, the hierarchical analysis method and the percentage weight method were employed to empower the index system.

The program used for this study is shown in Fig.  1 .

Study design. AHP, analytic hierarchy process

Establishing the preliminary indicator pool

Search process.

A literature search was performed in the China National Knowledge Infrastructure (CNKI), WanFang, PubMed, Web of Science and Scopus databases to collect the initial evaluation indicators. The time span ranged from the establishment of the database to July 2022. We used a combination of several MeSH terms in our searches:(("Medical Staff"[Mesh] OR "Nurses"[Mesh] OR "Physicians"[Mesh])) AND (("Literacy"[Mesh]) OR "Aptitude"[Mesh]). We also used several Title/Abstract searches, including keywords such as: Evaluation, scientific literacy, research ability.

The inclusion criteria were as follows: (1)The subjects were nurses, medicial staff and other personnel engaged in the medical industry; (2) Explore topics related to scientific literacy, such as research ability, and literature that can clarify the structure or dependency between indicators of scientific literacy; (3) Select articles published in countries such as China, the United States, the United Kingdom, Australia and Canada; (4) Research published in English or Chinese is considered to be eligible. The exclusion criteria are as follows: (1) indicators not applicable to medical staff; (2) Conference abstracts, case reports or review papers; (3) Articles with repeated descriptions; (4) There are no full-text articles or grey literature. A total of 78 articles were retrieved and 60 were retained after screening according to inclusion and exclusion criteria.

The research was conducted by two graduate students and two undergraduate students who participated in the literature search and screening. The entire research process was supervised and guided by one professor. All five members were from the fields of social medicine and health management. The professor was engaged in hospital management and health policy research for many years.

Nominal group technique

The nominal group technique was introduced at Hospital H in Beijing in July 2022. This hospital, with over 2,500 beds and 3,000 doctors, is a leading comprehensive medical center also known for its educational and research achievements, including numerous national research projects and awards.

The interview questions were based on the research question: What research literacy should medical staff have? 16 clinicians and nurses from Hospital H were divided into 2 equal groups and asked to provide their opinions on important aspects of research literacy based on their positions and experiences. Once all participants had shared their thoughts, similar responses were merged and polished. If anyone had further inputs after this, a second round of interviews was held until no new inputs were given. The entire meeting, including both rounds, was documented by researchers with audio recordings on a tape recorder.

Scientific literacy dimensions

Based on the search process, the research group extracted 58 tertiary indicators. To ensure the practicality and comprehensiveness of the indicators, the Nominal group technique was used on the basis of the literature search. Panelists summarized the entries shown in the interviews and merged similar content to obtain 32 third-level indicators. The indicators obtained from the literature search were compared. Several indicators with similar meanings, such as capture information ability, language expression ability, communication ability, and scientific research integrity, were merged. Additionally, the indicators obtained from the literature search, such as scientific research ethics, database use ability, feasibility and analysis ability, were added to the 15 indicators. A total of 47 third-level indicators were identified.

Fengling Dai and colleagues developed an innovation ability index system with six dimensions covering problem discovery, information retrieval, research design, practice, data analysis, and report writing, which represents the whole of innovative activity. Additionally, the system includes an innovation spirit index focusing on motivation, thinking, emotion, and will, reflecting the core of the innovation process in terms of competence [ 13 ]. Liao et al. evaluated the following five dimensions in their study on scientific research competence: literature processing, experimental manipulation, statistical analysis, manuscript production, and innovative project design [ 14 ]. Mohan claimed that scientific literacy consists of four core components: problem solving, critical thinking, communication skills, and the ability to interpret data [ 15 ].

This study structured scientific literacy into 2 primary indicators (research literacy and research competence) and 9 secondary indicators (basic qualifications, research ethics, research attitude, problem identification, literature use, professional capacity, subject implementation, data processing, thesis writing, and research output).

Using the Delphi method to develop an index system

Expert selection.

This study used the Delphi method to distribute expert consultation questionnaires online, allowing experts to exchange opinions anonymously to ensure that the findings were more desirable and scientific. No fixed number of experts is required for a Delphi study, but the more experts involved, the more stable the results will be [ 16 ]; this method generally includes 15 to 50 experts [ 17 ]. We selected clinicians from several tertiary hospitals in the Beijing area to serve as Delphi study consultants based on the following inclusion criteria: (1) they had a title of senior associate or above; (2) they had more than 10 years of work experience in the field of clinical scientific research, and (3) they were presiding over national scientific research projects. The exclusion criteria were as follows: (1) full-time scientific researchers, and (2) personnel in hospitals who were engaged only in management. To ensure that the selected experts were representative, this study selected 20 experts from 14 tertiary hospitals affiliated with Capital Medical University, Peking University, the Chinese Academy of Medical Sciences and the China Academy of Traditional Chinese Medicine according to the inclusion criteria; the hospitals featured an average of 1,231 beds each, and 9 hospitals were included among the 77 hospitals in the domestic comprehensive hospital ranking (Fudan Hospital Management Institute ranking). The experts represented various specialties and roles from different hospitals, including cardiology, neurosurgery, neurology, ear and throat surgery, head and neck surgery, radiology, imaging, infection, vascular interventional oncology, pediatrics, general practice, hematology, stomatology, nephrology, urology, and other related fields. This diverse group included physicians, nurses, managers, and vice presidents. The selected experts had extensive clinical experience, achieved numerous scientific research accomplishments and possessed profound knowledge and experience in clinical scientific research. This ensured the reliability of the consultation outcomes.

Design of the expert consultation questionnaire

The Delphi survey for experts included sections on their background, familiarity with the indicator system, system evaluation, and opinions. Experts rated indicators on importance, feasibility, and sensitivity using a 1–10 scale and their own familiarity with the indicators on a 1–5 scale. They also scored their judgment basis and impact on a 1–3 scale, considering theoretical analysis, work experience, peer understanding, and intuition. Two rounds of Delphi surveys were carried out via email with 20 experts to evaluate and suggest changes to the indicators. Statistical coefficients were calculated to validate the Delphi process. Feedback from the first round led to modifications and the inclusion of an AHP questionnaire for the second round. After the second round, indicators deemed less important were removed, and expert discussion finalized the indicator weights based on their relative importance scores. This resulted in the development of an index system for medical staff scientific literacy. The questionnaire is included in Additional file 1 (first round) and Additional file 2 (second round).

Using the boundary value method to screen the indicators

In this study, the boundary value method was utilized to screen the indicators of medical staff's scientific literacy, and the importance, feasibility, and sensitivity of each indicator were measured using the frequency of perfect scores, the arithmetic mean, and the coefficient of variation, respectively. When calculating the frequency of perfect scores and arithmetic means, the boundary value was set as "mean-SD," and indicators with scores higher than this value were retained. When calculating the coefficient of variation, the cutoff value was set to "mean + SD," and indicators with values below this threshold were retained.

The principles of indicator screening are as follows:

To evaluate the importance of the indicators, if none of the boundary values of the three statistics met the requirements, the indicators were deleted.

If an indicator has two aspects, importance, feasibility, or sensitivity, and each aspect has two or more boundary values that do not meet the requirements, then the indicator is deleted.

If all three boundary values for an indicator meet the requirements, the research group discusses the modification feedback from the experts and determines whether the indicator should be used.

The results of the two rounds of boundary values are shown in Table  1 .

Using the AHP to assign weights

After the second round of Delphi expert consultations, the analytic hierarchy process (AHP) was used to determine the weights of the two first-level indicators and the nine second-level indicators. The weights of the 37 third-level indicators were subsequently calculated via the percentage weight method. The AHP, developed by Saaty in the 1980s, is used to determine the priority and importance of elements constituting the decision-making hierarchy. It is based on multicriteria decision-making (MCDM) and determines the importance of decision-makers' judgments based on weights derived from pairwise comparisons between elements. In the AHP, pairwise comparisons are based on a comparative evaluation in which each element's weight in the lower tier is compared with that of other lower elements based on the element in the upper tier [ 18 ].

AHP analysis involves the following steps:

Step 1: Establish a final goal and list related elements to construct a hierarchy based on interrelated criteria.

Step 2: Perform a pairwise comparison for each layer to compare the weights of each element. Using a score from 1 to 9, which is the basic scale of the AHP, each pair is compared according to the expert’s judgment, and the importance is judged [ 19 , 20 ].

Yaahp software was employed to analyze data by creating a judgment matrix based on the experts' scores and hierarchical model. The index system weights were obtained by combining the experts' scores. The percentage weight method used experts' importance ratings from the second round to calculate weights, ranking indicators by importance, calculating their scores based on frequency of ranking, and determining weighting coefficients by dividing these scores by the total of all third-level indicators' scores. The third-level indicator weighting coefficients were then calculated by multiplying the coefficients [ 21 ].

Data analysis

Expert positivity coefficient.

The expert positivity coefficient is indicated by the effective recovery rate of the expert consultation questionnaire, which represents the level of expert positivity toward this consultation and determines the credibility and scientific validity of the questionnaire results. Generally, a questionnaire with an effective recovery rate of 70% is considered very good [ 22 ].

In this study, 20 questionnaires were distributed in both rounds of Delphi expert counseling, and all 20 were effectively recovered, resulting in a 100% effective recovery rate. Consequently, the experts provided positive feedback on the Delphi counseling.

Expert authority coefficient (CR)

The expert authority coefficient (Cr) is the arithmetic mean of the judgment coefficient (Ca) and the familiarity coefficient (Cs), namely, Cr =  \(\frac{({\text{Ca}}+{\text{Cs}})}{2}\) . The higher the degree of expert authority is, the greater the predictive accuracy of the indicator. A Cr ≥ 0.70 was considered to indicate an acceptable level of confidence [ 23 ]. Ca represents the basis on which the expert makes a judgment about the scenario in question, while Cs represents the expert's familiarity with the relevant problem [ 24 ].

Ca is calculated on the basis of experts' judgments of each indicator and the magnitude of its influence. In this study, experts used "practical experience (0.4), "theoretical analysis (0.3), "domestic and foreign peers (0.2)" and "intuition (0.1)" as the basis for judgment and assigned points according to the influence of each basis for judgment on the experts' judgment. Ca = 1 when the basis for judgment has a large influence on the experts, and Ca = 0.5 when the influence of the experts' judgment is at a medium level. When no influence on expert judgment was evident, Ca = 0 [ 25 ] (Table  2 ).

Cs refers to the degree to which the expert was familiar with the question. This study used the Likert scale method to score experts’ familiarity with the question on a scale ranging from 0 to 1 (1 = very familiar, 0.75 = more familiar, 0.5 = moderately familiar, 0.25 = less familiar, 0 = unfamiliar). The familiarity coefficient for each expert (the average familiarity for each indicator) was calculated. The average familiarity coefficient was subsequently computed [ 26 ].

The Cr value of the primary indicator in this study was 0.83, and the Cr value of the secondary indicator was 0.82 (> 0.7); hence, the results of the expert consultation were credible and accurate, as shown in Table  3 .

The degree of expert coordination is an important indicator used to judge the consistency among various experts regarding indicator scores. This study used the Kendall W coordination coefficient test to determine the degree of expert coordination. A higher Kendall W coefficient indicates a greater degree of expert coordination and greater consistency in expert opinion, and P  <  0.05 indicates that the difference is significant [ 26 ]. The results of the three-dimensional harmonization coefficient test for each indicator in the two rounds of the expert consultation questionnaire were valid ( p  <  0.01 ), emphasizing the consistency of the experts' scores. The values of the Kendall W coordination coefficients for both rounds are shown in Table  4 .

Basic information regarding the participants

The 20 Delphi experts who participated in this study were predominantly male (80.0%) rather than female (20.0%). In addition, the participants’ ages were mainly concentrated in the range of 41–50 years old (60.0%). The majority of the experts were doctors by profession (85.0%), and their education and titles were mainly doctoral degree (90.0%) and full senior level (17.0%). The experts also exhibited high academic achievement in their respective fields and had many years of working experience, with the majority having between 21 and 25 years of experience (40.0%) (Table  5 ).

Index screening

The boundary value method was applied to eliminate indicators, leading to the removal of 6 third-level indicators in the first round. One of these, the ability to use statistical software, was associated with a more significant second-level indicator involving data processing, which was kept after expert review. Six indicators were merged into three indicators due to duplication, and 5 third-level indicators were added, resulting in 2 primary indicators, 10 secondary indicators, and 43 third-level indicators.

In the second round of Delphi expert consultation, 5 third-level indicators were deleted, as shown in Additional file 3 , and only one third-level indicator, "Scientific spirit", remained under the secondary indicator "research attitude". The secondary indicator "Research attitude" was combined with "Research ethics" and the third-level indicator "Scientific spirit" was also considered part of "Research ethics". After expert discussion, these were merged into a new secondary indicator "Research attitude" with three third-level indicators: "Research ethics", "Research integrity", and "Scientific spirit". The final index system included two primary indicators, nine secondary indicators, and thirty-eight third-level indicators, as shown in Additional File 3 .

Final index system with weights

The weights of the two primary indexes, research literacy and research ability, were equal. This was determined using the hierarchical analysis method and the percentage weight method based on the results of the second round of Delphi expert consultation (Table  6 ). The primary indicator of research literacy encompasses the fundamental qualities and attitudes medical staff develop over time, including basic qualifications and approach to research. The primary indicator of research ability refers to medical professionals' capacity to conduct scientific research in new areas using suitable methods, as well as their skills needed for successful research using scientific methods.

In this study, the Delphi method was employed, and after two rounds of expert consultation, in accordance with the characteristics and scientific research requirements of medical staff in China, an index system for the scientific literacy of medical staff in China was constructed. The index system for medical staff's scientific literacy in this study consists of 2 first-level indicators, 9 second-level indicators, and 38 third-level indicators. Medical institutions at all levels can use this index system to scientifically assess medical staff's scientific literacy.

In 2014, the Joint Task Force for Clinical Trial Competency (JTF) published its Core Competency Framework [ 27 ]. The Framework focuses more on the capacity to conduct clinical research. These include principles such as clinical research and quality practices for drug clinical trials. However, this framework does not apply to the current evaluation of scientific literacy in hospitals. Because these indicators do not apply to all staff members, there is a lack of practical scientific research, such as information about the final paper output. Therefore, the experts who constructed the index system in this study came from different specialties, and the indicators can be better applied to scientific researchers in all fields. This approach not only addresses clinical researchers but also addresses the concerns of hospital managers, and the indicators are more applicable.

The weighted analysis showed that the primary indicators "research literacy" and "research ability" had the same weight (0.50) and were two important components of scientific literacy. Research ability is a direct reflection of scientific literacy and includes the ability to identify problems, the ability to use literature, professional capacity, subject implementation capacity, data-processing capacity, thesis-writing skills, and research output capacity. Only by mastering these skills can medical staff carry out scientific research activities more efficiently and smoothly. The ability to identify problems refers to the ability of medical staff to obtain insights into the frontiers of their discipline and to identify and ask insightful questions. Ratten claimed that only with keen insight and sufficient sensitivity to major scientific issues can we exploit the opportunities for innovation that may lead to breakthroughs [ 28 ]. Therefore, it is suggested that in the process of cultivating the scientific literacy of medical staff, the ability to identify problems, including divergent thinking, innovative sensitivity, and the ability to produce various solutions, should be improved. Furthermore, this study included three subentries of the secondary indicator "research attitude", namely, research ethics, research integrity, and scientific spirit. This is likely because improper scientific research behavior is still prevalent. A study conducted in the United States and Europe showed that the rate of scientific research misconduct was 2% [ 13 ]. A small survey conducted in Indian medical schools and hospitals revealed that 57% of the respondents knew that someone had modified or fabricated data for publication [ 28 ]. The weight of this index ranked first in the secondary indicators, indicating that scientific attitude is an important condition for improving research quality, relevance, and reliability. Countries and hospitals should develop, implement, and optimize policies and disciplinary measures to combat academic misconduct.

In addition, the third-level indicator "scheduling ability" under the second-level indicator "basic qualification" has a high weight, indicating that medical staff attach importance to management and distribution ability in the context of scientific research. Currently, hospitals face several problems, such as a shortage of medical personnel, excessive workload, and an increase in the number of management-related documents [ 29 , 30 ]. These factors result in time conflicts between daily responsibilities and scientific research tasks, thereby presenting significant obstacles to the allocation of sufficient time for scientific inquiry [ 31 ]. Effectively arranging clinical work and scientific research time is crucial to improving the overall efficiency of scientific research. In the earlier expert interviews, most medical staff believed that scientific research work must be combined with clinical work rather than focused only on scientific research. Having the ability to make overall arrangements is essential to solving these problems. The high weight given to the second-level index of 'subject implementation capacity', along with its associated third-level indicators, highlights the challenges faced by young medical staff in obtaining research subjects. Before implementing a project, researchers must thoroughly investigate, analyze, and compare various aspects of the research project, including its technical, economic, and engineering aspects. Moreover, potential financial and economic benefits, as well as social impacts, need to be predicted to determine the feasibility of the project and develop a research plan [ 32 ]. However, for most young medical staff in medical institutions, executing such a project can be challenging due to their limited scientific research experience [ 33 ]. A researcher who possesses these skills can truly carry out independent scientific research.

The weights of the second-level index "research output capacity" cannot be ignored. In Chinese hospitals, the ability to produce scientific research output plays a certain role in employees’ ability to obtain rewards such as high pay, and this ability is also used as a reference for performance appraisals [ 34 ]. The general scientific research performance evaluation includes the number of projects, scientific papers and monographs, scientific and technological achievements, and patents. In particular, the publication of papers is viewed as an indispensable aspect of performance appraisal by Chinese hospitals [ 35 ]. Specifically, scientific research papers are the carriers of scientific research achievements and academic research and thus constitute an important symbol of the level of medical development exhibited by medical research institutions; they are thus used as recognized and important indicators of scientific research output [ 36 ]. This situation is consistent with the weight evaluation results revealed by this study.

The results of this study are important for the training and management of the scientific research ability of medical personnel. First, the index system focuses not only on external characteristics such as scientific knowledge and skills but also on internal characteristics such as individual traits, motivation, and attitudes. Therefore, when building a research team and selecting and employing researchers, hospital managers can use the index system to comprehensively and systematically evaluate the situation of researchers, which is helpful for optimizing the allocation of a research team, learning from each other's strengths, and strengthening the strength of the research team. Second, this study integrates the content of existing research to obtain useful information through in-depth interviews with medical staff and constructs an evaluation index system based on Delphi expert consultation science, which comprehensively includes the evaluation of the whole process of scientific research activities. These findings can provide a basis for medical institutions to formulate scientific research training programs, help medical personnel master and improve scientific research knowledge and skills, and improve their working ability and quality. Moreover, the effectiveness of the training can also be evaluated according to the system.

In China, with the emergence of STEM rankings, hospitals pay more and more attention to the scientific research performance of medical personnel. Scientific literacy not only covers the abilities of medical personnel engaged in scientific research, but also reflects their professional quality in this field. Having high quality medical personnel often means that they have excellent scientific research ability, and their scientific research performance will naturally rise. In view of this,,medical institutions can define the meaning of third-level indicators and create Likert scales to survey medical staff. Based on the weights assigned to each indicator, comprehensive scores can be calculated to evaluate the level of scientific literacy among medical staff. Through detailed data analysis, they can not only reveal their shortcomings in scientific research ability and quality, but also provide a strong basis for subsequent training and promotion. Through targeted inspection, we can not only promote the comprehensive improvement of the ability of medical staff, but also promote the steady improvement of their scientific research performance, and inject new vitality into the scientific research cause of hospitals.

Limitations

This study has several limitations that need to be considered. First, the participants were only recruited from Beijing (a city in China), potentially lacking geographical diversity. We plan to select more outstanding experts from across the country to participate. Second, the index system may be more suitable for countries with medical systems similar to those of China. When applying this system in other countries, some modifications may be necessary based on the local context. Last, While this study has employed scientific methods to establish the indicator system, the index system has yet to be implemented on a large sample of medical staff. Therefore, the reliability and validity of the index system must be confirmed through further research. In conclusion, it is crucial to conduct further detailed exploration of the effectiveness and practical application of the index system in the future.

This study developed an evaluation index system using the Delphi method to assess the scientific literacy of medical staff in China. The system comprises two primary indicators, nine secondary indicators, and thirty-eight third-level indicators, with each index assigned a specific weight. The index system emphasizes the importance of both attitudes and abilities in the scientific research process for medical staff and incorporates more comprehensive evaluation indicators. In the current era of medical innovation, enhancing the scientific literacy of medical staff is crucial for enhancing the competitiveness of individuals, hospitals, and overall medical services in society. This evaluation index system is universally applicable and beneficial for countries with healthcare systems similar to those of China. This study can serve as a valuable reference for cultivating highly qualified and capable research personnel and enhancing the competitiveness of medical research.

Availability of data and materials

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Coloma J, Harris E. From construction workers to architects: developing scientific research capacity in low-income countries. PLoS Biol. 2009;7(7):e1000156. https://doi.org/10.1371/journal.pbio.1000156 .

Article   Google Scholar  

Brauer SG, Haines TP, Bew PG. Fostering clinician-led research. Aust J Physiother. 2007;53(3):143–4. https://doi.org/10.1016/s0004-9514(07)70020-x .

The L. China’s research renaissance. Lancet. 2019;393(10179):1385. https://doi.org/10.1016/S0140-6736(19)30797-4 .

Hannay DR. Evaluation of a primary care research network in rural Scotland. Prim Health Care ResDevelop. 2006;7(3):194–200. https://doi.org/10.1191/1463423606pc296oa .

Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376:1923–58.

Xie Y, Wang J, Li S, Zheng Y. Research on the Influence Path of Metacognitive Reading Strategies on Scientific Literacy. J Intell. 2023;11(5):78. https://doi.org/10.3390/jintelligence11050078 . PMID: 37233327; PMCID: PMC10218841.

Pang YH, Cheng JL. Revise of scientific research ability self-evaluation rating scales of nursing staff. Chin Nurs Res. 2011;13:1205–8. https://doi.org/10.3969/j.issn.1009-6493.2011.13.040 .

Zhang J, Jianshan MAO, Gu Y. On the cultivation of scientific research literacy of medical graduate students. Continu Med Educ China. 2023;15(3):179–82. https://doi.org/10.3969/j.issn.1674-9308.2023.03.043 .

Rillero P. Process skills and content knowledge. Sci Act. 1998;3:3–4.

Google Scholar  

Liu RS. Study on reliability and validity of self rating scale for scientific research ability of nursing staff. Chinese J Pract Nurs. 2004;9:8–10. https://doi.org/10.3760/cma.j.issn.1672-7088.2004.09.005 .

Moule P, Goodman M. Nursing research: An introduction. London, UK: Sage; 2013.

Xue J, Chen X, Zhang Z, et al. Survey on status quo and development needs of research and innovation capabilities of young researchers at university-affiliated hospitals in China: a cross-sectional survey. Ann Transl Med. 2022;10(18):964. https://doi.org/10.21037/atm-22-3692 .

Fanelli D, Costas R, Fang FC, et al. Testing hypotheses on risk factors for scientific misconduct via matched-control analysis of papers containing problematic image duplications. Sci Eng Ethics. 2019;25(3):771–89. https://doi.org/10.1007/s11948-018-0023-7 .

Liao Y, Zhou H, Wang F, et al. The Impact of Undergraduate Tutor System in Chinese 8-Year Medical Students in Scientific Research. Front Med (Lausanne). 2022;9:854132. https://doi.org/10.3389/fmed.2022.854132 .

Mohan L, Singh Y, Kathrotia R, et al. Scientific literacy and the medical student: A cross-sectional study. Natl Med J India. 2020;33(1):35–7. https://doi.org/10.4103/0970-258X.308242 .

Jorm AF. Using the Delphi expert consensus method in mental health research. Aust N Z J Psychiatry. 2015;49(10):887–97. https://doi.org/10.1177/0004867415600891 .

Xinran S, Heping W, Yule H, et al. Defining the scope and weights of services of a family doctor service project for the functional community using Delphi technique and analytic hierarchy process. Chinese Gen Pract. 2021;24(34):4386–91.

Park S, Kim HK, Lee M. An analytic hierarchy process analysis for reinforcing doctor-patient communication. BMC Prim Care. 2023;24(1):24. https://doi.org/10.1186/s12875-023-01972-3 . Published 2023 Jan 21.

Zhou MLY, Yin H, et al. New screening tool for neonatal nutritional risk in China: a validation study. BMJ Open. 2021;11(4):e042467. https://doi.org/10.1136/bmjopen-2020-042467 .

Wang K, Wang Z, Deng J, et al. Study on the evaluation of emergency management capacity of resilient communities by the AHP-TOPSIS method. Int J Environ Res Public Health. 2022;19(23):16201. https://doi.org/10.3390/ijerph192316201 .

Yuwei Z, Chuanhui Y, Junlong Z, et al. Application of analytic Hierarchy Process and percentage weight method to determine the weight of traditional Chinese medicine appropriate technology evaluation index system. Chin J Tradit Chinese Med. 2017;32(07):3054–6.

Babbie E. The practice of social research. 10th Chinese language edition. Huaxia Publisher, 2005: 253–4.

Liu W, Hu M, Chen W. Identifying the Service Capability of Long-Term Care Facilities in China: an e-Delphi study. Front Public Health. 2022;10:884514. https://doi.org/10.3389/fpubh.2022.884514 .

Zeng G. Modern epidemiological methods and application. Pecking Union Medical College Union Press, 1996.

Geng Y, Zhao L, Wang Y, et al. Competency model for dentists in China: Results of a Delphi study. PLoS One. 2018;13(3):e0194411. https://doi.org/10.1371/journal.pone.0194411 .

Cong C, Liu Y, Wang R. Kendall coordination coefficient W test and its SPSS implementation. Journal of Taishan Medical College. 2010;31(7):487–490. https://doi.org/10.3969/j.issn.1004-7115.2010.07.002 .

Sonstein S, Seltzer J, Li R, et al. Moving from compliance to competency: a harmonized core competency framework for the clinical research professional. Clin Res. 2014;28(3):17–23.

Madan C, Kruger E, Tennant M. 30 Years of dental research in Australia and India: a comparative analysis of published peer review literature. Indian J Dent Res. 2012;23(2):293–4. https://doi.org/10.4103/0970-9290.100447 .

Siemens DR, Punnen S, Wong J, Kanji N. A survey on the attitudes towards research in medical school. BMC Med Educ. 2010;10:4. https://doi.org/10.1186/1472-6920-10-4 .

Solomon SS, Tom SC, Pichert J, Wasserman D, Powers AC. Impact of medical student research in the development of physician-scientists. J Investig Med. 2003;51(3):149–56. https://doi.org/10.1136/jim-51-03-17 .

Misztal-Okonska P, Goniewicz K, Hertelendy AJ, et al. How Medical Studies in Poland Prepare Future Healthcare Managers for Crises and Disasters: Results of a Pilot Study. Healthcare (Basel). 2020;8(3):202. https://doi.org/10.3390/healthcare8030202 .

Xu G. On the declaration of educational scientific research topics. Journal of Henan Radio & TV University. 2013;26(01):98–101.

Ju Y, Zhao X. Top three hospitals clinical nurse scientific research ability present situation and influence factors analysis. J Health Vocational Educ. 2022;40(17):125–8.

Zhu Q, Li T, Li X, et al. Industry gain public hospital medical staff performance distribution mode of integration, exploring. J Health Econ Res. 2022;33(11):6-82–6.

Sun YLL. Analysis of hospital papers published based on performance appraisal. China Contemp Med. 2015;22(31):161–3.

Jian Y, Wu J, Liu Y. Citation analysis of seven tertiary hospitals in Yunnan province from 2008 to 2012. Yunnan Medicine. 2014;(6):700–704.

Download references

Acknowledgements

The authors thank all who participated in the nominal group technique and two rounds of the Delphi study.

This study was supported by the National Natural Science Foundation of China (72074160) and the Natural Science Foundation Project of Beijing (9222004).

Author information

Shuyu Liang and Ziyan Zhai contributed equally to this work and joint first authors.

Kai Meng and Yuan Gao contributed equally to this work and share corresponding author.

Authors and Affiliations

Aerospace Center Hospital, No. 15 Yuquan Road, Haidian District, Beijing, 100049, China

Xiaozhi Sun, Jingxuan Jiao & Yuan Gao

School of Public Health, Capital Medical University, No.10 Xitoutiao, Youanmenwai Street, Fengtai District, Beijing, 100069, China

Shuyu Liang, Ziyan Zhai, Xingmiao Feng & Kai Meng

Beijing Tiantan Hospital, Capital Medical University, No. 119 South Fourth Ring West Road, Fengtai District, Beijing, 100070, China

You can also search for this author in PubMed   Google Scholar

Contributions

S.L. and Z.Z. contributed equally to this paper. S.L. took charge of the nominal group technique, data analysis, writing the first draft and revising the manuscript; Z.Z. was responsible for the Delphi survey, data analysis, and writing of the first draft of the manuscript; XF was responsible for the rigorous revision of Delphi methods; X.S. and J.J. were responsible for the questionnaire survey and data collection; Y.G. contributed to the questionnaire survey, organization of the nominal group interview, supervision, project administration and resources; and K.M. contributed to conceptualization, methodology, writing—review; editing, supervision, and project administration. All the authors read and approved the final manuscript.

Corresponding authors

Correspondence to Yuan Gao or Kai Meng .

Ethics declarations

Ethics approval and consent to participate.

This study involved human participants and was approved by the Ethical Review Committee of the Capital Medical University (No. Z2022SY089). Participation in the survey was completely voluntary, and written informed consent was obtained from the participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Liang, S., Zhai, Z., Feng, X. et al. Development of an index system for the scientific literacy of medical staff: a modified Delphi study in China. BMC Med Educ 24 , 397 (2024). https://doi.org/10.1186/s12909-024-05350-0

Download citation

Received : 25 October 2023

Accepted : 26 March 2024

Published : 10 April 2024

DOI : https://doi.org/10.1186/s12909-024-05350-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical staff
• Scientific literacy
• Evaluation indicators

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 10 January 2024

A scoping review of theories, models and frameworks used or proposed to evaluate knowledge mobilization strategies

• Saliha Ziam   ORCID: orcid.org/0000-0002-8892-9572 1 ,
• Sèverine Lanoue 2 ,
• Esther McSween-Cadieux 2 ,
• Mathieu-Joël Gervais 3 ,
• Julie Lane 2 , 4 ,
• Dina Gaid 5 ,
• Laura Justine Chouinard 1 ,
• Christian Dagenais 6 ,
• Valéry Ridde 7 , 8 ,
• Emmanuelle Jean 9 ,
• France Charles Fleury 10 ,
• Quan Nha Hong 5 &
• Ollivier Prigent 2  

Health Research Policy and Systems volume  22 , Article number:  8 ( 2024 ) Cite this article

2374 Accesses

6 Altmetric

Metrics details

Evaluating knowledge mobilization strategies (KMb) presents challenges for organizations seeking to understand their impact to improve KMb effectiveness. Moreover, the large number of theories, models, and frameworks (TMFs) available can be confusing for users. Therefore, the purpose of this scoping review was to identify and describe the characteristics of TMFs that have been used or proposed in the literature to evaluate KMb strategies.

A scoping review methodology was used. Articles were identified through searches in electronic databases, previous reviews and reference lists of included articles. Titles, abstracts and full texts were screened in duplicate. Data were charted using a piloted data charting form. Data extracted included study characteristics, KMb characteristics, and TMFs used or proposed for KMb evaluation. An adapted version of Nilsen (Implement Sci 10:53, 2015) taxonomy and the Expert Recommendations for Implementing Change (ERIC) taxonomy (Powell et al. in Implement Sci 10:21, 2015) guided data synthesis.

Of the 4763 search results, 505 were retrieved, and 88 articles were eligible for review. These consisted of 40 theoretical articles (45.5%), 44 empirical studies (50.0%) and four protocols (4.5%). The majority were published after 2010 ( n  = 70, 79.5%) and were health related ( n  = 71, 80.7%). Half of the studied KMb strategies were implemented in only four countries: Canada, Australia, the United States and the United Kingdom ( n  = 42, 47.7%). One-third used existing TMFs ( n  = 28, 31.8%). According to the adapted Nilsen taxonomy, process models ( n  = 34, 38.6%) and evaluation frameworks ( n  = 28, 31.8%) were the two most frequent types of TMFs used or proposed to evaluate KMb. According to the ERIC taxonomy, activities to “train and educate stakeholders” ( n  = 46, 52.3%) were the most common, followed by activities to “develop stakeholder interrelationships” ( n  = 23, 26.1%). Analysis of the TMFs identified revealed relevant factors of interest for the evaluation of KMb strategies, classified into four dimensions: context, process, effects and impacts.

Conclusions

This scoping review provides an overview of the many KMb TMFs used or proposed. The results provide insight into potential dimensions and components to be considered when assessing KMb strategies.

Peer Review reports

Contribution to the literature

The evaluation of KMb strategies is a critical dimension of the KMb process that is still poorly documented and warrants researchers’ attention.

Our review identified the most common theories, models and frameworks (TMFs) proposed or used to assess KMb strategies and the main components to consider when evaluating a KMb strategy.

By developing an integrative reference framework, this work contributes to improving organizations’ capacity to evaluate their KMb initiatives.

It is widely recognized that research evidence has the potential to inform, guide, and improve practices, decisions, and policies [ 1 ]. Unfortunately, for diverse reasons, the best available evidence is still too seldom taken into account and used [ 2 , 3 , 4 , 5 , 6 , 7 ]. The field of research on knowledge mobilization (KMb) has been growing rapidly since the early 2000s [ 2 , 3 , 8 , 9 , 10 , 11 ]. Its purpose is to better understand how to effectively promote and support evidence use.

Knowledge mobilization is one of many terms and concepts developed over recent decades to describe processes, strategies, and actions to bridge the gap between research and practice. Other common terms often paired interchangeably with the term “knowledge” are “translation”, “transfer”, “exchange”, “sharing” and “dissemination”, among others. [ 12 , 13 ]. Some are more closely linked than others to specific fields or jurisdictions. For this study, we adopted the term knowledge mobilization (KMb) because it conveys the notions of complexity and multidirectional exchanges that characterize research-to-action processes. We used it as an umbrella concept that encompasses the efforts made to translate knowledge into concrete actions and beneficial impacts on populations [ 1 ]. Moreover, the term KMb is also used by research funding agencies in Canada to emphasize the medium- and long-term effects that research knowledge or research results can have on potential users [ 1 , 14 ].

KMb represents all processes from knowledge creation to action and includes all strategies implemented to facilitate these processes [ 14 ]. A KMb strategy is understood as a coordinated set of activities to support evidence use, such as dissemination activities to reach target audiences (for example, educational materials, practical guides, decision support tools) or activities to facilitate knowledge application in a specific context and support professional behaviour change (for example, community of practice, educational meetings, audits and feedback, reminders, deliberative dialogues) [ 15 ]. A KMb process may vary in intensity, complexity or actor engagement depending on the nature of the research knowledge and the needs and preferences of evidence users [ 7 ].

KMb is considered a complex process, in that numerous factors can facilitate or hinder its implementation and subsequent evidence use. The past two decades have seen the emergence of a deeper understanding of these factors [ 2 , 3 , 16 ]. These may be related to the knowledge mobilized (for example, relevance, reliability, clarity, costs), the individuals involved in the KMb process (for example, openness to change, values, time available, resources), the KMB strategies (for example, fit with stakeholder needs and preferences, regular interactions, trust relationships, timing), and organizational and political contexts (for example, culture of evidence use, leadership, resources) [ 2 , 6 , 17 , 18 ]. However, more studies are needed to understand which factors are more important in which contexts, and to evaluate the effects of KMb strategies.

On this last point, while essential, it is often very complex to study KMb impacts empirically to demonstrate the effectiveness of KMb strategies [ 19 , 20 , 21 ]. Partly for this reason, high-quality studies that evaluate process, mechanisms and effects of KMb strategies are still relatively rare [ 2 , 22 , 23 , 24 , 25 ]. As a result, knowledge about the effectiveness of different KMb strategies remains limited [ 10 , 17 , 19 , 23 , 26 , 27 , 28 ] and their development cannot be totally evidence informed [ 3 , 19 , 20 , 23 , 29 , 30 ], which may seem incompatible with the core values and principles of KMb.

The growing interest in KMb has led to an impressive proliferation of conceptual propositions, such as theories, models and frameworks (TMF) [ 2 , 3 , 9 , 11 , 12 , 31 , 32 ]. Many deplore the fact that these are poorly used [ 11 , 30 , 33 ] and insist on the need to test, refine and integrate existing ones [ 3 , 31 , 34 ]. Indeed, the conceptual and theoretical development of the field has outpaced its empirical development. This proliferation appears to have created confusion among certain users, such as organizations that need to evaluate their KMb strategies. Besides implementing and funding KMb strategies, knowledge organizations such as granting agencies, governments and public organizations, universities and health authorities are often required to demonstrate the impact of their strategies [ 21 , 35 , 36 ]. Yet this can be a significant challenge [ 20 , 23 , 29 ]. They may have difficulty knowing which TMFs to choose, in what context and how to use them effectively in their evaluation process [ 12 , 37 ].

Indeed, the evaluation of KMb strategies is still relatively poorly documented, with respect to the phases of their development and implementation. Our aim in this scoping review is to clarify, conceptually and methodologically, this crucial dimension of the KMb process. This would help organizations gain access to evidence-based, operational and easy-to-use evaluation toolkits for assessing the impacts of their KMb strategies.

To survey the available knowledge on evaluation practices for KMb strategies, we conducted a scoping review. According to Munn et al. [ 38 ], a scoping review is indicated to identify the types of available evidence and knowledge gaps, to clarify concepts in the literature and to identify key characteristics or factors related to a concept. This review methodology also allows for the inclusion of a diversity of publications, regardless of their nature or research design, to produce the most comprehensive evidence mapping possible [ 39 ]. The objective of the scoping review was to identify and describe the characteristics of theories, models and frameworks (TMFs) used or proposed to evaluate KMb strategies. The specific research questions were:

What TMFs to evaluate KMb strategies exist in the literature?

What KMb strategies do they evaluate (that is types of KMb objectives, activities, target audiences)?

What dimensions and components are included in these TMFs?

This scoping review was conducted based on the five steps outlined by Arksey and O’Malley [ 39 ]: (1) formulating the research questions; (2) identifying relevant studies; (3) selecting relevant studies; (4) extracting and charting data; and (5) analysing, collating, summarizing and presenting the data. Throughout the process, researchers and knowledge users (KMb practitioners) were involved in decisions regarding the research question, search strategy, selection criteria for studies and categories for data charting. We followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines [ 40 ]. No protocol was registered for this review.

Search strategy and information sources

The search strategy was developed, piloted and refined in consultation with our team’s librarian. Search terms included controlled vocabulary and keywords related to three main concepts: (1) knowledge mobilization (for example [knowledge or evidence or research] and transfer, translation, diffusion, dissemination, mobilization, implementation science, exchange, sharing, use, uptake, evidence-based practice, research-based evidence), (2) evaluation (for example, evaluat*, measur*, impact, outcome, assess, apprais*, indicator) and (3) TMF (for example, framework*, model*, method*, guide*, theor*). See Additional file 1 for the search terms and strategies used in the electronic searches.

The following databases were searched from January 2000 to August 2023: MEDLINE (Ovid), PsycInfo (Ovid), ERIC (ProQuest), Sociological Abstracts (ProQuest), Dissertations & Theses (Proquest), Érudit and Cairn. These databases were chosen to identify relevant references in the health, education and social fields. Several search strategies were tested by the librarian to optimize the retrieval of citations known to the investigators and to increase the likelihood that all relevant studies would be retrieved. We also searched reference lists of included articles and previous systematic reviews [ 11 , 12 , 15 , 41 ].

Eligibility criteria

A publication was considered eligible if it (1) presented or used a theory, model, or framework (TMF), (2) described dimensions or specific components to consider in the evaluation of KMb strategies, (3) presented or discussed KMb strategies or activities (any initiatives to improve evidence use), and (4) proposed outcomes that might result directly or indirectly from the KMb strategies. Studies were excluded from analysis if they (1) presented a TMF to assess the impact of research without mentioning KMb strategies or an intervention not related to KMb and (2) presented evaluation dimensions or components that could not be generalized. We considered publications in English or French. All types of articles and study designs were eligible, including study protocols.

Study selection

The results of the literature search were imported into Covidence, which the review team used for screening. After duplicate articles were removed, the titles and abstracts were screened independently by two of the three reviewers (EMC, MJG, GL). Publications identified as potentially relevant were retrieved in full text and screened independently by three reviewers (EMC, MJG, GL). Discrepancies regarding the inclusion of any publication were resolved through discussion and consensus among reviewers. The principal investigator (SZ) validated the final selection of articles.

Data synthesis

A data charting form was developed in Microsoft Excel and piloted by the research team. Data extracted included study characteristics (authors, authors’ country of affiliation, year, journal, discipline, article type, study setting, study aim), KMb strategies of interest, KMb objectives, KMb target audiences and TMFs used or proposed for KMb evaluation (existing or new TMF, specific dimensions or components of TMF and so on). Data were extracted by a single reviewer (SL, JC or OP) and validated by a second reviewer (SZ). Disagreements were discussed between reviewers and resolved by consensus. No quality appraisal of included studies was conducted, as this is optional in scoping reviews and the purpose was only to describe the content of identified TMFs [ 42 ].

Data analysis and presentation of results

Data were summarized according to study characteristics, KMb strategy characteristics (activities, objectives, target audiences), types of TMFs, and dimensions or components to consider for KMb evaluation. Disagreements during the process were discussed and resolved through consensus (SL, DG, SZ). A KMb strategy might have one or more objectives and include one or more activities. Thus, the objectives and activities of the KMb strategies extracted from the selected studies were summarized based on existing categorizations. The categorization of KMb objectives was inspired by Gervais et al. [ 15 ] and Farkas et al. [ 43 ] (Table  1 ).

The KMb activities were categorized according to the Expert Recommendations for Implementing Change (ERIC) taxonomy [ 44 ]. The activities were first classified according to the full taxonomy and then grouped into the nine categories proposed by Waltz et al. [ 45 ] (Table  2 ).

The TMFs were categorized according to the categories of theoretical approaches described by Nilsen [ 32 ]: process models, evaluation frameworks, determinant frameworks and classic theories (Table  3 ). The category “implementation theories” originally described by Nilsen [ 32 ] was not used because we did not identify any article that fit this category. We also added a category named “logic models” due to the nature of the identified TMFs. Logic models are often used in theory-driven evaluation approaches and are usually developed to show the links among inputs (resources), activities and outputs (outcomes and short-, medium- and long-term effects) [ 46 ].

Finally, the content extracted from the TMFs was analysed using mainly an inductive method. This method allows, among other things, to develop a reference framework or a model from the emerging categories that are evident in the text data [ 50 ].

The classification of concepts is the result of multiple readings and interpretations. The concepts associated with each dimension of the framework were classified according to their meaning. Similar concepts were grouped together to form components. These grouped components were then associated with the subdimensions and main dimensions of the framework.

Search results

The searches yielded 4763 articles. Of those, 4258 were excluded during the title and abstract screening. Of the 505 full-text articles, we retained 88 in our final sample. The results of the search and selection processes (PRISMA flowchart) are summarized in Fig.  1 .

PRISMA flowchart summarizing search strategy and selection results [ 40 ]

Publication characteristics

Most articles were published after 2010 ( n  = 70, 79.5%), with an average of 5 articles per year between 2010 and 2023 compared with an average of 2.1 articles per year between 2001 and 2009; there were no eligible articles from 2000. The search was conducted in August 2023, and only five articles were published in these 7 months of the year. Table 4 presents the main characteristics of the selected articles. A full list of the included articles with their main characteristics is presented in Additional file 2 .

The number of theoretical and empirical articles was relatively similar. Among the theoretical articles, 19 descriptive articles (21.6%) were aimed at describing a KMb strategy, a KMb infrastructure or a TMF related to a specific programme or context; 18 articles (20.5%) synthesized knowledge to propose a TMF (new or revised); and three articles conducted systematic reviews (3.4%).

The empirical articles category included studies with different methodological approaches (quantitative, qualitative, mixed methods). We will not report the details of the methodologies used, as this would result in a long list with few occurrences. The empirical articles can be divided into three categories: (1) studies that evaluated a TMF related to KMb ( n  = 16, 18.2%), (2) studies that evaluated a KMb strategy ( n  = 21, 23.9%) and (3) studies that evaluated both a KMb strategy and a TMF ( n  = 7, 8.0%).

Most articles were related to healthcare ( n  = 71, 80.7%). This field of study was divided into three subdomains. The healthcare and social services articles usually described or assessed a KMb strategy targeting health professionals’ practices in a variety of fields (for example, occupational therapy, dentistry, mental health, pharmacology, gerontology, nursing and so on). The health policy and systems articles usually described or assessed KMb strategies targeting decision-making processes, decision-makers or public health interventions and policies. The continuing education articles assessed training programmes for health professionals aimed at increasing knowledge and skills in a specific field. The articles in the general field described or discussed TMFs and KMb strategies that could be applied to multiple disciplines or contexts. Finally, the articles in the education field described or assessed a KMb strategy targeting education professionals.

Almost half of the articles ( n  = 42, 47.7%) studied KMB strategies implemented in only four countries: Canada, Australia, the United States and the United Kingdom. Countries in South America, the Caribbean, Africa, Asia, the Middle East, China and Europe were underrepresented ( n  = 8, 9.1%). The remaining 34 articles (38.6%) did not specify an implementation context and were mostly theoretical articles. Regarding the authors’ countries of affiliation, Canada, the United States, Australia and the United Kingdom were again the most represented countries, featuring in 85% of the articles ( n  = 75).

What theories, models or frameworks exist in the literature to evaluate KMb strategies?

Several articles proposed a new TMF ( n  = 37, 42.0%), and some articles proposed a logic model specifically developed to evaluate their KMb strategy ( n  = 17, 19.3%). One-third of the articles used existing TMFs ( n  = 28, 31.8%). A few articles only referred to existing TMFs but did not use them to guide a KMb strategy evaluation ( n  = 6, 8.5%).

The identified TMFs were then categorized according to their theoretical approaches (adapted from Nilsen, [ 32 ]) (Table  5 ). Five articles used or proposed more than one TMF, and three TMFs could be classified in two categories. Several articles proposed or used a process model ( n  = 34, 38.6%) or an evaluation framework ( n  = 28, 31.8%); these were the two most frequently identified types of TMFs. Fewer articles proposed or used a logic model ( n  = 17, 19.3%), a determinant framework ( n  = 12, 13.6%) or a classic theory ( n  = 7, 8.0%). The TMFs most often identified in the articles were the RE-AIM framework ( n  = 5, 5.7%), the Knowledge-to-Action framework [ 9 ] ( n  = 4, 4.5%), the Theory of Planned Behavior [ 51 ] ( n  = 3, 3.4%) and the Expanded Outcomes framework for planning and assessing continuing medical education [ 52 ] ( n  = 3, 3.4%). In total, we identified 87 different TMFs in the 88 articles. Only nine TMFS were retrieved in more than one article.

What KMb strategies do the TMFs evaluate (activities, objectives, target audience)?

Thirty-eight articles reported using more than one activity in their KMb strategy. According to the ERIC compilation, “Train and educate stakeholders” activities were the most common, followed by “Develop stakeholder interrelationships” and “Use evaluative and iterative strategies”. Table 6 presents the various types of activities and the number of articles that referred to each.

Of the 88 articles analysed, 18 (20.4%) did not specify a KMb objective. The remaining articles proposed one or more KMb strategy objectives. Specifically, 39 (36.4%) articles had one objective, 15 (17.0%) had two, three (3.4%) had three, and 13 (14.8%) had four or five. Table 7 presents the different types of objectives and the number of times they were identified.

The target audiences for KMb strategies were clearly specified in half of the articles ( n  = 44, 50.0%). Generally, these were empirical articles that targeted specific professionals ( n  = 36, 40.9%) or decision-makers ( n  = 8, 9.1%). Just under one-third of the articles identified a broad target audience (for example, professionals and managers in the health system, a health organization) ( n  = 26, 29.5%). Finally, 18 articles (20.4%) did not specify a target audience for KMb; these were most often theoretical articles.

What are the dimensions and components included in TMFs for evaluating KMb strategies?

The analysis of the identified TMFs revealed many factors of interest relevant for the evaluation of KMb strategies. These specific components were inductively classified into four main dimensions: context, process, effects and impacts (Fig.  2 ). The context dimension refers to the assessment of the conditions in place when the KMb strategy is implemented. These include both the external (that is, sociopolitical, economic, environmental and cultural characteristics) and internal environments (that is, characteristics of organizations, individuals and stakeholder partnerships). These factors are understood to influence the selection and tailoring of a KMb strategy. The process dimension refers to the assessment of the planning, levels and mechanisms of implementation, as well as to the characteristics of the KMb strategy implemented. The effects dimension refers to the assessment of outcomes following the KMb strategy implementation. The potential effects vary depending on the strategy’s objectives and can be either the immediate results of the KMb strategy or short-, medium- and long-term outcomes. The conceptual gradation of effects was generally represented in a similar way in the TMFs analysed, but the temporality of effects could vary. A medium-term outcome in one study could be understood as a long-term outcome in another. However, the majority of authors group these effects into three categories (Gervais et al. 2016: p. 6): (1) short-term effects, measured by success of KMb strategy measured by success of KMb strategy (number of people reached, satisfaction, participation and so on); (2) medium-term effects linked to changes in individual attitude and the use of knowledge; and (3) the long-term effects that result from achieving the KMb objective (for example, improved practices and services, changed collective behaviour, sustainable use of knowledge).

The main evaluation dimensions that emerged from the TMFs analysed

Finally, the impacts dimension refers to the ultimate effects of KMb products or interventions on end users, as measured by the organization (Phipps et al. [ 36 ], p. 34). The evaluation of these ultimate effects can be measured by the integration of a promising practice into organizational routines, by the effects on service users or by the effects on the health and well-being of communities and society in general.

This gradation shows the importance of measuring effects at different points in time, to take account of the time they take to appear and their evolving nature (Gervais et al., 2016: p. 6).

Most of the articles presented the dimensions that should be evaluated, whereas the empirical articles presented the dimensions but also used them in practice to evaluate a KMb strategy. Only five articles (5.7%) did not mention specific dimensions that could be classified.

Table 8 presents both the number of articles that presented dimensions to be evaluated and the number of articles that evaluated them in practice. These results showed that the effects dimension was both the most often named and the most evaluated in practice. The other three dimensions (context, process, impacts), while quite often mentioned as relevant to assess, were less often evaluated in practice. For example, only five articles (5.7%) reported having assessed the impacts dimension.

As previously mentioned, the components relevant for the evaluation of KMb strategies were extracted from the identified TMFs. Table 9 presents these components, which represent the more specific factors of interest for assessing context, process, effects and impacts.

Although often overlooked, the evaluation of KMb strategies is an essential step in guiding organizations seeking to determine whether the expected outcomes of their initiatives are being realized. Evaluation not only allows organizations to make adjustments if the initiatives are not producing the expected results, but also helps them to justify their funding of such initiatives. Evaluation is also essential if the KMb science is to truly inform KMb practice, such that the strategies developed are based on empirical data [ 30 ]. To make KMb evaluation more feasible, evaluation must be promoted and practices improved.

This scoping review meets the first objective of our project, which was to provide an overview of reference frameworks used or proposed for evaluating KM strategies, and to propose a preliminary version of a reference framework for evaluating KM strategies. Several key findings emerged from this scoping review:

Proliferation of theories, models and frameworks, but few frequently used

We are seeing a proliferation of TMFs in KMb and closely related fields [ 132 , 133 ]. Thus, the results of this scoping review support the argument that the conceptual and theoretical development of the field is outpacing its empirical development. Most of the reviewed articles (42.0%) proposed a new TMF rather than using existing ones. Furthermore, we identified relatively few empirical studies (50.0%) that focused on the evaluation of KMb strategies. Consequently, the TMFs used were poorly consolidated, which does not provide a solid empirical foundation to guide the evaluation of KMb strategies. Also, not all the TMFs proposed in the articles were specifically developed for evaluation; some were focused on KMb implementation processes. These may still provide elements to consider for evaluation, although they were not designed to propose specific indicators.

A scoping review published in 2018 identified 596 studies using 159 different KMb TMFs, 95 of which had been used only once [ 11 ]. Many authors reported that these are rarely reused and validated [ 11 , 30 , 33 ] and that it is important to test, refine and integrate existing ones [ 3 , 31 , 34 , 133 ]. A clear, collective and consistent use of existing TMFs is recommended and necessary to advance KMb science and closely related fields [ 12 , 31 ]. The systematic review by Strifler et al. [ 11 ] highlights the diversity of available TMFs and the difficulty users may experience when choosing TMFs to guide their KMb initiatives or evaluation process. Future work should focus on the development of tools to better support users of TMFs, especially those working in organizations. By consolidating a large number of TMFs, the results of this scoping review contribute to these efforts.

The importance of improving evaluation practices for complex multifaceted KMb strategies

Another noteworthy finding was the emphasis on the evaluation of strategies focused on education and professional training for practice improvement (52.3%). Relatively few of the reviewed articles looked at, for example, the evaluation of KMb strategies aimed at informing or influencing decision-making (13.6%), or KMb strategies targeting decision-makers (9.1%). These results reaffirm the importance of conducting more large-scale evaluations of complex and multifaceted KMb strategies. These involve a greater degree of interaction and engagement, are composed of networks of multiple actors, mobilize diverse sources of knowledge and have simultaneous multilevel objectives [ 19 , 134 ].

The fact that some KMb strategies are complex interventions implemented in complex contexts [ 134 ] presents a significant and recurring challenge to their evaluation. Methodological designs, approaches and tools are often ill-suited to capture the short-, medium- and long-term outcomes of KMb strategies, as well as to identify the mechanisms by which these outcomes were produced in a specific context. It is also difficult to link concrete changes in practice and decision-making to tangible longer-term impacts at the population level. Moreover, these impacts can take years to be achieved [ 36 ] and can be influenced by several other factors in addition to KMb efforts [ 2 , 19 , 24 ]. Comprehensive, dynamic and flexible evaluation approaches [ 135 , 136 , 137 ] using mixed methods [ 20 ] appear necessary to understand why, for whom, how, when and in what context KMb strategies achieve their objectives [ 2 , 21 , 25 ]. For instance, realist evaluation, which belongs to theory-based evaluation, may be an approach that addresses issues of causality without sacrificing complexity [ 134 , 138 , 139 ]. This evaluation approach aims to identify the underlying generative mechanisms that can explain how the outcomes were generated and what characteristics of the context affected, or not, those mechanisms. This approach is used to test and refine theory about how interventions with a similar logic of action actually work [ 139 ].

Large heterogeneity of methodologies used in empirical studies

Despite the growth of the KMb field, a recurring issue is the relatively limited number of high-quality studies that evaluate KMb outcomes and impacts. This observation is shared by many of the authors of our scoping articles [ 2 , 22 , 23 , 24 , 25 ]. Only a limited number of empirical articles met the selection criteria ( n  = 44/88) in this scoping review. Synthesizing these studies is challenging due to the diversity of research designs used and the large number of potential evaluation components identified. In addition, most of the identified studies used TMFs and measurement tools that were not validated [ 20 , 29 ] and that were specifically developed for their study [ 16 , 25 , 140 ]. Moreover, these studies did not describe the methods used to justify their choice of evaluation dimensions and components [ 25 ], which greatly hinders the ability to draw inferences and develop generalizable theories through replication in similar studies [ 110 , 140 , 141 , 142 , 143 ]. The lack of a widely used evaluation approach across the field is therefore an important issue [ 16 , 20 ] also highlighted by this scoping review.

Our aim in this review was not to identify specific indicators or measurement tools (for example, questionnaires) for assessing KMb strategies, but rather to describe dimensions and component of TMFs used for KMb evaluation. However, a recent scoping review [ 144 ] looked at measurement tools and revealed that only two general potential tools have been identified to assess KMb activities in any sector or organization: the Level of Knowledge Use Survey (LOKUS) [ 145 ] and the Knowledge Uptake and Utilization Tool (KUUT) [ 95 ]. The authors also assert the importance of developing standardized tools and evaluation processes to facilitate comparison of KMb activities’ outcomes across organizations [ 144 ].

Lack of description and reporting of KMb strategies and evaluation

Another important finding from this review was the sparsity of descriptions of KMb strategies in the published articles. In general, the authors provided little information on the operationalization of their KMb strategies (for example, objectives, target audiences, details of activities implemented, implementation context, expected effects). The KMb strategy objectives and the implemented activities should be carefully selected and empirically, theoretically or pragmatically justified before the evaluation components and specific indicators can be determined [ 146 ].

To improve consistency in the field and to contribute to the development of KMb science, many authors reported the need to better describe and report KMb strategies and their context [ 8 , 54 , 146 , 147 , 148 , 149 , 150 ]. KMb strategies are often inconsistently labelled across studies, poorly described and rarely justified theoretically [ 146 , 150 , 151 ]. It was not possible in this scoping review to associate the evaluation components to be used with the objectives and types of KMb strategies, as too much information was missing in the articles. Over the past 10 years, several guidelines have been proposed to improve the reporting of interventions such as KMb strategies: the “Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations checklist” [ 147 ], the “Standards for Reporting Implementation Studies (StaRI)” [ 150 ] and the “Template for Intervention Description and Replication (TIDieR)” [ 152 ]. These guidelines should be used more often to enhance the reporting of KMb strategies and help advance the field [ 153 ].

Implications for future research

This scoping review provides an overview of potential factors of interest for assessing the context, process, effects and impacts of a KMb strategy. It also proposes a preliminary inventory of potential dimensions and components to consider when planning the evaluation of a KMb strategy. Given the broad spectrum of factors of interest identified across studies, not all of them can be assessed in every context. Rather, they should be targeted according to the objectives of the evaluation, the nature of the KMb strategy and the resources available to conduct the evaluation. Thus, this inventory should not be understood as a prescriptive, normative and exhaustive framework, but rather as a toolbox to identify the most relevant factors to include in the evaluation of a given KMB strategy, and to address a need often expressed by organizations wishing to evaluate their KMb efforts.

Additional work is needed to validate and operationalize these dimensions, to identify relevant measurement tools related to the different components and to see how this inventory could support KMb evaluation practices in organizations.

This scoping review is the first stage of a larger research project aimed at improving organizations’ capacity to evaluate their KMb initiatives by developing an integrative, interdisciplinary and easy-to-use reference framework. In the second phase of the project, the relevance and clarity of the evaluation dimensions identified in the scoping review will be validated through a Delphi study with KMb specialists and researchers. The enriched framework will then be pilot tested in two organizations carrying out and evaluating KMb strategies, to adapt the framework to their needs and to further clarify how the dimensions can be measured in practice. In this third phase, guidance will be provided to help organizations adopt the framework and its support kit. The aim of the project is to go beyond proposing a theoretical framework, and to help build organizations’ capacity to evaluate KT strategies by proposing tools adapted to their realities.

Review limitations

Some limitations of this scoping review should be acknowledged. First, given the numerous different terms used to describe and conceptualize the science of using evidence, it is possible that our search strategy did not capture all relevant publications. However, to limit this risk, we manually searched the reference lists of the selected articles. Second, the literature search was limited to articles published in English or French, and the articles were mostly from high-income countries (for example, North America); therefore, the application of the identified concepts in this scoping review to other contexts should be further explored.

In addition, the search strategy focused on scientific publications to assess progress made in the field of knowledge mobilization strategy evaluation. The grey literature was not examined. It should be considered in future research to complete the overview of evaluation needs in the field of knowledge mobilization.

Finally, the paucity of information in the articles sometimes made it difficult to classify the TMFs according to the taxonomies [ 32 , 44 ], which may have led to possible misinterpretation. However, to limit the risk of errors, the categorization was performed by two reviewers and validated by a third in cases of uncertainty.

Given the increasing demand from organizations for the evaluation of KMb strategies, along with the poorly consolidated KMb research field, a scoping review was needed to identify the range, nature and extent of the literature. This scoping review enabled us to synthesize the breadth of the literature, provide an overview of the many theories, models and frameworks used, and identify and categorize the potential dimensions and components to consider when evaluating KMb initiatives. This scoping review is part of a larger research project, in which the next steps will be to validate the integrative framework and develop a support kit to facilitate its use by organizations involved in KMb.

Availability of data and materials

The dataset supporting the conclusions of this article is included within the article and its additional files.

Abbreviations

• Knowledge mobilization
• Theories, models, and frameworks

Social Sciences and Humanities Research Council. Guidelines for Effective Knowledge Mobilization. 2019. https://www.sshrc-crsh.gc.ca/funding-financement/policies-politiques/knowledge_mobilisation-mobilisation_des_connaissances-eng.aspx Accessed 28 Dec 2022.

Boaz A, Davies H, Fraser A, Nutley S. What works now? evidence-informed policy and practice. Bristol: Policy press; 2019.

Book   Google Scholar  

Curran JA, Grimshaw JM, Hayden JA, Campbell B. Knowledge translation research: the science of moving research into policy and practice. J Contin Educ Heal Prof. 2011;31(3):174–80.

Article   Google Scholar  

Global Commission on Evidence. The Evidence Commission report: A wake-up call and path forward for decision-makers, evidence intermediaries, and impact-oriented evidence producers. McMaster University; 2022 p. 144. https://www.mcmasterforum.org/networks/evidence-commission/report/english

Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20.

Article   PubMed   PubMed Central   Google Scholar  

Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS ONE. 2011;6(7): e21704.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Straus SE, Tetroe J, Graham ID, editors. Knowledge translation in health care: moving from evidence to practice. 2nd ed. Chichester, West Sussex ; Hoboken, NJ: Wiley/BMJ Books; 2013, 406

Barwick M, Dubrowski R, Petricca K. Knowledge translation: The rise of implementation. 2020; https://ktdrr.org/products/kt-implementation/KT-Implementation-508.pdf

Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Continuing Educ Health Professions. 2006;26(1):13–24.

Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7(1):50.

Strifler L, Cardoso R, McGowan J, Cogo E, Nincic V, Khan PA, et al. Scoping review identifies significant number of knowledge translation theories, models, and frameworks with limited use. J Clin Epidemiol. 2018;100:92–102.

Article   PubMed   Google Scholar  

Esmail R, Hanson HM, Holroyd-Leduc J, Brown S, Strifler L, Straus SE, et al. A scoping review of full-spectrum knowledge translation theories, models, and frameworks. Implement Sci. 2020;15(1):11.

McKibbon KA, Lokker C, Wilczynski NL, Ciliska D, Dobbins M, Davis DA, et al. A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: a Tower of Babel? Implement Sci. 2010;5(1):16.

Fonds de recherche du Québec. Stratégie de mobilisation des connaissances 2014–2017. 2014. https://frq.gouv.qc.ca/en/mobilization-of-knowledge/ . Accessed 28 Dec 2022.

Gervais MJ, Souffez K, Ziam S. Quel impact avons-nous ? Vers l’élaboration d’un cadre pour rendre visibles les retombées du transfert des connaissances. TUC Revue francophone de recherche sur le transfert et l’utilisation des connaissances. 2016;1(2):21.

Google Scholar  

Williams NJ, Beidas RS. Annual research review: the state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field. J Child Psychol Psychiatr. 2019;60(4):430–50.

Mitton C, Adair CE, Mckenzie E, Patten SB, Perry BW. Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q. 2007;85(4):729–68.

Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14(1):2.

Fazey I, Bunse L, Msika J, Pinke M, Preedy K, Evely AC, et al. Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Glob Environ Chang. 2014;25:204–20.

Gervais MJ, Marion C, Dagenais C, Chiocchio F, Houlfort N. Dealing with the complexity of evaluating knowledge transfer strategies: guiding principles for developing valid instruments. Res Eval. 2016;25(1):62–9.

Reed MS, Bryce R, Machen R. Pathways to policy impact: a new approach for planning and evidencing research impact. Evid policy. 2018;14(3):431–58.

Kim C, Wilcher R, Petruney T, Krueger K, Wynne L, Zan T. A research utilisation framework for informing global health and development policies and programmes. Health Res Policy Sys. 2018;16(1):9.

Langer L, Tripney J, Gough D University of London, Social Science Research Unit, Evidence for Policy and Practice Information and Co-ordinating Centre. The science of using science: researching the use of research evidence in decision-making. 2016.

Rajić A, Young I, McEwen SA. Improving the utilization of research knowledge in agri-food public health: a mixed-method review of knowledge translation and transfer. Foodborne Pathog Dis. 2013;10(5):397–412.

Scarlett J, Forsberg BC, Biermann O, Kuchenmüller T, El-Khatib Z. Indicators to evaluate organisational knowledge brokers: a scoping review. Health Res Policy Syst. 2020;18(1):93.

Bornbaum CC, Kornas K, Peirson L, Rosella LC. Exploring the function and effectiveness of knowledge brokers as facilitators of knowledge translation in health-related settings: a systematic review and thematic analysis. Implement Sci. 2015;10(1):162.

Sarkies MN, Bowles KA, Skinner EH, Haas R, Lane H, Haines TP. The effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare: a systematic review. Implement Sci. 2017;12(1):132.

Scott SD, Albrecht L, O’Leary K, Ball GD, Hartling L, Hofmeyer A, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7(1):70.

Dagenais C, Malo M, Robert É, Ouimet M, Berthelette D, Ridde V. Knowledge transfer on complex social interventions in public health: a scoping study. PLoS ONE. 2013;8(12): e80233.

Davies HT, Powell AE, Nutley SM. Mobilising knowledge to improve UK health care: learning from other countries and other sectors – a multimethod mapping study. Health Serv Deliv Res. 2015;3(27):1–190.

Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res. 2020;283: 112461.

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):53.

Ellen ME, Panisset U, Araujo de Carvalho I, Goodwin J, Beard J. A knowledge translation framework on ageing and health. Health Policy. 2017;121(3):282–91.

Wensing M, Bosch M, Grol R. Developing and selecting interventions for translating knowledge to action. CMAJ. 2010;182(2):E85–8.

Bennet A, Bennet D, Fafard K, Fonda M, Lomond T, Messier L, et al. Knowledge mobilization in the social sciences and humanities: moving from research to action. Frost: MQI Press; 2007.

Phipps D, Cummins J, Pepler D, Craig W, Cardinal S. The Co-produced Pathway to Impact Describes Knowledge Mobilization Processes. JCES. 2016;9(1). https://jces.ua.edu/articles/258 . Accessed 17 Nov 2022.

Birken SA, Rohweder CL, Powell BJ, Shea CM, Scott J, Leeman J, et al. T-CaST: an implementation theory comparison and selection tool. Implement Sci. 2018;13(1):143.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Moullin JC, Sabater-Hernandez D, Fernandez-Llimos F, Benrimoj SI. A systematic review of implementation frameworks of innovations in healthcare and resulting generic implementation framework. Health Res Policy Syst. 2015;13(101170481):16.

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synthesis Methods. 2014;5(4):371–85.

Farkas M, Jette AM, Tennstedt S, Haley SM, Quinn V. Knowledge dissemination and utilization in gerontology: an organizing framework. Gerontologist. 2003;43:47–56.

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10(1):109.

Smith JD, Li DH, Rafferty MR. The implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84.

Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.

Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7:149–58.

Sketris IS, Carter N, Traynor RL, Watts D, Kelly K, following contributing members of the CNODES Knowledge Translation Team: Pierre Ernst JG Brenda Hemmelgarn, Colleen Metge, Michael Paterson, Robert Platt W and Gary Teare. Building a framework for the evaluation of knowledge translation for the Canadian Network for Observational Drug Effect Studies. Pharmacoepidemiol Drug Saf. 2020;29 Suppl 1(d0r, 9208369):8–25.

Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval. 2006;27(2):237–46.

Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50(2):179–211.

Moore DE, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1–15.

Tschida JE, Drahota A. Fidelity to the ACT SMART Toolkit: an instrumental case study of implementation strategy fidelity. Implement Sci Commun. 2023;4(1):52.

Colquhoun H, Leeman J, Michie S, Lokker C, Bragge P, Hempel S, et al. Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. Implement Sci. 2014;9(1):781.

Bertone MP, Meessen B, Clarysse G, Hercot D, Kelley A, Kafando Y, et al. Assessing communities of practice in health policy: a conceptual framework as a first step towards empirical research. Health Res Policy Sys. 2013;11(1):39.

Gagliardi AR, Legare F, Brouwers MC, Webster F, Wiljer D, Badley E, et al. Protocol: developing a conceptual framework of patient mediated knowledge translation, systematic review using a realist approach. Implement Sci. 2011;6(101258411):25.

Sargeant J, Borduas F, Sales A, Klein D, Lynn B, Stenerson H. CPD and KT: models used and opportunities for synergy. J Contin Educ Heal Prof. 2011;31(3):167–73.

Stetler CB, Ritchie J, Rycroft-Malone J, Schultz A, Charns M. Improving quality of care through routine, successful implementation of evidence-based practice at the bedside: an organizational case study protocol using the Pettigrew and Whipp model of strategic change. Implement Sci. 2007;2(101258411):3.

Kok MO, Schuit AJ. Contribution mapping: a method for mapping the contribution of research to enhance its impact. Health Res Policy Syst. 2012;10(101170481):21.

Dadich A. From bench to bedside: methods that help clinicians use evidence-based practice. Aust Psychol. 2010;45(3):197–211.

Brown P, Bahri P. Engagement’ of patients and healthcare professionals in regulatory pharmacovigilance: establishing a conceptual and methodological framework. Eur J Clin Pharmacol. 2019;75(9):1181–92.

Article   CAS   PubMed   Google Scholar  

Dobbins M, Ciliska D, Cockerill R, Barnsley J, DiCenso A. A framework for the dissemination and utilization of research for health-care policy and practice. Worldviews Evid Based Nurs Presents Arch Online J Knowl Synthesis Nurs. 2002;9(1):149–60.

Gagliardi AR, Brouwers MC, Bhattacharyya OK. The guideline implementability research and application network (GIRAnet): an international collaborative to support knowledge exchange: study protocol. Implement Sci. 2012;7(101258411):26.

Brooks SP, Zimmermann GL, Lang M, Scott SD, Thomson D, Wilkes G, et al. A framework to guide storytelling as a knowledge translation intervention for health-promoting behaviour change. Implement sci commun. 2022;3(1):35.

Cullen L, Hanrahan K, Edmonds SW, Reisinger HS, Wagner M. Iowa implementation for sustainability framework. Implement Sci. 2022;17(1):1.

Labbé D, Mahmood A, Miller WC, Mortenson WB. Examining the impact of knowledge mobilization strategies to inform urban stakeholders on accessibility: a mixed-methods study. Int J Environ Res Public Health. 2020;17(5):1561.

Straus SE, Tetroe J, Graham ID, Zwarenstein M, Bhattacharyya O, Shepperd S. Monitoring use of knowledge and evaluating outcomes. Can Med Assoc J. 2010;182(2):E94–8.

Bennett S, Whitehead M, Eames S, Fleming J, Low S, Caldwell E. Building capacity for knowledge translation in occupational therapy: learning through participatory action research. BMC Med Educ. 2016;16(1):257.

Brown C, Rogers S. Measuring the effectiveness of knowledge creation as a means of facilitating evidence-informed practice in early years settings in one London Borough. Lond Rev Educ. 2014;12(3):245–60.

Talbott E, De Los RA, Kearns DM, Mancilla-Martinez J, Wang M. Evidence-based assessment in special education research: advancing the use of evidence in assessment tools and empirical processes. Except Child. 2023;89(4):467–87.

Rosella LC, Bornbaum C, Kornas K, Lebenbaum M, Peirson L, Fransoo R, et al. Evaluating the process and outcomes of a knowledge translation approach to supporting use of the Diabetes Population Risk Tool (DPoRT) in public health practice. Canadian J Program Eval. 2018;33(1):21–48.

Couineau AL, Forbes D. Using predictive models of behavior change to promote evidence-based treatment for PTSD. Psychol Trauma Theory Res Pract Policy. 2011;3(3):266–75.

Dufault M. Testing a collaborative research utilization model to translate best practices in pain management. Worldviews Evid Based Nurs. 2004;1:S26-32.

Beckett K, Farr M, Kothari A, Wye L, le May A. Embracing complexity and uncertainty to create impact: exploring the processes and transformative potential of co-produced research through development of a social impact model. Health Res Policy Syst. 2018;16(1):118.

Kramer DM, Wells RP, Carlan N, Aversa T, Bigelow PP, Dixon SM, et al. Did you have an impact? A theory-based method for planning and evaluating knowledge-transfer and exchange activities in occupational health and safety. Int J Occup Saf Ergon. 2013;19(1):41–62.

Duhamel F, Dupuis F, Turcotte A, Martinez AM, Goudreau J. Integrating the illness beliefs model in clinical practice: a family systems nursing knowledge utilization model. J FAM NURS. 2015;21(2):322–48.

Wimpenny P, Johnson N, Walter I, Wilkinson JE. Tracing and identifying the impact of evidence-use of a modified pipeline model. Worldviews Evid Based Nurs. 2008;5(1):3–12.

Ward V, Smith S, House A, Hamer S. Exploring knowledge exchange: a useful framework for practice and policy. Soc Sci Med. 2012;74(3):297–304.

Grooten L, Vrijhoef HJM, Alhambra-Borras T, Whitehouse D, Devroey D. The transfer of knowledge on integrated care among five European regions: a qualitative multi-method study. BMC Health Serv Res. 2020;20(1):11.

Stetler CB. Updating the Stetler Model of research utilization to facilitate evidence-based practice. Nurs Outlook. 2001;49(6):272–9.

Ward V. Why, whose, what and how? A framework for knowledge mobilisers. Evid Policy J Res Debate Pract. 2017;13(3):477–97.

Levin RF, Fineout-Overholt E, Melnyk BM, Barnes M, Vetter MJ. Fostering evidence-based practice to improve nurse and cost outcomes in a community health setting: a pilot test of the advancing research and clinical practice through close collaboration model. Nurs Adm Q. 2011;35(1):21–33.

Currie M, King G, Rosenbaum P, Law M, Kertoy M, Specht J. A model of impacts of research partnerships in health and social services. Eval Program Plann. 2005;28(4):400–12.

Richard L, Chiocchio F, Essiembre H, Tremblay MC, Lamy G, Champagne F, et al. Communities of practice as a professional and organizational development strategy in local public health organizations in Quebec, Canada: an evaluation model. Healthc Policy. 2014;9(3):26–39.

PubMed   PubMed Central   Google Scholar  

Rycroft-Malone J, Wilkinson J, Burton CR, Harvey G, McCormack B, Graham I, et al. Collaborative action around implementation in collaborations for leadership in applied health research and care: towards a programme theory. J Health Serv Res Policy. 2013;18(3 Suppl):13–26.

Gagliardi AR, Fraser N, Wright FC, Lemieux-Charles L, Davis D. Fostering knowledge exchange between researchers and decision-makers: exploring the effectiveness of a mixed-methods approach. Health Policy. 2008;86(1):53–63.

Paquette-Warren J, Harris SB, Naqshbandi Hayward M, Tompkins JW. Case study of evaluations that go beyond clinical outcomes to assess quality improvement diabetes programmes using the Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE). J Eval Clin Pract. 2016;22(5):644–52.

Paquette-Warren J, Tyler M, Fournie M, Harris SB. The diabetes evaluation framework for innovative national evaluations (DEFINE): construct and content validation using a modified Delphi method. Can J diabetes. 2017;41(3):281–96.

Abbot ML, Lee KK, Rossiter MJ. Evaluating the effectiveness and functionality of professional learning communities in adult ESL Programs. TESL Canada J. 2018;35(2):1–25.

Ho K, Bloch R, Gondocz T, Laprise R, Perrier L, Ryan D, et al. Technology-enabled knowledge translation: frameworks to promote research and practice. J Contin Educ Heal Prof. 2004;24(2):90–9.

Yu X, Hu D, Li N, Xiao Y. Comprehensive evaluation on teachers’ knowledge sharing behavior based on the improved TOPSIS method. Comput Intell Neurosci. 2022;2022(101279357):2563210.

Arora S, Kalishman SG, Thornton KA, Komaromy MS, Katzman JG, Struminger BB, et al. Project ECHO: a telementoring network model for continuing professional development. J Contin Educ Health Prof. 2017;37(4):239–44.

Smidt A, Balandin S, Sigafoos J, Reed VA. The Kirkpatrick model: a useful tool for evaluating training outcomes. J Intellect Dev Disabil. 2009;34(3):266–74.

Jeffs L, Sidani S, Rose D, Espin S, Smith O, Martin K, et al. Using theory and evidence to drive measurement of patient, nurse and organizational outcomes of professional nursing practice. Int J Nurs Pract. 2013;19(2):141–8.

Skinner K. Developing a tool to measure knowledge exchange outcomes. Can J Program Eval. 2007;22(1):49–75.

Lavis J, Ross S, McLeod C, Gildiner A. Measuring the impact of health research. J Health Serv Res Policy. 2003;8(3):165–70.

Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N. Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Soc Sci Med. 2012;75(11):1938–45.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

Gainforth HL, Latimer-Cheung AE, Athanasopoulos P, Martin Ginis KA. Examining the feasibility and effectiveness of a community-based organization implementing an event-based knowledge mobilization initiative to promote physical activity guidelines for people with spinal cord injury among support personnel. Health Promot Pract. 2015;16(1):55–62.

Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health. 2019. https://doi.org/10.3389/fpubh.2019.00064 .

Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. 2020;8(101616579):134.

Bender BG, Simmons B, Konkoly N, Liu AH. The asthma toolkit bootcamp to improve rural primary care for pediatric asthma. J Allergy Clin Immunol Pract. 2021;9(8):3091-3097.e1.

de la Garza Iga FJ, Mejia Alvarez M, Cockroft JD, Rabin J, Cordon A, Elias Rodas DM, et al. Using the project ECHO TM model to teach mental health topics in rural Guatemala: an implementation science-guided evaluation. Int J Soc Psychiatry. 2023;69(8):2031–41.

Alkin M, Taut S. Unbundling evaluation use. Stud Educ Eval. 2003;29(1):1–12.

Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Syst. 2020;18(1):64.

McCabe KE, Wallace A, Crosland A. A model for collaborative working to facilitate knowledge mobilisation in public health. Evid Policy. 2015;11(4):559–76.

Gonzales R, Handley MA, Ackerman S, O’sullivan PS. A framework for training health professionals in implementation and dissemination science. Acad Med. 2012;87(3):271–8.

Edgar L, Herbert R, Lambert S, MacDonald JA, Dubois S, Latimer M. The joint venture model of knowledge utilization: a guide for change in nursing. Nurs Leadersh. 2006;9(2):41–55.

Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A Guide for applying a revised version of the PARIHS framework for implementation. Implement Sci. 2011;6(101258411):99.

Brennan SE, Cumpston M, Misso ML, McDonald S, Murphy MJ, Green SE. Design and formative evaluation of the policy liaison initiative: a long-term knowledge translation strategy to encourage and support the use of cochrane systematic reviews for informing. Evid Policy. 2016;12(1):25–52.

Hinchcliff R, Senserrick T, Travaglia J, Greenfield D, Ivers R. The enhanced knowledge translation and exchange framework for road safety: a brief report on its development and potential impacts. Inj Prev. 2017;23(2):114–7.

Ye J, Woods D, Bannon J, Bilaver L, Kricke G, McHugh M, et al. Identifying contextual factors and strategies for practice facilitation in primary care quality improvement using an informatics-driven model: framework development and mixed methods case study. JMIR Hum Factors. 2022;9(2): e32174.

Brangan J, Quinn S, Spirtos M. Impact of an evidence-based practice course on occupational therapist’s confidence levels and goals. Occup Ther Health Care. 2015;29(1):27–38.

Bonetti D, Johnston M, Pitts NB, Deery C, Ricketts I, Tilley C, et al. Knowledge may not be the best target for strategies to influence evidence-based practice: using psychological models to understand RCT effects. Int J Behav Med. 2009;16(3):287–93.

Buckley LL, Goering P, Parikh SV, Butterill D, Foo EKH. Applying a “stages of change” model to enhance a traditional evaluation of a research transfer course. J Eval Clin Pract. 2003;9(4):385–90.

Boyko JA, Lavis JN, Dobbins M, Souza NM. Reliability of a tool for measuring theory of planned behaviour constructs for use in evaluating research use in policymaking. Health Res Policy Syst. 2011;24(9):29.

Imani-Nasab MH, Yazdizadeh B, Salehi M, Seyedin H, Majdzadeh R. Validity and reliability of the Evidence Utilisation in Policymaking Measurement Tool (EUPMT). Health Res Policy Syst. 2017;15(1):66.

Dwan KM, McInnes P, Mazumdar S. Measuring the success of facilitated engagement between knowledge producers and users: a validated scale. Evid Policy. 2015;11(2):239–52.

Haynes A, Rowbotham S, Grunseit A, Bohn-Goldbaum E, Slaytor E, Wilson A, et al. Knowledge mobilisation in practice: an evaluation of the Australian Prevention Partnership Centre. Health Res Policy Sys. 2020;18(1):13.

Haines M, Brown B, Craig J, D’Este C, Elliott E, Klineberg E, et al. Determinants of successful clinical networks: the conceptual framework and study protocol. Implement Sci. 2012;7(101258411):16.

Ko LK, Jang SH, Friedman DB, Glanz K, Leeman J, Hannon PA, et al. An application of the science impact framework to the cancer prevention and control research network from 2014–2018. Prev Med. 2019;12: 105821.

Leeman J, Sommers J, Vu M, Jernigan J, Payne G, Thompson D, et al. An evaluation framework for obesity prevention policy interventions. Prev Chronic Dis. 2012;9(101205018):E120.

Pettman TL, Armstrong R, Waters E, Allender S, Love P, Gill T, et al. Evaluation of a knowledge translation and exchange platform to advance non-communicable disease prevention. Evid Policy. 2016;12(1):109–26.

Yearwood AC. Applying a logical theory of change for strengthening research uptake in policy: a case study of the Evidence Informed Decision Making Network of the Caribbean. Rev Panam Salud Publica. 2018;42: e91.

Thomson D, Brooks S, Nuspl M, Hartling L. Programme theory development and formative evaluation of a provincial knowledge translation unit. Health Res Policy Syst. 2019;17(1):40.

Garad R, Kozica-Olenski S, Teede HJ. Evaluation of a center of research excellence in polycystic ovary syndrome as a large-scale collaborative research translation initiative, including evaluating translation of guideline impact. Semin Reprod Med. 2018;36(1):42–9.

Reddy S, Wakerman J, Westhorp G, Herring S. Evaluating impact of clinical guidelines using a realist evaluation framework. J Eval Clin Pract. 2015;21(6):1114–20.

Van Eerd D, Moser C, Saunders R. A research impact model for work and health. Am J Ind Med. 2021;64(1):3–12.

Yip O, Huber E, Stenz S, Zullig LL, Zeller A, De Geest SM, et al. A contextual analysis and logic model for integrated care for frail older adults living at home: The INSPIRE Project. Int J Integr Care. 2021;21(2):9.

Guo R, Bain BA, Willer J. Application of a logic model to an evidence-based practice training program for speech-language pathologists and audiologists. J Allied Health. 2011;40(1):e23–8.

PubMed   Google Scholar  

McDonald S, Turner T, Chamberlain C, Lumbiganon P, Thinkhamrop J, Festin MR, et al. Building capacity for evidence generation, synthesis and implementation to improve the care of mothers and babies in South East Asia: methods and design of the SEA-ORCHID Project using a logical framework approach. BMC Med Res Methodol. 2010;10(100968545):61.

Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50.

Wensing M, Grol R. Knowledge translation in health: how implementation science could contribute more. BMC Med. 2019;17(1):88.

Kreindler SA. Advancing the evaluation of integrated knowledge translation. Health Res Policy Sys. 2018;16(1):104.

Best A, Holmes B. Systems thinking, knowledge and action: towards better models and methods. Evid Policy. 2010;6(2):145–59.

van Mil HGJ, Foegeding EA, Windhab EJ, Perrot N, van der Linden E. A complex system approach to address world challenges in food and agriculture. Trends Food Sci Technol. 2014;40(1):20–32.

Wehrens R. Beyond two communities – from research utilization and knowledge translation to co-production? Public Health. 2014;128(6):545–51.

Ridde V, Pérez D, Robert E. Using implementation science theories and frameworks in global health. BMJ Glob Health. 2020;5(4): e002269.

Salter KL, Kothari A. Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implement Sci. 2014;9(1):115.

Van Eerd D, Cole D, Keown K, Irvin E, Kramer D, Gibson B, et al. Report on knowledge transfer and exchange practices: A systematic review of the quality and types of instruments used to assess KTE implementation and impact. Toronto: Institute for Work & Health; 2011 p. 130. https://www.iwh.on.ca/sites/iwh/files/iwh/reports/iwh_sys_review_kte_evaluation_tools_2011_rev.pdf

Dobbins M, Robeson P, Ciliska D, Hanna S, Cameron R, O’Mara L, et al. A description of a knowledge broker role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implement Sci. 2009;4(1):23.

Rychetnik L, Bauman A, Laws R, King L, Rissel C, Nutbeam D, et al. Translating research for evidence-based public health: key concepts and future directions. J Epidemiol Community Health. 2012;66(12):1187–92.

Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.

Bhawra J, Skinner K. Examination of tools associated with the evaluation of knowledge uptake and utilization: a scoping review. Eval Program Plann. 2020;83: 101875.

Lane JP, Stone VI, Nobrega A, Tomita M. Level Of Knowledge Use Survey (LOKUS): a validated instrument for tracking knowledge uptake and use. Stud Health Technol Inform. 2015;217:106–10.

Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8(1):139.

Albrecht L, Archibald M, Arseneau D, Scott SD. Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations. Implement Sci. 2013;8(1):52.

Bragge P, Grimshaw JM, Lokker C, Colquhoun H, Albrecht L, Baron J, et al. AIMD - a validated, simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. BMC Med Res Methodol. 2017;17(1):38.

Kastner M, Makarski J, Hayden L, Lai Y, Chan J, Treister V, et al. Improving KT tools and products: development and evaluation of a framework for creating optimized, Knowledge-activated Tools (KaT). Implement Sci Commun. 2020;1(1):47.

Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for Reporting Implementation Studies (StaRI) Statement. BMJ. 2017;6(356): i6795.

Lokker C, McKibbon KA, Colquhoun H, Hempel S. A scoping review of classification schemes of interventions to promote and integrate evidence into practice in healthcare. Implement Sci. 2015;10(1):27.

Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;7(348): g1687.

Wilson PM, Sales A, Wensing M, Aarons GA, Flottorp S, Glidewell L, et al. Enhancing the reporting of implementation research. Implement Sci. 2017;12(1):13.

Download references

Acknowledgements

We wish to thank Julie Desnoyers for designing and implementing the search strategy, Gabrielle Legendre for her contribution in the screening phase and Karine Souffez and Caroline Tessier for their input during the project.

This project was supported by an Insight Grant from the Social Sciences and Humanities Research Council of Canada (SSHRC) and by the Équipe RENARD (FRQ-SC). The funding bodies had no role in the conduct of this scoping review.

Author information

Authors and affiliations.

School of Business Administration, Université TÉLUQ, Montreal, Canada

Saliha Ziam & Laura Justine Chouinard

Department of School and Social Adaptation Studies, Faculty of Education, Université de Sherbrooke, Sherbrooke, Canada

Sèverine Lanoue, Esther McSween-Cadieux, Julie Lane & Ollivier Prigent

Department of Psychology, Université du Québec à Montréal, Montreal, Canada

Mathieu-Joël Gervais

Centre RBC d’expertise Universitaire en Santé Mentale, Université de Sherbrooke, Sherbrooke, Canada

School of Rehabilitation, Faculty of Medicine, Université de Montréal, Montreal, Canada

Dina Gaid & Quan Nha Hong

Department of Psychology, Université de Montréal, Montreal, Canada

Christian Dagenais

Université Paris Cité, IRD (Institute for Research on Sustainable Development, CEPED, Paris, France

Valéry Ridde

Institute of Health and Development (ISED), Cheikh Anta Diop University, Dakar, Senegal

Public Health Intelligence and Knowledge Translation Division, Public Health Agency of Canada, Ottawa, Canada

Emmanuelle Jean

Coordinator of the Interregional Consortium of Knowledge in Health and Social Services (InterS4), Rimouski, Canada

France Charles Fleury

You can also search for this author in PubMed   Google Scholar

Contributions

SZ, MJG, EMC, JL, CD, EJ, KS, VR and CT were involved in developing and designing the scoping review. EMC, MJG and GL (collaborator) screened articles in duplicate. SL, DG, LJC and OP extracted data from the included articles. SL and DG synthesized the data. SL, SZ and EMC drafted the manuscript. SZ led the project, supervised and assisted the research team at every stage, and secured the funding. All authors provided substantive feedback and approved the manuscript prior to submission.

Corresponding author

Correspondence to Saliha Ziam .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Keywords and search strategy.

Additional file 2.

Summary of included articles.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ziam, S., Lanoue, S., McSween-Cadieux, E. et al. A scoping review of theories, models and frameworks used or proposed to evaluate knowledge mobilization strategies. Health Res Policy Sys 22 , 8 (2024). https://doi.org/10.1186/s12961-023-01090-7

Download citation

Received : 16 June 2023

Accepted : 07 December 2023

Published : 10 January 2024

DOI : https://doi.org/10.1186/s12961-023-01090-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Knowledge translation
• Scoping review

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]