questions to ask about research articles

• Follow us on Facebook
• Follow us on Twitter
• Criminal Justice
• Environment
• Politics & Government
• Race & Gender

Expert Commentary

Eight questions to ask when interpreting academic studies: A primer for media

Scholarly research is a great source for rigorous, unbiased information, but making judgments about its quality can be difficult. Here are some important questions to ask when reading studies.

Republish this article

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License .

by Justin Feldman and John Wihbey, The Journalist's Resource March 26, 2015

This <a target="_blank" href="https://journalistsresource.org/home/interpreting-academic-studies-primer-media/">article</a> first appeared on <a target="_blank" href="https://journalistsresource.org">The Journalist's Resource</a> and is republished here under a Creative Commons license.<img src="https://journalistsresource.org/wp-content/uploads/2020/11/cropped-jr-favicon-150x150.png" style="width:1em;height:1em;margin-left:10px;">

Reading scholarly studies can help journalists integrate rigorous, unbiased sources of information into their reporting. These studies are typically carried out by professors and professional researchers — at universities, think tanks and government institutions — and are published through a peer-review process in which those familiar with the study area ensure that there are no major flaws.

Even for people who carry out research, however, interpreting scientific (and social science) studies and making judgments about their quality can be difficult tasks. In a now-famous article, Stanford professor John Ioannidis argues that “ most published research findings are false ” due to inherent limitations in how researchers design studies. (Health and medical studies can be particularly attractive to media, but be aware that there is a long history of faulty findings .) Occasionally, too, studies can be the product of outright fraud: A 1998 study falsely linking vaccines and autism is now perhaps the canonical example, as it spurred widespread and long-lasting societal damage . Journalists should also always examine the funding sources behind the study, which are frequently declared at the study’s conclusion.

Before journalists write about research and speak with authors, they should be able to both interpret a study’s results generally and understand the appropriate degree of skepticism that a given study’s findings warrant. This requires data literacy , some familiarity with statistical terms and a basic knowledge of hypothesis testing and construction of theories .

Journalists should also be well aware that most academic research contains careful qualifications about findings. The common complaint from scientists and social scientists is that news media tend to pump up findings and hype studies through catchy headlines, distorting public understanding. But landmark studies sometimes do no more than tighten the margin of error around a given measurement — not inherently flashy, but intriguing to an audience if explained with rich context and clear presentation.

Here are some important questions to ask when reading a scientific study:

1. What are the researchers’ hypotheses?

A hypothesis is a research question that a study seeks to answer. Sometimes researchers state their hypotheses explicitly, but more often their research questions are implicit. Hypotheses are testable assertions usually involving the relationship between two variables. In a study of smoking and lung cancer, the hypothesis might be that smokers develop lung cancer at a higher rate than non-smokers over a five-year period.

It is also important to note that there are formal definitions of null and alternative hypotheses for use with statistical analysis.

2. What are the independent and dependent variables?

Independent variables are factors that influence particular outcomes. Dependent variables are measures of the outcomes themselves. In the study assessing the relationship between smoking and lung cancer, smoking is the independent variable because the researcher assumes it predicts lung cancer, the dependent variable. (Some fields use related terms such as “exposure” and “outcome.”)

Pay particular attention to how the researchers define all of the variables — there can be quite a bit of nuance in the definitions. Also look at the methods by which the researchers measure the variables. Generally speaking, a variable measured using a subject’s response to a survey question is less trustworthy than one measured through more objective means — reviewing laboratory findings in their medical records, for example.

3. What is the unit of analysis?

For most studies involving human subjects, the individual person is the unit of analysis. However, studies are sometimes interested in a different level of analysis that makes comparisons between classrooms, hospitals, schools or states, for example, rather than between individuals.

4. How well does the study design address causation?

Most studies identify correlations or associations between variables, but typically the ultimate goal is to determine causation . Certain study designs are more useful than others for the purpose of determining causation.

At the most basic level, studies can be placed into one of two categories: experimental and observational . In experimental studies, the researchers decide who is exposed to the independent variable and who is not. In observational studies, the researchers do not have any control over who is exposed to the independent variable — instead they make comparisons between groups that are already different from one another. In nearly all cases, experimental studies provide stronger evidence than observational studies.

Here are descriptions of some of the most common study designs, presented along with their respective values for inferring causation:

• Randomized controlled trials (RCTs), also known as clinical trials, are experimental studies that are considered the “gold standard” in research. Out of all study designs, they have the most value for determining causation although they do have limitations. In an RCT, researchers randomly divide subjects into at least two groups: One that receives a treatment, and the other — the control group — that receives either no treatment or a simulated version of the treatment called a placebo . The independent variable in these experiments is whether or not the subject receives the real treatment. Ideally an RCT should be double-blind — the participants should not know to which treatment group they have been assigned, nor should the study staff know. This arrangement helps to avoid bias. Researchers commonly use RCTs to meet regulatory requirements, such as evaluating pharmaceuticals for the Food and Drug Administration. Due to issues of cost, logistics and ethics, RCTs are fairly uncommon for other purposes. Example: “ Short-Term Soy Isoflavone Intervention in Patients with Localized Prostate Cancer ”
• Longitudinal studies , like RCTs, follow the same subjects over a given time period. Unlike in RCTs, they are observational. Researchers do not assign the independent variable in longitudinal studies — they instead observe what happens in the real world. A longitudinal study might compare the risk for heart disease among one group of people who are exposed to high levels of air pollution to the risk of heart disease among another group exposed to low levels of air pollution. The problem is that, because there is no random assignment, the groups may differ from one another in other important ways and, as a result, we cannot completely isolate the effects of air pollution. These differences result in confounding and other forms of bias. For that reason, longitudinal studies have less validity for inferring causation than RCTs and other experimental study designs. Longitudinal studies have more validity than other kinds of observational studies, however. Example: “ Mood after Moderate and Severe Traumatic Brain Injury: A Prospective Cohort Study ”
• Case-control studies are technically a type of longitudinal study, but they are unique enough to discuss separately. Common in public health and medical research, case-control studies begin with a group of people who have already developed a particular disease and compare them to a similar but disease-free group recruited by the researchers. These studies are more likely to suffer from bias than other longitudinal studies for two reasons. First, they are always retrospective , meaning they collect data about independent variables years after the exposures of interest occurred — sometimes even after the subject has died. Second, the group of disease-free people is very likely to differ from the group that developed the disease, creating a substantial risk for confounding. Example: “ Risk Factors for Preeclampsia in Women from Colombia ”.
• Cross-sectional studies are a kind of observational study that measure both dependent and independent variables at a single point in time. Although researchers may administer the same cross-sectional survey every few years, they do not follow the same subjects over time. An important part of determining causation is establishing that the independent variable occurred for a given subject before the dependent variable occurred. But because they do not measure the variables over time, cross-sectional studies cannot determine that a hypothesized cause precedes its effect, so the design is limited to making inferences about correlations rather than causation. Example : “ Physical Predictors of Cognitive Performance in Healthy Older Adults ”
• Ecological studies are observational studies that are similar to cross-sectional studies except that they measure at least one variable on the group-level rather that the subject-level. For example, an ecological study may look at the relationship between individuals’ meat consumption and their incidence of colon cancer. But rather than using individual-level data, the study relies on national cancer rates and national averages for meat consumption. While it might seem that higher meat consumption is linked to a higher risk of cancer, there is no way to know if the individuals eating more meat within a country are the same people who are more likely to develop cancer. This means that ecological studies are not only inadequate for inferring causation, they are also inadequate for establishing a correlation. As a consequence, they should be regarded with strong skepticism. Example: “ A Multi-country Ecological Study of Cancer Incidence Rates in 2008 with Respect to Various Risk-Modifying Factors ”
• Systematic reviews are surveys of existing studies on a given topic. Investigators specify inclusion and exclusion criteria to weed out studies that are either irrelevant to their research question or poorly designed. Using keywords, they systematically search research databases, present the findings of the studies they include and draw conclusions based on their consideration of the findings. Assuming that the review includes only well-designed studies, systematic reviews are more useful for inferring causation than any single well-designed study. Example: “ Enablers and Barriers to Large-Scale Uptake of Improved Solid Fuel Stoves. ” For a sense of how systematic reviews are interpreted and used by researchers in the field, see “How to Read a Systematic Review and Meta-analysis and Apply the Results to Patient Care,” published in the Journal of the American Medical Association (JAMA.)
• Meta-analyses are similar to systematic reviews but use the original data from all included studies to create a new analysis. As a result, a meta-analysis is able to draw conclusions that are more meaningful than a systematic review. Again, a meta-analysis is more useful for inferring causation than any single study, assuming that all studies are well-designed. Example: “ Occupational Exposure to Asbestos and Ovarian Cancer ”

5. What are the study’s results?

There are several aspects involved in understanding a study’s results:

• Understand whether or not the study found statistically significant relationships between the dependent and independent variables. If the relationship is statistically significant, it means that any difference observed between groups is unlikely to be due to random chance. P-values help researchers to decide whether observed differences are simply due to chance or represent a true difference between groups.
• If the relationship is statistically significant, it is then important to determine the effect size , which is the size of the difference observed between the groups. Subjects enrolled in a weight loss program may have experienced a statistically significant reduction in weight compared to those in a control group, but is that difference one ounce, one pound or ten pounds? There are myriad ways in which studies present effect sizes — such obscure terms as regression coefficients, odds ratios, and population attributable fractions may come into play. Unfortunately, research articles sometimes fail to interpret effect sizes in words. In these cases, it may be best to consult an expert to help develop a plain-English interpretation.
• Even if there is a statistically significant difference between comparison groups, this does not mean the effect size is meaningful. A weight loss program that leads to a total weight reduction of one ounce on average or a policy that saves one life out of a billion may not be meaningful. Again, consulting an expert in the field can help to determine how meaningful an effect size is, a determination that is ultimately a subjective judgment call.

6. How generalizable are the results?

Study results are useful because they help us make inferences about the relationship between independent and dependent variables among a larger population. The subjects enrolled in the study must be similar to those in the larger population, however, in order to generalize the findings. Even a perfectly designed study may be of limited value when its results cannot be generalized. It is important to pay attention to the composition of the study sample. If the unit of analysis is the individual, important factors to consider regarding the group’s composition include age, race/ethnicity, gender, socioeconomic status, and geographic location. While some samples are deliberately constructed to be representative of a country or region, most are not.

7. What limitations do the authors note?

Within a research article, authors often state some of the study’s limitations explicitly. This information can be very helpful in determining the strength of the evidence presented in the study.

8. What conclusions do similar studies draw?

With some notable exceptions, a single study is unlikely to fundamentally change what is already known about the research question it addresses. It is important to compare a new study’s findings to existing studies that address similar research questions, particularly systematic reviews or meta-analyses if available.

Further: One hidden form of bias that is easily missed is what’s called “selecting on the dependent variable,” which is the research practice of focusing on only those areas where there are effects and ignoring ones where there are not. This can lead to exaggerated conclusions (and thereby false media narratives). For example, it is tempting to say that “science has become polarized,” as survey data suggest significant differences in public opinion on issues such as climate change, vaccinations and nuclear power. However, on most scientific issues, there is almost no public debate or controversy . Additionally, the reality of “publication bias” — academic journals have traditionally been more interested in publishing studies that show effects, rather than no effects — can create a biased incentive structure that distorts larger truths.

For an updated overview, see a 2014 paper by Stanford’s John Ioannidis, “How to Make More Published Research True.”

Keywords: training

About the Authors

Justin Feldman

John Wihbey

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Starting the research process
• Writing Strong Research Questions | Criteria & Examples

Writing Strong Research Questions | Criteria & Examples

Published on October 26, 2022 by Shona McCombes . Revised on November 21, 2023.

A research question pinpoints exactly what you want to find out in your work. A good research question is essential to guide your research paper , dissertation , or thesis .

All research questions should be:

• Focused on a single problem or issue
• Researchable using primary and/or secondary sources
• Feasible to answer within the timeframe and practical constraints
• Specific enough to answer thoroughly
• Complex enough to develop the answer over the space of a paper or thesis
• Relevant to your field of study and/or society more broadly

Table of contents

How to write a research question, what makes a strong research question, using sub-questions to strengthen your main research question, research questions quiz, other interesting articles, frequently asked questions about research questions.

You can follow these steps to develop a strong research question:

• Choose your topic
• Do some preliminary reading about the current state of the field
• Narrow your focus to a specific niche
• Identify the research problem that you will address

The way you frame your question depends on what your research aims to achieve. The table below shows some examples of how you might formulate questions for different purposes.

Using your research problem to develop your research question

Note that while most research questions can be answered with various types of research , the way you frame your question should help determine your choices.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Research questions anchor your whole project, so it’s important to spend some time refining them. The criteria below can help you evaluate the strength of your research question.

Focused and researchable

Feasible and specific, complex and arguable, relevant and original.

Chances are that your main research question likely can’t be answered all at once. That’s why sub-questions are important: they allow you to answer your main question in a step-by-step manner.

Good sub-questions should be:

• Less complex than the main question
• Focused only on 1 type of research
• Presented in a logical order

Here are a few examples of descriptive and framing questions:

• Descriptive: According to current government arguments, how should a European bank tax be implemented?
• Descriptive: Which countries have a bank tax/levy on financial transactions?
• Framing: How should a bank tax/levy on financial transactions look at a European level?

Keep in mind that sub-questions are by no means mandatory. They should only be asked if you need the findings to answer your main question. If your main question is simple enough to stand on its own, it’s okay to skip the sub-question part. As a rule of thumb, the more complex your subject, the more sub-questions you’ll need.

Try to limit yourself to 4 or 5 sub-questions, maximum. If you feel you need more than this, it may be indication that your main research question is not sufficiently specific. In this case, it’s is better to revisit your problem statement and try to tighten your main question up.

Prevent plagiarism. Run a free check.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

The way you present your research problem in your introduction varies depending on the nature of your research paper . A research paper that presents a sustained argument will usually encapsulate this argument in a thesis statement .

A research paper designed to present the results of empirical research tends to present a research question that it seeks to answer. It may also include a hypothesis —a prediction that will be confirmed or disproved by your research.

As you cannot possibly read every source related to your topic, it’s important to evaluate sources to assess their relevance. Use preliminary evaluation to determine whether a source is worth examining in more depth.

This involves:

• Reading abstracts , prefaces, introductions , and conclusions
• Looking at the table of contents to determine the scope of the work
• Consulting the index for key terms or the names of important scholars

A research hypothesis is your proposed answer to your research question. The research hypothesis usually includes an explanation (“ x affects y because …”).

A statistical hypothesis, on the other hand, is a mathematical statement about a population parameter. Statistical hypotheses always come in pairs: the null and alternative hypotheses . In a well-designed study , the statistical hypotheses correspond logically to the research hypothesis.

Formulating a main research question can be a difficult task. Overall, your question should contribute to solving the problem that you have defined in your problem statement .

However, it should also fulfill criteria in three main areas:

• Researchability
• Feasibility and specificity
• Relevance and originality

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 21). Writing Strong Research Questions | Criteria & Examples. Scribbr. Retrieved April 8, 2024, from https://www.scribbr.com/research-process/research-questions/

Is this article helpful?

Shona McCombes

Other students also liked, how to define a research problem | ideas & examples, how to write a problem statement | guide & examples, 10 research question examples to guide your research project, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Evaluating Research Articles

Understanding research statistics, critical appraisal.

Imagine for a moment that you are trying to answer a clinical (PICO) question regarding one of your patients/clients. Do you know how to determine if a research study is of high quality? Can you tell if it is applicable to your question? In evidence based practice, there are many things to look for in an article that will reveal its quality and relevance. This guide is a collection of resources and activities that will help you learn how to evaluate articles efficiently and accurately.

Is health research new to you? Or perhaps you're a little out of practice with reading it? The following questions will help illuminate an article's strengths or shortcomings. Ask them of yourself as you are reading an article:

• Is the article peer reviewed?
• Are there any conflicts of interest based on the author's affiliation or the funding source of the research?
• Are the research questions or objectives clearly defined?
• Is the study a systematic review or meta analysis?
• Is the study design appropriate for the research question?
• Is the sample size justified? Do the authors explain how it is representative of the wider population?
• Do the researchers describe the setting of data collection?
• Does the paper clearly describe the measurements used?
• Did the researchers use appropriate statistical measures?
• Are the research questions or objectives answered?
• Did the researchers account for confounding factors?
• Have the researchers only drawn conclusions about the groups represented in the research?
• Have the authors declared any conflicts of interest?

If the answer to these questions about an article you are reading are mostly YESes , then it's likely that the article is of decent quality. If the answers are most NOs , then it may be a good idea to move on to another article. If the YESes and NOs are roughly even, you'll have to decide for yourself if the article is good enough quality for you. Some factors, like a poor literature review, are not as important as the researchers neglecting to describe the measurements they used. As you read more research, you'll be able to more easily identify research that is well done vs. that which is not well done.

Determining if a research study has used appropriate statistical measures is one of the most critical and difficult steps in evaluating an article. The following links are great, quick resources for helping to better understand how to use statistics in health research.

• How to read a paper: Statistics for the non-statistician. II: “Significant” relations and their pitfalls This article continues the checklist of questions that will help you to appraise the statistical validity of a paper. Greenhalgh Trisha. How to read a paper: Statistics for the non-statistician. II: “Significant” relations and their pitfalls BMJ 1997; 315 :422 *On the PMC PDF, you need to scroll past the first article to get to this one.*
• A consumer's guide to subgroup analysis The extent to which a clinician should believe and act on the results of subgroup analyses of data from randomized trials or meta-analyses is controversial. Guidelines are provided in this paper for making these decisions.

Statistical Versus Clinical Significance

When appraising studies, it's important to consider both the clinical and statistical significance of the research. This video offers a quick explanation of why.

If you have a little more time, this video explores statistical and clinical significance in more detail, including examples of how to calculate an effect size.

• Statistical vs. Clinical Significance Transcript Transcript document for the Statistical vs. Clinical Significance video.
• Effect Size Transcript Transcript document for the Effect Size video.
• P Values, Statistical Significance & Clinical Significance This handout also explains clinical and statistical significance.
• Absolute versus relative risk – making sense of media stories Understanding the difference between relative and absolute risk is essential to understanding statistical tests commonly found in research articles.

Critical appraisal is the process of systematically evaluating research using established and transparent methods. In critical appraisal, health professionals use validated checklists/worksheets as tools to guide their assessment of the research. It is a more advanced way of evaluating research than the more basic method explained above. To learn more about critical appraisal or to access critical appraisal tools, visit the websites below.

• Last Updated: Mar 19, 2024 1:28 PM
• URL: https://libguides.massgeneral.org/evaluatingarticles

How to identify reliable scientific sources: The top six questions to ask

As a clinician, you know how important it is to keep up to date on medical literature. Only by reading scientific literature will you be able to practice evidence-based medicine. But we also know that there is a lot of literature out there, and as a busy professional there’s only so much time you can spend sifting through it. That said, what good is your ability to stay current on the latest research findings and literature if you pick the wrong articles?

This Mini Guide summarizes the top six questions to ask yourself when evaluating a scientific article. These six questions will help you determine if a study is credible and beneficial to your patients. By identifying reliable scientific articles, you’ll be equipped to reject faulty articles and provide the latest evidence-based medicine to your patients.

These are the six questions you should ask when reviewing an article:

• Is the article published in a credible journal?
• Are the authors of the study qualified and free of conflicts of interest?
• Are the study participants a good representation of the target population?
• Are the study designs used appropriately for the article’s research question?
• Are the results interpreted with the correct statistical methods?
• Are the right conclusions drawn based on the results presented?

Let’s take a look at each of these questions in a little more detail.

Question 1: Is the article published in a credible journal?

The majority of scientific articles are published in academic journals. The highest quality journals are those that provide peer-reviews before publishing any articles. This gives you, the reader, the assurance that a group of your scientific peers has approved the paper’s content and methodology before publication.

But, there are journal publications that prey on unsuspecting researchers with the promise of publishing the researchers work—regardless of the quality. These are known as predatory journals that may have broad names and often cover multiple disciplines. You can use impact factors to help identify these less-than-stellar journals.

In fact, you can use a journal’s impact factor to get an idea of the quality of articles that a journal prints. An impact factor is the average number of citations that an article in the journal receives in a year. A journal full of highly cited articles will have a high impact factor, and a journal full of low-cited articles will have a low impact factor. Think of it like this: a great product will generate positive reviews, and people will recommend it to others. By choosing to reference an article in their own work, a researcher is giving credibility to an article and the article’s publication.

Since a journal’s impact factor is based on an average, it’s also a good idea to check the article’s citation frequency. You can use a tool like Google Scholar to get an idea about the credibility and quality of a specific article by looking at how many times the article has been cited by other authors.

Here’s the take-home message: if you find an article that interests you, take a minute to look into where the paper is published. Journal publications with high impact factors and a peer review process are less likely to publish shoddy scientific articles.

Are you still unsure if you would be able to tell a good article from a bad one? Check out this article to delve deeper into how to use peer reviews and impact factors to assess an article's credibility with peer reviews and impact factors.

Question 2: Are the authors of the study qualified and free of conflicts of interest?

As a clinician, you’ve spent years learning and honing your skills to reach this point in your career. Shouldn’t you expect the authors of the studies you’re relying on to also spend a significant amount of time researching their subject matters?

An expert author is more likely to produce reliable scientific articles. A look into the background of a study’s author can give you an idea of the study’s quality. For example, if you’re interested in a new drug for HIV mentioned in a promising article, it would help to know that the study’s author has spent decades researching infectious diseases.

What if you discovered that the same author holds stocks in the pharmaceutical company that manufactures this new drug? Can you be sure that the author is unbiased in their interpretation for this new treatment, and not simply looking to line their own pockets? This is why it’s important to watch out for any potential conflicts that an author may have.

Conflicts of interest are supposed to be listed at the end of a study article (after the conclusions section). However, a look at the study’s funding is also a good idea. This information is typically found in the acknowledgments section of a paper.

Want to dig deeper into what makes an author credible? We discuss analyzing an author’s background, potential conflicts of interest, and funding in our article on how to assess an author’s credibility .

Question 3: Are the study participants a good representation of the target population?

You’ve probably heard of the common phrase, you can’t compare apples to oranges . Let’s say that you find a case study of a 4-year-old patient exhibiting the same symptoms as a 61-year-old patient. Would you be comfortable relying on the case study findings to determine a treatment protocol for your own patient?

The answer is ultimately up to you. But, taking the time to examine the study’s sample population will help you determine the article’s usefulness in your clinical practice. An article with a study population that closely matches your patient would, of course, be better.

You should also consider whether the article's study population is optimal for the research question being studied. For example, we know that the prevalence of type 2 diabetes is almost two times higher in African Americans than in Caucasian Americans (Marshall Jr 2005). So, a study testing a new diabetes drug that only included Caucasian participants would not be a representative study since a significant portion of the population is missing.

Most of the information on the study’s population demographics (such as age, race, and socioeconomic status) can usually be found in Table 1 of an article.

Do you want to get better at evaluating sample populations? Check out our article on sample populations where we expand on how participant selection, exclusion criteria, inclusion criteria, and how these factors affect an article’s credibility.

Become a great clinician with our video courses and workshops

Question 4: is the study design appropriate for the article’s research question.

Let’s say that you wanted to bake a cake. Would you look up a soup recipe? Probably not! You know that a soup recipe would not give you the information you need. What if the recipe promised a cake but instead gave you the ingredients for soup? The end product would not be the promised (and desired) result. You’d want a cake recipe that included the right ingredients and methods to bake a cake.

Well, in research it’s the same! The correct study design must be used to answer a study question. Otherwise, an article may not properly answer the research question.

For example, if you want to know whether a specific drug works better than a placebo at decreasing blood sugar levels in diabetic patients, a study asking people with diabetes how they feel about taking medicine wouldn’t be helpful. Likewise, if you want to evaluate how people with diabetes feel about increasing the number of medications they take, a study showing blood insulin levels wouldn’t give you an answer.

A good research article will clearly state the research question in the introduction. The methods section should also include steps that detail which research design was chosen, why it was chosen, and the steps that are taken to carry out the design. Reading over these sections will give you an idea of the factors that the authors considered when conducting their study. You can then decide whether they chose the most suitable methods to address the research question. It’s all about making sure that the study can answer the question that you need answered.

So, how do you know if the correct study design was used for a particular study? It helps to know the most common study designs and what types of questions they answer. If you’re feeling a little rusty when it comes to study designs—we’ve got you covered. In our article on study designs , we delve into different research questions and highlight the most commonly used study designs.

Question 5: Are the results interpreted with the correct statistical methods?

When you’re searching through the latest research findings to help your patients, you’re probably looking for an answer to a question. When you find a study with results answering your question, you want to know how confident you can be with those results so you can apply them in the clinic.

Researchers rely on statistical measures like the p value to determine whether the study results are statistically significant. For you to evaluate if the results are clinically significant, you’ll need to wade through the results, interpretations, and basic statistical methods to understand what the findings mean in a clinical setting.

Let’s say that you want to know whether prescribing a novel drug to your patients with the flu would help them recover. You find a study claiming that the novel drug helps patients experiencing flu symptoms. The results show a p value of less than 0.05, which is considered statistically significant when the alpha level is set to 0.05. Sounds great, right?

Well, what if a further look into the results showed that the novel drug only shortened the flu symptom duration by a few hours? How helpful would these results be in the clinical setting?

As clinicians, you’ll need toassess the results and determine whether they are statistically significant and clinically relevant to your patients. In our article on statistical methods , we discuss what a p value really means, and other factors that should be considered when evaluating an article’s results.

Question 6: Are the right conclusions drawn based on the results presented?

Imagine that you come across a scientific study claiming that eating pizza could help you lose weight. The study results showed that participants who ate one slice of pizza lost weight compared to their controls who didn’t eat pizza. Sounds promising, right?

But maybe something isn’t right and there is something else going on in the study! Just because this study found a relationship between eating a slice of pizza and losing weight doesn’t mean that eating the pizza caused the participants to lose weight. Researchers explain this concept as correlation versus causation. Put simply, two variables being correlated does not guarantee that one variable causes the other.

What if there was another reason for the weight loss that had nothing to do with the pizza? Could there be another variable that is affecting the results? Did the authors not account for some of the data, or incorrectly interpret the statistical analyses?

It’s possible for researchers to share their study design, methods, and results in a paper and still draw the wrong conclusions. In fact, it’s also possible that the authors did everything by the book and still drew incorrect conclusions. This is why it’s important to read over the results and determine if the right conclusions are reached.

Want to be more efficient at picking out possible misinterpretations in the data? Our article on data interpretation examines incorrect conclusions that researchers may draw about correlations and causations, missing data, and possible confounders.

Well, there you have it: the top six questions you should ask yourself when reading medical literature.

Carefully assessing the quality of the article and the publication , as well as checking out the author's credibility , will help you select reliable scientific articles. As well, a quick look at the sample populations , study designs , analyses , and conclusions can save you from relying on inaccurate findings for your patients.

With so many articles being published today, it’s become even more important to be critical of the papers you come across. Remember that these questions will help you save time as you wade through the mass of literature. Be sure to check out the other articles in this series for more information on each of these topics.

Are you interested in learning more about epidemiological study designs and statistics, but reading isn’t really your thing? Good news! We’ve got a short video series that answers all the questions and more. Sign up for a free trial and check out our Epidemiology Essentials Course today!

Reference list

• Marshall Jr, MC. 2005. Diabetes in African Americans.  Postgrad Med J .  81 : 734–740.  PMID: 16344294

About the author

Next article

What makes a scientific article credible a look at peer review and impact factors.

Click here to learn the three easy tricks for recognizing credible journal publications and articles.

Evaluating Information Sources

• Evaluate Your Sources
• Publication Types and Bias

Structure of Scientific Papers

Reading a scholarly article, additional reading tips, for more information.

• Reading Scholarly Articles
• Impact Factors and Citation Counts
• Predatory Publishing

Research papers generally follow a specific format. Here are the different parts of the scholarly article.

Abstract (Summary)

The abstract, generally written by the author(s) of the article, provides a concise summary of the whole article. Usually it highlights the focus, study results and conclusion(s) of the article. 

Introduction (Why)

In this section, the authors introduce their topic, explain the purpose of the study, and present why it is important, unique or how it adds to existing knowledge in their field. Look for the author's hypothesis or thesis here. 

Introduction - Literature Review (Who else)

Many scholarly articles include a summary of previous research or discussions published on this topic, called a "Literature Review".  This section outlines what others have found and what questions still remain.

Methodology  / Materials and Methods (How) 

Find the details of how the study was performed in this section. There should be enough specifics so that you could repeat the study if you wanted. 

Results   (What happened)

This section includes the findings from the study. Look for the data and statistical results in the form of tables, charts, and graphs. Some papers include an analysis here.

Discussion  / Analysis  (What it means)

This section should tell you what the authors felt was significant about their results. The authors analyze their data and describe what they believe it means.

Conclusion (What was learned)

Here the authors offer their final thoughts and conclusions and may include: how the study addressed their hypothesis, how it contributes to the field, the strengths and weaknesses of the study, and recommendations for future research. Some papers combine the discussion and conclusion.

A scholarly paper can be difficult to read. Instead of reading straight through, try focusing on the different sections and asking specific questions at each point.

What is your research question? 

When you select an article to read for a project or class, focus on your topic. Look for information in the article that is relevant to your research question. 

Read the abstract first  as it covers basics of the article. Questions to consider: 

• What is this article about? What is the working hypothesis or thesis?
• Is this related to my question or area of research?

Second: Read the introduction and discussion/conclusion.  These sections offer the main argument and hypothesis of the article. Questions to consider for the introduction: 

• What do we already know about this topic and what is left to discover?
• What have other people done in regards to this topic?
• How is this research unique?
• Will this tell me anything new related to my research question?

Questions for the discussion and conclusion: 

• What does the study mean and why is it important?
• What are the weaknesses in their argument?
• Is the conclusion valid?

Next: Read about the Methods/Methodology.  If what you've read addresses your research question, this should be your next section. Questions to consider:

• How did the author do the research? Is it a qualitative or quantitative project?
• What data are the study based on?
• Could I repeat their work? Is all the information present in order to repeat it?

Finally: Read the Results and Analysis.  Now read the details of this research. What did the researchers learn? If graphs and statistics are confusing, focus on the explanations around them. Questions to consider: 

• What did the author find and how did they find it?
• Are the results presented in a factual and unbiased way?
• Does their analysis agree with the data presented?
• Is all the data present?
• What conclusions do you formulate from this data? (And does it match with the Author's conclusions?)

Review the References (anytime): These give credit to other scientists and researchers and show you the basis the authors used to develop their research.  The list of references, or works cited, should include all of the materials the authors used in the article. The references list can be a good way to identify additional sources of information on the topic. Questions to ask:

• What other articles should I read?
• What other authors are respected in this field?
• What other research should I explore?

When you read these scholarly articles, remember that you will be writing based on what you read.

While you are Reading:

• Keep in mind your research question
• Focus on the information in the article relevant to your question (feel free to skim over other parts)
• Question everything you read - not everything is 100% true or performed effectively
• Think critically about what you read and seek to build your own arguments
• Read out of order! This isn't a mystery novel or movie, you want to start with the spoiler
• Use any keywords printed by the journals as further clues about the article
• Look up words you don't know

How to Take Notes on the Article

Try different ways, but use the one that fits you best. Below are some suggestions:

• Print the article and highlight, circle and otherwise mark while you read (for a PDF, you can use the highlight text  feature in Adobe Reader)
• Take notes on the sections, for example in the margins (Adobe Reader offers pop-up  sticky notes )
• Highlight only very important quotes or terms - or highlight potential quotes in a different color
• Summarize the main or key points

Reflect on what you have read - draw your own conclusions . As you read jot down questions that come to mind. These may be answered later on in the article or you may have found something that the authors did not consider. Here are a few questions that might be helpful:

• Have I taken time to understand all the terminology?
• Am I spending too much time on the less important parts of this article?
• Do I have any reason to question the credibility of this research?
• What specific problem does the research address and why is it important?
• How do these results relate to my research interests or to other works which I have read?
• Anatomy of a Scholarly Article (Interactive tutorial) Andreas Orphanides, North Carolina State University Libraries, 2009
• How to Read an Article in a Scholarly Journal (Research Guide) Cayuga Community College Library, 2016
• How To Read a Scholarly Journal Article (YouTube Video) Tim Lockman, Kishwaukee College Library, 2012.
• How To Read a Scientific Paper (Interactive tutorial) Michael Fosmire, Purdue University Libraries, 2013. PDF
• How to Read a Scientific Paper (Online article) Science Buddies, 2012
• How to Read a Scientific Research Paper (Article) Durbin Jr., C. G. Respiratory Care, 2009
• The Illusion of Certainty and the Certainty of Illusion: A Caution when Reading Scientific Articles (Article) T. A. Lang, International Journal of Occupational and Environmental Medicine, 2011,
• Infographic: How to Read Scientific Papers Natalia Rodriguez, Elsevier, 2015
• Library Research Methods: Read & Evaluate Culinary Institute of America Library, 2016
• << Previous: Publication Types and Bias
• Next: Impact Factors and Citation Counts >>
• Last Updated: Mar 8, 2024 1:17 PM
• URL: https://libguides.usc.edu/evaluate

Child Care and Early Education Research Connections

Key questions to ask.

This section outlines key questions to ask in assessing the quality of research and describes internal validity, external validity, and construct validity.

Was the research peer reviewed?

Peer reviewed research studies have already been evaluated by experienced researchers with relevant expertise. Most journal articles, books and government reports have gone through a peer review process. Keep in mind that there are many types of peer reviews. Reports issued by the federal government have been subject to many levels of internal review and approval before being issued. Articles published in professional journals with peer review have been evaluated by researchers that are experts in the field and who can vouch for the soundness of the methodology and the analysis applied. As a result, peer-reviewed research is usually of high quality. A research consumer, however, should still critically evaluate the study's methodology and conclusions.

Can a study's quality be evaluated with the information provided?

Every study should include a description of the population of interest, an explanation of the process used to select and gather data on study subjects, definitions of key variables and concepts, descriptive statistics for main variables, and a description of the analytic techniques. Research consumers should be cautious when drawing conclusions from studies that do not provide sufficient information about these key research components.

Are there any potential threats to the study's validity?

A valid study answers research questions in a scientifically rigorous manner. Threats to a study's validity are found in three areas: 

Internal Validity

External validity, construct validity.

Internal Validity refers to whether the outcomes observed in a study are due to the independent variables or experimental manipulations investigated in the study and not to some other factor or set of factors. To determine whether a research study has internal validity, a research consumer should ask whether changes in the outcome could be attributed to alternative explanations that are not explored in the study. For example, a study may show that a new curriculum had a significant positive effect on children's reading comprehension.

The study must rule out alternative explanations for the increase in reading comprehension, such as a new teacher, in order to attribute the increase in reading comprehension to the new curriculum. Studies that specifically explain how alternative explanations were ruled out are more likely to have internal validity. Threats to a study's internal validity can compromise the confidence consumers have in the findings from a study and include:

The introduction of events while the study is being conducted that may affect the outcome or dependent variable of the study. For example, while studying the effectiveness of children's participation in an early childhood program, the program was closed for an extended period of time due to damage from a hurricane.

Changes in the dependent variable due to normal developmental processes in study participants. For example, young children's performance on a battery of outcome measures (e.g., reading and math assessments) may decline during the testing or observation period due to fatigue or other factors.

The circumstances around the testing that is used to assess the dependent variable. For example, preschool children's performance on a standardized test may be questionable if test items are presented to children in unfamiliar ways or in group settings.

Participants leaving or dropping out of the study before it is completed. This can be especially problematic if those who leave the study are different from those who stay. For example, in a longitudinal study of the effects of a school lunch program on children's academic achievement, the validity of the findings could be problematic if the most disadvantaged children in the program left the study at a higher rate than other children.

Changes to or inconsistencies in how the dependent and independent variables were measured. For example, changing the way in which children's math skills are measured at two time points could introduce error if the two measures were developed using different assessment frameworks (i.e., they were developed to assess different math content and processes). Inconsistencies are also introduced when different staff follow different procedures when administering the same measure. For example, when administering an assessment to bilingual children, some staff give children credit for answering correctly in English or Spanish, and other staff only give credit for answering correctly in English.

Statistical regression or regression to the mean can affect the outcome of a study. It is the movement of test scores (post-test scores) toward the mean (average score), independent of any effect of an independent variable. It is especially a concern when assessing the skills of low performing individuals and comparing their skills to those with average or above average performance. For example, kindergarten children with the weakest reading skills at the start of the school year may show the greatest gains in their skills over the school year (e.g., between fall and spring assessments) independent of the instruction they received from their teachers.

External Validity refers to the extent to which the results of a study can be generalized to other settings (ecological validity), other people (population validity) and over time (historical validity). To assess whether a study has external validity, a research consumer should ask whether the findings apply to individuals whose place and circumstances differ from those of study participants. For example, a research study shows that a new curriculum improved reading comprehension of third-grade children in Iowa. As a research consumer, you want to ask whether this new curriculum may also be effective with third graders in New York or with children in other elementary grades. Studies that randomly select participants from the most diverse and representative populations and that are conducted in natural settings are more likely to have external validity. Threats to a study's external validity come from several sources, including:

The sample is not representative of the population of interest. As a result, findings from the study may be biased (sample selection bias) and do not accurately represent the population. Several factors can lead to a sample not being representative of the population.

The list of all those in the population who are eligible to be sampled is incomplete or contains duplicates. For example, in a household survey, the list of housing units from which the sample will be drawn may be missing housing units (e.g., one or the two housing units in a duplex home). Or, an address list that will be used to drawn a sample may have some households listed twice.

Some members of the population or members of certain groups may not be adequately represented in the sample (undercoverage). For example, a survey of adult education that relies on a published list of telephone numbers to select its sample may not get an accurate estimate of the participation of adults in different education programs because young adults who have higher rates of participation are less likely to have landlines and to have numbers published.

Not all individuals who are sampled agree to participate in the study. When those who participate are different in meaningful ways from those who do not, there is the potential for the findings from the study to be biased (nonresponse bias). That is, the findings may not represent an accurate picture of the total population.

Selecting samples using non-probability methods (e.g., purposive sample, volunteer samples), which tend to over- or under-represent certain groups in the population. For example, volunteer surveys on controversial topics such as school vouchers and sex education are more likely to overrepresent individuals with strong opinions. And, shopping mall surveys in general only represent the small group of individuals who are shopping at a particular location and at specific times.

The findings from one study are difficult to replicate across locations, groups, and time. Despite the best efforts, it is extremely difficult to introduce and implement a program (treatment) exactly the same way in different locations. Similarly, it is difficult to conduct a study the same way each time. While researchers have control over many features of their studies, there are factors that are beyond their control (e.g., willingness of potential subjects to participate, scheduling conflicts that could lead to cancellations of data collection activities, data collection being suspended due to natural disasters). For example, the ability to carry out a study of school-age children's reading and math achievement in one school or in one school district may be affected by teachers' willingness to surrender instructional time for students to participate in a series of standardized assessments. In some cases, modifications to the study design (e.g., shorten the assessment, limit sensitive questions on a teacher or parent survey) must be made to accommodate the concerns of school and district leaders.

Changes in the behaviors and reported attitudes of study participants as a result of being included in a research study (Hawthorne effect). For example, parents participating in a research study on children's early development may change the ways in which they support their child's learning at home.

Construct Validity refers to the degree to which a variable, test, questionnaire or instrument measures the theoretical concept that the researcher hopes to measure. To assess whether a study has construct validity, a research consumer should ask whether the study has adequately measured the key concepts in the study. For example, a study of reading comprehension should present convincing evidence that reading tests do indeed measure reading comprehension. Studies that use measures that have been independently validated in prior studies are more likely to have construct validity.

There are many threats to construct validity. These can arise during: the planning and design stage, assessment or survey administration, and data processing and analysis. Some are attributed to researchers and others to the subjects of the research. Here are some of the more common threats:

Threats that occur during the planning and design stage include:

Poorly defined constructs are perhaps the largest threat to construct validity. This applies to constructs that are too narrowly defined as well as those that are defined too broadly.

Validity can also be affected by the measures a researcher chooses to measure a construct. Measures that include too few items to adequately represent the construct pose a threat as do measures that include items that tap other constructs. For example, a math assessment administered to four- and five-year old children that only includes items that require children to count would not be adequate to represent their math skills. A math assessment administered to this same group of children that was made up mostly of word problems would be tapping both their math and language skills. A valid measure should cover all aspects of the theoretical construct and only aspects of the theoretical construct.

Assessment items or survey questions that are poorly written are threats to validity. Such items would include double-barreled questions that ask multiple questions within a single item (e.g., are you happily married and do you and your spouse argue?). Other examples of poorly written questions include those that use language that is above the reading level of most respondents, use professional jargon or are written in such a way as to trigger a socially desirable response.

The validity of an assessment is threatened if there are too many items that are outside the ability of the individual being assessed (e.g., too many very easy items and too many very difficult items). For example, an early literacy assessment that only included passages that children were asked to read and answer questions about would not result in a valid assessment of children's early literacy.

Threats that occur during administration include:

Threats that are introduced by interviewers and assessment staff. Actions by these individuals that can affect the reliability and thus the validity of the assessment occur when they deviate from the research protocol and when they signal a correct answer to the study participant through their actions. For example, an assessment of young children's English language vocabulary may specify that only responses in English are acceptable. However, when assessing bilingual children, some assessors comply with this rule while others accept responses in English or in the child's home language (e.g., Spanish). Assessors may unintentionally signal to children the correct responses on an assessment by 'staring' at the correct response to a multiple-choice item or by smiling and giving praise only when the child answers correctly.

Threats to validity can also be introduced by the research participants. These would include participant apprehension or anxiety that could result in poorer performance on an assessment or to incorrect or ambiguous responses to a series of interview items. These threats must be taken seriously and addressed when administering standardized assessments to young children, many of whom will have limited experience with these types of tests. The language used when administering an assessment can also threaten its validity, if subjects do not have the language skills to understand what they are being asked to do and the language skills needed to respond.

Threats that occur during data processing and analysis include:

Coding errors - Coding errors that are systematic as compared to those that are random are especially problematic.

Poor inter-coder or inter-rater reliability - When coding responses to open-ended survey items or assigning scores to behaviors observed during a video interaction, it is important that different coders or raters assign the same code or score for the same response or behavior. That is, the goal is high inter-coder or inter-rater reliability. When inter-coder or inter-rater reliability is poor, it can have an adverse effect on the validity of a measure. For example, the construct validity of an observation measure of the quality of parent-child interactions could be compromised should individual members of a group coding a set of videotaped mother-child interactions apply different standards as to what they deem as intrusive parenting practices.

Inconsistencies in how data are analyzed and missing data handled - Missing data may be handled in a number of different ways, and the approach that is chosen could prove to be problematic for a construct, especially when the data are not missing at random. For example, if items tapping certain math skills are missing disproportionately, the validity of the measure could be jeopardized if a researcher assigns the mean score for those items or if he simply averages the scores for the non-missing items.

Top 10 Questions for a Complete Literature Review

An excellent literature review integrates information in such a way that it provides a new framework to build upon. It is a way of contextualizing your work and showcasing a bigger picture before you pin down to your research problem. It not only highlights principle issues in your field but also provides new perspectives on the research topic. Careful skimming of literature introduces the readers to relevant terminologies frequently used in context of their work. Literature review assists in recognizing related research findings and relevant theories. Furthermore, it aids in pinpointing the methodologies that one may adopt for research.

5 Steps to Begin the Literature Review

There are five steps that one should follow before preparing to conduct the literature review :

• Identify all the literature relevant to your topic of interest. Explore all the different types of literature including theoretical literature, applied literature, literature that talks about research methods, or a combination thereof.
• Using multiple keywords and strategies capture the most accurate and relevant data. Conduct an extensive search in multi-disciplinary databases.
• Group your findings into a detailed summary of what is known and what needs to be explored.
• Identify existing gaps or any unresolved issues
• Formulate broad questions that warrant further research

How to Best Critique a Research paper

For extracting maximum information from a research paper , researchers must ask the following questions!

• Has the author formulated an appropriate research question based on the problem/issue?
• Is the research question clearly defined in terms of its scope and relevance?
• Was there an alternative or better perspective to approach the research question?
• What is the author’s orientation towards the research problem – is it a critical analysis or interpretation based?
• Has the author extensively evaluated the literature considering both latest and relevant articles?
• How has the author defined the basic components (population, interventions, outcomes) of the study? Are the measurements valid, accurate and statistically significant? Are the conclusions based accurate interpretations of the data?
• Is there an objective based, unbiased reasoning provided for the problem statement or is the author merely attempting to prove his/her preconceived beliefs and opinions?
• How does this article contribute to your understanding of the research problem?
• What are the strengths, limitations and shortcomings of the study?

10 Questions for a Comprehensive Literature Review

1. Do I have clearly defined research aims prior to commencing the review?

It is important to choose a focused question that can efficiently direct your search. It can assist you to create a list of keywords related to your research problem. Furthermore, it helps in identifying relevant databases to search for related journals and articles.

2. Have I correctly identified all the sources that will help me define my problem statement or research question?

Literature is not limited to journal articles, thesis, and dissertations. One should also refer to credible internet sources, conference proceedings that provide latest unpublished papers, as well as government and corporate reports. Books, although do not have latest information, can serve as a good starting point to read background information.

3. Have I considered all kinds of literature – including both qualitative and quantitative research articles?

An exhaustive literature survey helps you position your research within the context of existing literature effectively creating a case as to why further study is necessary. Your search has to be robust enough to ensure that you have browsed through all the relevant and latest articles. Rather than reading everything, researchers must refer and follow the most relevant work!

4. Do I have enough empirical or theoretical evidence to support my hypothesis?

Discovering new patterns and trends becomes easy if you gather credible evidence from earlier works. Furthermore, it helps in rationalizing the significance of your study.

5. Have I identified all the major inconsistencies or other shortcomings related to my research topic?

Researchers should not only refer to articles that present supporting evidence but also focus on those that provide inconclusive or contradictory information. It helps to identify any open questions left by researchers in previous studies.

6. Is my relationship diagram ready?

A relationship diagram is an effective way of recognizing links between different elements of a complex research topic. It is an immensely important tool that helps in clarifying and structuring research specific findings and interpretations at various stages of the project. It is an effective way of representing your current understanding of the research topic. In addition, a good relationship diagram can help you find new insights owing to a clear picture of all the probable relationships between key concepts, variables and key factors.

7. Have I gathered sufficient evidence from the literature about the accuracy and validity of the designs or methods that I plan to use in my experiments?

It is paramount to use methodologies and research techniques that have scientific reliability. Moreover, since methods especially used in qualitative research are often more subjective, it becomes crucial for researchers to reflect on the approach and explain the criteria for selecting a particular method.

8. Have I identified the purpose for which articles have been shortlisted for literature review?

You can expedite your literature writing process if you tag your articles based on its purpose of inclusion in the review report. Following are the tags that can be added to articles:

• Show how latest developments or develop a theoretical base to your study
• Demonstrate limitations, inconsistencies or shortcomings of previous studies
• Critique or support certain methods or findings
• Replicate the study in a different setting (region/population)
• Indicate how the study supports or contradicts your findings
• Use it as a reference to further build your research
• Provide a general understanding of concerns relevant to your research topic

9. Have I recorded all the bibliographic information regarding my information sources?

Recording and cataloguing your bibliographical details and references is absolutely crucial for every researcher. You may use commercial software such as Reference manager, End Note, and Pro Cite to manage your references. Furthermore, you may also keep a record of keyword searches that you have performed.

10. Will my literature review reflect a report that is created after a through critical analysis of the literature?

An excellent literature review must be structured, logical, and coherent. It is a great opportunity to demonstrate that you have critically analyzed and understood the relevant body of literature underpinning your research. It is important to structure your literature into appropriate sections that discuss themes or presents trends. Grouping your literature helps in indicating relationships and making comparisons.

Still have more queries related to literature review and synthesis? Post your queries here and our experts will be happy to answer them! You can also visit our Q&A forum for frequently asked questions related to research writing and publishing answered by our team that comprises subject-matter experts, eminent researchers, and publication experts.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Reporting Research

AI Assistance in Academia for Searching Credible Scholarly Sources

The journey of academia is a grand quest for knowledge, more specifically an adventure to…

Beyond Spellcheck: How copyediting guarantees error-free submission

Submitting a manuscript is a complex and often an emotional experience for researchers. Whether it’s…

• Old Webinars
• Webinar Mobile App

How to Find the Right Journal and Fix Your Manuscript Before Submission

Selection of right journal Meets journal standards Plagiarism free manuscripts Rated from reviewer's POV

• Manuscripts & Grants

Research Aims and Objectives: The dynamic duo for successful research

Picture yourself on a road trip without a destination in mind — driving aimlessly, not…

How Academic Editors Can Enhance the Quality of Your Manuscript

Avoiding desk rejection Detecting language errors Conveying your ideas clearly Following technical requirements

Writing a Research Literature Review? — Here are tips to guide you through!

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What should universities' stance be on AI tools in research and academic writing?

Evaluating Information

• Understanding Primary and Secondary Sources
• Exploring and Evaluating Popular, Trade, and Scholarly Sources

Reading a Scholarly Article

Common components of original research articles, while you read, reading strategies, reading for citations, further reading, learning objectives.

This page was created to help you:

Identify the different parts of a scholarly article

Efficiently analyze and evaluate scholarly articles for usefulness

This page will focus on reading scholarly articles — published reports on original research in the social sciences, humanities, and STEM fields. Reading and understanding this type of article can be challenging. This guide will help you develop these skills, which can be learned and improved upon with practice.

We will go over:

There are many different types of articles that may be found in scholarly journals and other academic publications. For more, see:

• Types of Information Sources

Reading a scholarly article isn’t like reading a novel, website, or newspaper article. It’s likely you won’t read and absorb it from beginning to end, all at once.

Instead, think of scholarly reading as inquiry, i.e., asking a series of questions as you do your research or read for class. Your reading should be guided by your class topic or your own research question or thesis.

For example, as you read, you might ask yourself:

• What questions does it help to answer, or what topics does it address?
• Are these relevant or useful to me?
• Does the article offer a helpful framework for understanding my topic or question (theoretical framework)?
• Do the authors use interesting or innovative methods to conduct their research that might be relevant to me?
• Does the article contain references I might consult for further information?

In Practice

Scanning and skimming are essential when reading scholarly articles, especially at the beginning stages of your research or when you have a lot of material in front of you.

Many scholarly articles are organized to help you scan and skim efficiently. The next time you need to read an article, practice scanning the following sections (where available) and skim their contents:

• The abstract: This summary provides a birds’ eye view of the article contents.
• The introduction:  What is the topic(s) of the research article? What is its main idea or question?
• The list of keywords or descriptors
• Methods: How did the author(s) go about answering their question/collecting their data?
• Section headings:  Stop and skim those sections you may find relevant.
• Figures:  Offer lots of information in quick visual format.
• The conclusion:  What are the findings and/or conclusions of this article?

Mark Up Your Text

Read with purpose.

• Scanning and skimming with a pen in hand can help to focus your reading.
• Use color for quick reference. Try highlighters or some sticky notes. Use different colors to represent different topics.
• Write in the margins, putting down thoughts and questions about the content as you read.
• Use digital markup features available in eBook platforms or third-party solutions, like Adobe Reader or Hypothes.is.

Categorize Information

Create your own informal system of organization. It doesn’t have to be complicated — start basic, and be sure it works for you.

• Jot down a few of your own keywords for each article. These keywords may correspond with important topics being addressed in class or in your research paper.  
• Write keywords on print copies or use the built-in note taking features in reference management tools like Zotero and EndNote.  
• Your keywords and system of organization may grow more complex the deeper you get into your reading.

Highlight words, terms, phrases, acronyms, etc. that are unfamiliar to you. You can highlight on the text or make a list in a notetaking program.

• Decide if the term is essential to your understanding of the article or if you can look it up later and keep scanning.

You may scan an article and discover that it isn’t what you thought it was about. Before you close the tab or delete that PDF, consider scanning the article one more time, specifically to look for citations that might be more on-target for your topic.  

You don’t need to look at every citation in the bibliography — you can look to the literature review to identify the core references that relate to your topic. Literature reviews are typically organized by subtopic within a research question or thesis. Find the paragraph or two that are closely aligned with your topic, make note of the author names, then locate those citations in the bibliography or footnote.

See the Find Articles page for what to do next:

• Find Articles

See the Citation Searching page for more on following a citation trail:

• Citation Searching
• Taking notes effectively. [blog post] Raul Pacheco-Vega, PhD
• How to read an academic paper. [video] UBCiSchool. 2013
• How to (seriously) read a scientific paper. (2016, March 21). Science | AAAS.
• How to read a paper. S. Keshav. 2007. SIGCOMM Comput. Commun. Rev. 37, 3 (July 2007), 83–84.

This guide was designed to help you:

• << Previous: Exploring and Evaluating Popular, Trade, and Scholarly Sources
• Last Updated: Feb 16, 2024 3:55 PM
• URL: https://libguides.brown.edu/evaluate

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

• YJBM Updates
• Editorial Board
• Colloquium Series
• Podcast Series
• Open Access
• Call for Papers
• PubMed Publications
• News & Views
• YJBM Podcasts (by topic)
• About YJBM PodSquad
• Why Publish in YJBM?
• Types of Articles Published
• Manuscript Submission Guidelines
• YJBM Ethical Guidelines

Points to Consider When Reviewing Articles

• Writing and Submitting a Review
• Join Our Peer Reviewer Database

INFORMATION FOR

• Residents & Fellows
• Researchers

General questions that Reviewers should keep in mind when reviewing articles are the following:

• Is the article of interest to the readers of YJBM ?
• What are the strengths and weaknesses of the manuscript?
• How can the Editors work with the Authors to improve the submitted manuscripts, if the topic and scope of the manuscript is of interest to YJBM readers?

The following contains detailed descriptions as to what should be included in each particular type of article as well as points that Reviewers should keep in mind when specifically reviewing each type of article.

YJBM will ask Reviewers to Peer Review the following types of submissions:

Download PDF

Frequently asked questions.

These manuscripts should present well-rounded studies reporting innovative advances that further knowledge about a topic of importance to the fields of biology or medicine. The conclusions of the Original Research Article should clearly be supported by the results. These can be submitted as either a full-length article (no more than 6,000 words, 8 figures, and 4 tables) or a brief communication (no more than 2,500 words, 3 figures, and 2 tables). Original Research Articles contain five sections: abstract, introduction, materials and methods, results and discussion.

Reviewers should consider the following questions:

• What is the overall aim of the research being presented? Is this clearly stated?
• Have the Authors clearly stated what they have identified in their research?
• Are the aims of the manuscript and the results of the data clearly and concisely stated in the abstract?
• Does the introduction provide sufficient background information to enable readers to better understand the problem being identified by the Authors?
• Have the Authors provided sufficient evidence for the claims they are making? If not, what further experiments or data needs to be included?
• Are similar claims published elsewhere? Have the Authors acknowledged these other publications? Have the Authors made it clear how the data presented in the Author’s manuscript is different or builds upon previously published data?
• Is the data presented of high quality and has it been analyzed correctly? If the analysis is incorrect, what should the Authors do to correct this?
• Do all the figures and tables help the reader better understand the manuscript? If not, which figures or tables should be removed and should anything be presented in their place?
• Is the methodology used presented in a clear and concise manner so that someone else can repeat the same experiments? If not, what further information needs to be provided?
• Do the conclusions match the data being presented?
• Have the Authors discussed the implications of their research in the discussion? Have they presented a balanced survey of the literature and information so their data is put into context?
• Is the manuscript accessible to readers who are not familiar with the topic? If not, what further information should the Authors include to improve the accessibility of their manuscript?
• Are all abbreviations used explained? Does the author use standard scientific abbreviations?

Case reports describe an unusual disease presentation, a new treatment, an unexpected drug interaction, a new diagnostic method, or a difficult diagnosis. Case reports should include relevant positive and negative findings from history, examination and investigation, and can include clinical photographs. Additionally, the Author must make it clear what the case adds to the field of medicine and include an up-to-date review of all previous cases. These articles should be no more than 5,000 words, with no more than 6 figures and 3 tables. Case Reports contain five sections: abstract; introduction; case presentation that includes clinical presentation, observations, test results, and accompanying figures; discussion; and conclusions.

• Does the abstract clearly and concisely state the aim of the case report, the findings of the report, and its implications?
• Does the introduction provide enough details for readers who are not familiar with a particular disease/treatment/drug/diagnostic method to make the report accessible to them?
• Does the manuscript clearly state what the case presentation is and what was observed so that someone can use this description to identify similar symptoms or presentations in another patient?
• Are the figures and tables presented clearly explained and annotated? Do they provide useful information to the reader or can specific figures/tables be omitted and/or replaced by another figure/table?
• Are the data presented accurately analyzed and reported in the text? If not, how can the Author improve on this?
• Do the conclusions match the data presented?
• Does the discussion include information of similar case reports and how this current report will help with treatment of a disease/presentation/use of a particular drug?

Reviews provide a reasoned survey and examination of a particular subject of research in biology or medicine. These can be submitted as a mini-review (less than 2,500 words, 3 figures, and 1 table) or a long review (no more than 6,000 words, 6 figures, and 3 tables). They should include critical assessment of the works cited, explanations of conflicts in the literature, and analysis of the field. The conclusion must discuss in detail the limitations of current knowledge, future directions to be pursued in research, and the overall importance of the topic in medicine or biology. Reviews contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Is the review accessible to readers of YJBM who are not familiar with the topic presented?
• Does the abstract accurately summarize the contents of the review?
• Does the introduction clearly state what the focus of the review will be?
• Are the facts reported in the review accurate?
• Does the Author use the most recent literature available to put together this review?
• Is the review split up under relevant subheadings to make it easier for the readers to access the article?
• Does the Author provide balanced viewpoints on a specific topic if there is debate over the topic in the literature?
• Are the figures or tables included relevant to the review and enable the readers to better understand the manuscript? Are there further figures/tables that could be included?
• Do the conclusions and outlooks outline where further research can be done on the topic?

Perspectives provide a personal view on medical or biomedical topics in a clear narrative voice. Articles can relate personal experiences, historical perspective, or profile people or topics important to medicine and biology. Long perspectives should be no more than 6,000 words and contain no more than 2 tables. Brief opinion pieces should be no more than 2,500 words and contain no more than 2 tables. Perspectives contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Does the abstract accurately and concisely summarize the main points provided in the manuscript?
• Does the introduction provide enough information so that the reader can understand the article if he or she were not familiar with the topic?
• Are there specific areas in which the Author can provide more detail to help the reader better understand the manuscript? Or are there places where the author has provided too much detail that detracts from the main point?
• If necessary, does the Author divide the article into specific topics to help the reader better access the article? If not, how should the Author break up the article under specific topics?
• Do the conclusions follow from the information provided by the Author?
• Does the Author reflect and provide lessons learned from a specific personal experience/historical event/work of a specific person?

Analyses provide an in-depth prospective and informed analysis of a policy, major advance, or historical description of a topic related to biology or medicine. These articles should be no more than 6,000 words with no more than 3 figures and 1 table. Analyses contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Does the abstract accurately summarize the contents of the manuscript?
• Does the introduction provide enough information if the readers are not familiar with the topic being addressed?
• Are there specific areas in which the Author can provide more detail to help the reader better understand the manuscript? Or are there places where the Author has provided too much detail that detracts from the main point?

Profiles describe a notable person in the fields of science or medicine. These articles should contextualize the individual’s contributions to the field at large as well as provide some personal and historical background on the person being described. More specifically, this should be done by describing what was known at the time of the individual’s discovery/contribution and how that finding contributes to the field as it stands today. These pieces should be no more than 5,000 words, with up to 6 figures, and 3 tables. The article should include the following: abstract, introduction, topics (with headings and subheadings), and conclusions.

• Does the Author provide information about the person of interest’s background, i.e., where they are from, where they were educated, etc.?
• Does the Author indicate how the person focused on became interested or involved in the subject that he or she became famous for?
• Does the Author provide information on other people who may have helped the person in his or her achievements?
• Does the Author provide information on the history of the topic before the person became involved?
• Does the Author provide information on how the person’s findings affected the field being discussed?
• Does the introduction provide enough information to the readers, should they not be familiar with the topic being addressed?

Interviews may be presented as either a transcript of an interview with questions and answers or as a personal reflection. If the latter, the Author must indicate that the article is based on an interview given. These pieces should be no more than 5,000 words and contain no more than 3 figures and 2 tables. The articles should include: abstract, introduction, questions and answers clearly indicated by subheadings or topics (with heading and subheadings), and conclusions.

• Does the Author provide relevant information to describe who the person is whom they have chosen to interview?
• Does the Author explain why he or she has chosen the person being interviewed?
• Does the Author explain why he or she has decided to focus on a specific topic in the interview?
• Are the questions relevant? Are there more questions that the Author should have asked? Are there questions that the Author has asked that are not necessary?
• If necessary, does the Author divide the article into specific topics to help the reader better access the article? If not, how should the author break up the article under specific topics?
• Does the Author accurately summarize the contents of the interview as well as specific lesson learned, if relevant, in the conclusions?

Greater Good Science Center • Magazine • In Action • In Education

Big Ideas Articles & More

10 questions to ask about scientific studies, never take a study at face value, including one you read about in greater good .

Here at Greater Good , we cover research into social and emotional well-being, and we try to help people apply findings to their personal and professional lives. We are well aware that our business is a tricky one.

Summarizing scientific studies and applying them to people’s lives isn’t just difficult for the obvious reasons, like understanding and then explaining scientific jargon or methods to non-specialists. It’s also the case that context gets lost when we translate findings into stories, tips, and tools for a more meaningful life, especially when we push it all through the nuance-squashing machine of the Internet. Many people never read past the headlines, which intrinsically aim to overgeneralize and provoke interest. Because our articles can never be as comprehensive as the original studies, they almost always omit some crucial caveats, such as limitations acknowledged by the researchers. To get those, you need access to the studies themselves.

And it’s very common for findings to seem to contradict each other. For example, we recently covered an experiment that suggests stress reduces empathy—after having previously discussed other research suggesting that stress-prone people can be more empathic. Some readers asked: Which one is correct? (You’ll find my answer here .)

But probably the most important missing piece is the future. That may sound like a funny thing to say, but, in fact, a new study is not worth the PDF it’s printed on until its findings are replicated and validated by other studies—studies that haven’t yet happened. An experiment is merely interesting until time and testing turns its finding into a fact.

Scientists know this, and they are trained to react very skeptically to every new paper. They also expect to be greeted with skepticism when they present findings. Trust is good, but science isn’t about trust. It’s about verification.

However, journalists like me, and members of the general public, are often prone to treat every new study as though it represents the last word on the question addressed. This particular issue was highlighted last week by—wait for it—a new study that tried to reproduce 100 prior psychological studies to see if their findings held up. The result of the three-year initiative is chilling: The team, led by University of Virginia psychologist Brian Nosek, got the same results in only 36 percent of the experiments they replicated. This has led to some predictably provocative, overgeneralizing headlines implying that we shouldn’t take psychology seriously.

I don’t agree.

Despite all the mistakes and overblown claims and criticism and contradictions and arguments—or perhaps because of them—our knowledge of human brains and minds has expanded dramatically during the past century. Psychology and neuroscience have documented phenomena like cognitive dissonance, identified many of the brain structures that support our emotions, and proved the placebo effect and other dimensions of the mind-body connection, among other findings that have been tested over and over again.

These discoveries have helped us understand and treat the true causes of many illnesses. I’ve heard it argued that rising rates of diagnoses of mental illness constitute evidence that psychology is failing, but in fact, the opposite is true: We’re seeing more and better diagnoses of problems that would have compelled previous generations to dismiss people as “stupid” or “crazy” or “hyper” or “blue.” The important thing to bear in mind is that it took a very, very long time for science to come to these insights and treatments, following much trial and error.

Science isn’t a faith, but rather a method that takes time to unfold. That’s why it’s equally wrong to uncritically embrace everything you read, including what you are reading on this page.

Given the complexities and ambiguities of the scientific endeavor, is it possible for a non-scientist to strike a balance between wholesale dismissal and uncritical belief? Are there red flags to look for when you read about a study on a site like Greater Good or in a popular self-help book? If you do read one of the actual studies, how should you, as a non-scientist, gauge its credibility?

I drew on my own experience as a science journalist, and surveyed my colleagues here at the UC Berkeley Greater Good Science Center. We came up 10 questions you might ask when you read about the latest scientific findings. These are also questions we ask ourselves, before we cover a study.

1. Did the study appear in a peer-reviewed journal?

Peer review—submitting papers to other experts for independent review before acceptance—remains one of the best ways we have for ascertaining the basic seriousness of the study, and many scientists describe peer review as a truly humbling crucible. If a study didn’t go through this process, for whatever reason, it should be taken with a much bigger grain of salt.

2. Who was studied, where?

Animal experiments tell scientists a lot, but their applicability to our daily human lives will be limited. Similarly, if researchers only studied men, the conclusions might not be relevant to women, and vice versa.

This was actually a huge problem with Nosek’s effort to replicate other people’s experiments. In trying to replicate one German study, for example, they had to use different maps (ones that would be familiar to University of Virginia students) and change a scale measuring aggression to reflect American norms. This kind of variance could explain the different results. It may also suggest the limits of generalizing the results from one study to other populations not included within that study.

As a matter of approach, readers must remember that many psychological studies rely on WEIRD (Western, educated, industrialized, rich and democratic) samples, mainly college students, which creates an in-built bias in the discipline’s conclusions. Does that mean you should dismiss Western psychology? Of course not. It’s just the equivalent of a “Caution” or “Yield” sign on the road to understanding.

3. How big was the sample?

In general, the more participants in a study, the more valid its results. That said, a large sample is sometimes impossible or even undesirable for certain kinds of studies. This is especially true in expensive neuroscience experiments involving functional magnetic resonance imaging, or fMRI, scans.

And many mindfulness studies have scanned the brains of people with many thousands of hours of meditation experience—a relatively small group. Even in those cases, however, a study that looks at 30 experienced meditators is probably more solid than a similar one that scanned the brains of only 15.

4. Did the researchers control for key differences?

Diversity or gender balance aren’t necessarily virtues in a research study; it’s actually a good thing when a study population is as homogenous as possible, because it allows the researchers to limit the number of differences that might affect the result. A good researcher tries to compare apples to apples, and control for as many differences as possible in her analysis.

5. Was there a control group?

One of the first things to look for in methodology is whether the sample is randomized and involved a control group; this is especially important if a study is to suggest that a certain variable might actually cause a specific outcome, rather than just be correlated with it (see next point).

For example, were some in the sample randomly assigned a specific meditation practice while others weren’t? If the sample is large enough, randomized trials can produce solid conclusions. But, sometimes, a study will not have a control group because it’s ethically impossible. (Would people still divert a trolley to kill one person in order to save five lives, if their decision killed a real person, instead of just being a thought experiment? We’ll never know for sure!)

The conclusions may still provide some insight, but they need to be kept in perspective.

6. Did the researchers establish causality, correlation, dependence, or some other kind of relationship?

I often hear “Correlation is not causation” shouted as a kind of battle cry, to try to discredit a study. But correlation—the degree to which two or more measurements seem to change at the same time—is important, and is one step in eventually finding causation—that is, establishing a change in one variable directly triggers a change in another.

The important thing is to correctly identify the relationship.

7. Is the journalist, or even the scientist, overstating the result?

Language that suggests a fact is “proven” by one study or which promotes one solution for all people is most likely overstating the case. Sweeping generalizations of any kind often indicate a lack of humility that should be a red flag to readers. A study may very well “suggest” a certain conclusion but it rarely, if ever, “proves” it.

This is why we use a lot of cautious, hedging language in Greater Good , like “might” or “implies.”

8. Is there any conflict of interest suggested by the funding or the researchers’ affiliations?

A recent study found that you could drink lots of sugary beverages without fear of getting fat, as long as you exercised. The funder? Coca Cola, which eagerly promoted the results. This doesn’t mean the results are wrong. But it does suggest you should seek a second opinion .

9. Does the researcher seem to have an agenda?

Readers could understandably be skeptical of mindfulness meditation studies promoted by practicing Buddhists or experiments on the value of prayer conducted by Christians. Again, it doesn’t automatically mean that the conclusions are wrong. It does, however, raise the bar for peer review and replication. For example, it took hundreds of experiments before we could begin saying with confidence that mindfulness can indeed reduce stress.

10. Do the researchers acknowledge limitations and entertain alternative explanations?

Is the study focused on only one side of the story or one interpretation of the data? Has it failed to consider or refute alternative explanations? Do they demonstrate awareness of which questions are answered and which aren’t by their methods?

I summarize my personal stance as a non-scientist toward scientific findings as this: Curious, but skeptical. I take it all seriously and I take it all with a grain of salt. I judge it against my experience, knowing that my experience creates bias. I try to cultivate humility, doubt, and patience. I don’t always succeed; when I fail, I try to admit fault and forgive myself. My own understanding is imperfect, and I remind myself that one study is only one step in understanding. Above all, I try to bear in mind that science is a process, and that conclusions always raise more questions for us to answer.

About the Author

Jeremy Adam Smith

Uc berkeley.

Jeremy Adam Smith edits the GGSC's online magazine, Greater Good . He is also the author or coeditor of five books, including The Daddy Shift , Are We Born Racist? , and (most recently) The Gratitude Project: How the Science of Thankfulness Can Rewire Our Brains for Resilience, Optimism, and the Greater Good . Before joining the GGSC, Jeremy was a John S. Knight Journalism Fellow at Stanford University.

You May Also Enjoy

This article — and everything on this site — is funded by readers like you.

Become a subscribing member today. Help us continue to bring “the science of a meaningful life” to you and to millions around the globe.

90 Questions to Ask a Researcher

Embarking on a journey to unravel the mysteries of research can be both exhilarating and daunting. Whether you’re a budding scholar, a curious journalist, or simply an individual with a thirst for knowledge, probing the minds of those at the forefront of discovery can provide invaluable insights.

This article offers an array of thought-provoking questions meticulously crafted to delve deep into the essence of research.

Table of Contents

Understanding Research Objectives

• What inspired the research question you’re exploring?
• Can you briefly describe the main goal of your research?
• How does your research fit within the broader field of study?
• Who are the intended beneficiaries of your research findings?
• Did the initial objectives evolve as the research progressed?
• How do you ensure your research objectives are achievable?
• What is the scope of your research?
• How did you narrow down your research questions?
• What significant problem does your research seek to address?
• What are the anticipated outcomes of your research?
• How do you prioritize different research objectives?
• What hypotheses are you testing in your research?
• Can you clarify any terms specific to your research objectives?
• How do your research objectives align with current trends in the field?
• What challenges did you face in defining your research objectives?

Exploring Methodology and Design

• What research methodology did you choose, and why?
• How did you ensure the reliability and validity of your methods?
• Can you explain the process of data collection in your research?
• Did you encounter any obstacles in your research design?
• How did your methodology affect the interpretation of results?
• What tools or technologies did you use in your research?
• How did you address potential biases in your research design?
• What criteria did you use for selecting your study sample?
• How do you manage the ethical considerations in your research?
• What measures did you take to ensure data security and privacy?
• How is your approach different from existing methodologies?
• Did you require any special approvals for your research methods?
• How do you keep your methodology transparent and replicable?
• Have you piloted your research design, and what were the results?
• Can you walk me through the timeline of the research process?

Discussing Findings and Interpretations

• What are the key findings of your research?
• Were there any surprising results you encountered?
• How do your findings contribute to the field?
• What do you believe is the reason behind these outcomes?
• How do your results compare to those of similar studies?
• What limitations should we consider when interpreting your findings?
• Can you discuss any patterns or trends revealed in your research?
• How might your findings influence future research?
• How robust are your findings to different analytical methods?
• Were any of your hypotheses not supported by the data?
• How do you plan to share and publish your findings?
• What is the potential for real-world application of your research?
• How do you validate the interpretations of your data?
• In what ways have you engaged with other researchers about your findings?
• What follow-up studies would you suggest based on your results?

Assessing Impact and Relevance

• How does your research address societal or environmental needs?
• In what ways could your research potentially affect industry practices?
• What long-term changes do you hope your research will inspire?
• How relevant is your research in today’s context?
• What is the envisioned impact on policy or public discourse?
• How do you gauge the success of your research’s impact?
• Who are the stakeholders most interested in your research?
• How do you disseminate your research to maximize impact?
• Can your research findings be generalized across different contexts?
• What collaborations did you form to enhance the impact of your research?
• How do you measure the social return on investment for your research?
• What has been the feedback from the community affected by your research?
• How has your research contributed to advancements in technology or science?
• What part of your research do you think will have the greatest impact?
• Are there any unintended consequences that may arise from your research?

Evaluating Credibility and Ethics

• How do you ensure the credibility of your research findings?
• What kind of peer review or scrutiny has your research undergone?
• Have you published in open-access journals or other accessible platforms?
• Can you explain the ethical considerations relevant to your research?
• How would you address any conflicts of interest in your work?
• How do you ensure that your research benefits are shared fairly?
• What steps do you take to maintain transparency in your research?
• How have you dealt with any ethical dilemmas during your study?
• In what ways do you work to maintain participant confidentiality?
• Can you discuss the ethical governance of your research project?
• What safeguards are in place for the ethical use of data?
• How does your research comply with legal and regulatory standards?
• Are there any socio-cultural sensitivities you had to consider in your research?
• How might your research confront or challenge ethical norms?
• What is your approach to obtaining informed consent?

Personal Insights and Journey in Research

• What inspired you to become a researcher in your field?
• Can you share a pivotal moment in your research career?
• How do you stay motivated when facing research challenges?
• What are the most rewarding aspects of conducting research?
• How do you balance personal biases and professional objectivity?
• What lessons have you learned from your time in research?
• How do you approach collaborative research projects?
• What are your top strategies for overcoming research obstacles?
• Can you describe a significant breakthrough you had in your research?
• How do you keep up-to-date with advancements in your field?
• What advice would you give to aspiring researchers?
• Can you share a mistake you made and how you addressed it?
• What has been the most unexpected turn in your research journey?
• How do you handle skepticism or criticism of your research?
• What personal qualities do you think are essential for a successful researcher?

Frequently Asked Questions

Why is it important to ask about a researcher’s personal insights.

Learning about a researcher’s personal insights provides context that enriches your understanding of their work. It can reveal the human element behind the research, including motivation, challenges, and the passion driving their inquiries.

How can I probe into the relevance and real-world application of research?

Ask direct questions about how the research tackles contemporary issues, its societal benefits, and its potential for practical application. Discussing envisioned impacts on policy or technology can also highlight the research’s real-world significance.

Final Thoughts

Beyond the data and analysis, these conversations are a portal into the multifaceted world of research – a blend of empirical rigor, ethical considerations, and personal dedication.

May these engaging inquiries lead you to profound discoveries and a deeper appreciation for the painstaking yet rewarding odyssey that researchers embark upon to advance human knowledge.

How useful was this post?

Click on a star to rate it!

As you found this post useful...

Share it on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Bea Mariel Saulo

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Braz J Cardiovasc Surg
• v.36(1); Jan-Feb 2021

The importance of asking questions and doing things for a reason

Rodolfo a. neirotti.

1 Master in Public Administration, Harvard Kennedy School. Harvard University, Cambridge, Massachusetts, United States.

2 Clinical Professor of Surgery and Pediatrics, Emeritus Michigan State University, Michigan, United States.

NB: Some of my thoughts expressed here have been shared in previous presentations.

Arts, artists and intellectuals, with their capacity of seeing with new eyes, sensing and perceiving mastery, finding beauty, meaning, elegance, rhythm, melody, harmony and composition-can help us to understand many aspects of our life, like:

"The two most important days in your life are the day you are born and the day you find out why." Mark Twain. 1835-1910. "Whoever has a reason to live will almost always find how." Frederick Nietzsche "I keep six honest serving-men. They taught me all I knew. Their names are: WHY, HOW, WHAT, WHEN, WHERE, WHO."

An exercise used as a problem analysis method based on Rudyard Kipling's poem, to ensure that all aspects were covered to improve interactions.\

"Stay away from difficult people- they have a problem for every solution. Attributed to Albert Einstein. How to handle them? This is a common question of business people that encloses the assumption that they are dealing with abrasive, competitive, and unethical behavior that makes them think that they are right and the other party wrong. However, in a conflictive situation the other party can think that the other is difficult and obstinate!

Daily we ask many questions, WHY? WHAT? HOW? Why we do things in difficult contexts due to cultural, political and economic adversity? To learn, pay attention to details: every aspect of your work, adopt an analytic mindset and ask Why? How? This Should be based on knowledge instead of feelings, habits and impulses. What? We all know what we do for a living-right or wrong.

Questions are useful tools, they open lines of communications; give us information; improve interactions, facilitate analysis and diagnostics of a situation; allow us to propose our own ideas; help to understand the priorities of others; stimulate motivation to learn; motivate creativity and more importantly scientific research, explanations and its applications happen in part through questions and answers.

Questioning everything helps to understand the world round us and Why we do what we do? How do we do that is about technical issues and What we do is usually known. Why reflects beliefs, reasons, purpose and objectives of an institution that eventually motivate its members to adopt them. Caution , failures of human psychology can induce people to consider a single factor. Responsible answers to questions can help to improve the quality of What we do. In addition, observations combined with curiosity and questions help us to learn WHY we do things. (3) Modified from Marilee Adams; Berrett Koehler Publishers, San Francisco, CA. 2016.

In general, people and societies know what they do and some are aware of How things are done but many of them ignore why they do what they do; that, in the end, has an impact on the outcomes. Please, if you do not know Why you do it, do not do it!

WHY Matters ? Because it is the question that every project team member should answer to explain the reason they are pursuing a venture. A compelling " WHY statement " is a useful tool that aligns the efforts of the leaders, and team members, to improve the chances of success. It sounds simple, but it's not. Often, a good why requires work and debate.

In medicine, doing things without knowing Why is risky. Many of the things that doctors and nurses do continue simply because that is the way we have always done it. Still, if they do know why, it does not mean that it was done correctly! Unfortunately, it can also be due to lack of knowledge, attitude, or practices that eventually became automatic. Doing things for no reason-ignoring why- can involuntary harm institutions and patients. Abraar Karam. BMJ opinion January 17, 2019.

For doing things for a reason, start with Why because virtually, everything we do and think is generated by questions that make you think. Many projects fail because their members are functioning without a good reason for doing things. Failures are often due to not discussing, agreeing, or learning why workers do things . K.A. Brown, N.L. Hyer and R. Ettenson. MIT Sloan Management Review. Fall 2013, Vol. 55 NO 1.

Why do we work? It is an appeal to reasoning rather than to emotions. Generally speaking, we work for many reasons. We work to live and live to work. Therefore, understanding why we work should help to improve our attitude, motivation, efficiency, productivity, team work and quality of life.

How to reconcile with work? Start by thinking what you owe to work rather than what work owes to you. Will more money motivate us to work harder? Actually, not quite. The reasons people work hard are more fundamental than most realize. Simply, feeling like you are part of the team and made progress on a task can give you the boost you need to keep going. Work for it! Modified from O. Kazhan, P Rosenfeld. The Atlantic Nov. 06, 2015.

WHY education? Because the educational system does not fulfill its purposes. Therefore, new teaching and learning strategies are needed in this evolving, technologically saturated world. This doesn't mean teaching people to accept a set of beliefs without making a proper analysis! This is because education reduces inequality, social problems, improves quality of life - that includes physical and mental health, family, work, income and the environment. Furthermore, democracy doesn't work without people capable to elect honest and competent politicians. In a country with significant and increasing inequality that is divided by political ideas, religious belief, self-interest, income, and with a heterogeneous population, it does not come as a surprise that education is divided. Social scientists offer competing models of class structure, and most agree that society is stratified, among other factors, by educational attainment. Inequality, poverty, suboptimal education and inadequate health care are barriers to Maslow' Sets of Needs. American Psychologist Abraham Maslow's Hierarchy of Needs Theory. 1943. In addition, "No society can surely be flourishing and happy, if the greater part of the numbers is poor and miserable" was the awkward phrase dropped by moral philosopher Adam Smith when he revised his thoughts for " The Wealth of Nations" published in 1776.

Altogether, this indicates that there is a need to modernize teaching techniques because there is a gap between what education systems provide and what is currently required by the society and employers. New technologies, have altered people's work and lives, pressing reformers to say that the traditional curriculum is not adequate.

Education and jobs can heal society. Authorities need to identify what skills are necessary for students to succeed in careers and personal lives, and then modernize their curriculums. Asking teachers to focus on a list of poorly defined skills is not enough. Interestingly, Angela Merkel during her address at the 2019 Harvard Graduation, address, stated "Nothing Has to Stay the Way It Is." Because the Berlin Wall limited peoples' opportunities, the German Chancellor invited the crowd to think with imagination about the possibility of precipitating what was previously an unimaginable change. "The Berlin Wall limited my opportunities but it couldn't impose limits on my inner thoughts... and that anything that seems to be set in stone or inalterable can, indeed, change."

Factors that can have either a positive or negative impact on the benefits of asking questions and doing things for a reason:

• Attitude is the tendency to respond positively or negatively to work, ideas, persons, objects or situations. In addition, it also impacts the individual's selection of actions, responses to challenges, incentives and prizes. An optimistic attitude, avoids negative thinking, and helps with daily activities. Talent is natural, and attitude cannot be taught.
• Motivation : the enthusiasm to do things, and a reason for people's actions, desires and needs. Inculcating motivation is not easy, but it's essential if you want your team to grow and stay satisfied with their jobs.
• Complacency and false urgency: Complacency : people do little or nothing to grow and improve, justify why they cannot do, and are unaware of the self-damage. False urgency : when people act and look busy, without adding value to what they are currently doing.
• Mental laziness -- a reluctance to doing something despite having the capability-due to a difficulty to put their brains to work. No matter how hard you work to get something done, mental laziness is when you stop midway because it is not easy. It is when you give up because you are tired or you feel that have done enough.
• Perfectionist , someone who avoid errors on a personal crusade for flawlessness. A boss, colleague, or even a work friend whose values have almost nothing to do with reality. Studies have tended to focus on their output rather than the effect they might have on their team climate or interpersonal relationships.

Finally, if we know Why , think carefully about What we must preserve. What we must improve? And what we must transform? In order to progress.

• Find Studies

For Researchers

• Learn about Research

Questions to ask the Researcher

You can ask a member of the research team questions at any point in the study.  You have the right to be informed! Here are some good questions to ask when you are considering participating.  You can bring these with you to your first study visit. 

What is the purpose of this study?

What will I be asked to do?

How will my information be kept private?

How long will the study last?

What are the incentives for being in the study?

Who can I contact if I have questions?

What happens if I no longer want to be in the study? 

Download a printable version

Copyright © 2013-2022 The NC TraCS Institute, the integrated home of the NIH Clinical and Translational Science Awards (CTSA) Program at UNC-CH.  This website is made possible by CTSA Grant UL1TR002489 and the National Center for Advancing Translational Sciences.

Helpful Links

• My UNC Chart
• Find a Doctor
• Accessibility
• Privacy Policy

Loading metrics

Open Access

Ten simple rules for reading a scientific paper

* E-mail: [email protected]

Affiliation Division of Infectious Diseases and International Health, Department of Medicine, University of Virginia School of Medicine, Charlottesville, Virginia, United States of America

• Maureen A. Carey, 
• Kevin L. Steiner, 
• William A. Petri Jr

Published: July 30, 2020

• https://doi.org/10.1371/journal.pcbi.1008032
• Reader Comments

Citation: Carey MA, Steiner KL, Petri WA Jr (2020) Ten simple rules for reading a scientific paper. PLoS Comput Biol 16(7): e1008032. https://doi.org/10.1371/journal.pcbi.1008032

Editor: Scott Markel, Dassault Systemes BIOVIA, UNITED STATES

Copyright: © 2020 Carey et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: MAC was supported by the PhRMA Foundation's Postdoctoral Fellowship in Translational Medicine and Therapeutics and the University of Virginia's Engineering-in-Medicine seed grant, and KLS was supported by the NIH T32 Global Biothreats Training Program at the University of Virginia (AI055432). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

“There is no problem that a library card can't solve” according to author Eleanor Brown [ 1 ]. This advice is sound, probably for both life and science, but even the best tool (like the library) is most effective when accompanied by instructions and a basic understanding of how and when to use it.

For many budding scientists, the first day in a new lab setting often involves a stack of papers, an email full of links to pertinent articles, or some promise of a richer understanding so long as one reads enough of the scientific literature. However, the purpose and approach to reading a scientific article is unlike that of reading a news story, novel, or even a textbook and can initially seem unapproachable. Having good habits for reading scientific literature is key to setting oneself up for success, identifying new research questions, and filling in the gaps in one’s current understanding; developing these good habits is the first crucial step.

Advice typically centers around two main tips: read actively and read often. However, active reading, or reading with an intent to understand, is both a learned skill and a level of effort. Although there is no one best way to do this, we present 10 simple rules, relevant to novices and seasoned scientists alike, to teach our strategy for active reading based on our experience as readers and as mentors of undergraduate and graduate researchers, medical students, fellows, and early career faculty. Rules 1–5 are big picture recommendations. Rules 6–8 relate to philosophy of reading. Rules 9–10 guide the “now what?” questions one should ask after reading and how to integrate what was learned into one’s own science.

Rule 1: Pick your reading goal

What you want to get out of an article should influence your approach to reading it. Table 1 includes a handful of example intentions and how you might prioritize different parts of the same article differently based on your goals as a reader.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pcbi.1008032.t001

Rule 2: Understand the author’s goal

In written communication, the reader and the writer are equally important. Both influence the final outcome: in this case, your scientific understanding! After identifying your goal, think about the author’s goal for sharing this project. This will help you interpret the data and understand the author’s interpretation of the data. However, this requires some understanding of who the author(s) are (e.g., what are their scientific interests?), the scientific field in which they work (e.g., what techniques are available in this field?), and how this paper fits into the author’s research (e.g., is this work building on an author’s longstanding project or controversial idea?). This information may be hard to glean without experience and a history of reading. But don’t let this be a discouragement to starting the process; it is by the act of reading that this experience is gained!

A good step toward understanding the goal of the author(s) is to ask yourself: What kind of article is this? Journals publish different types of articles, including methods, review, commentary, resources, and research articles as well as other types that are specific to a particular journal or groups of journals. These article types have different formatting requirements and expectations for content. Knowing the article type will help guide your evaluation of the information presented. Is the article a methods paper, presenting a new technique? Is the article a review article, intended to summarize a field or problem? Is it a commentary, intended to take a stand on a controversy or give a big picture perspective on a problem? Is it a resource article, presenting a new tool or data set for others to use? Is it a research article, written to present new data and the authors’ interpretation of those data? The type of paper, and its intended purpose, will get you on your way to understanding the author’s goal.

Rule 3: Ask six questions

When reading, ask yourself: (1) What do the author(s) want to know (motivation)? (2) What did they do (approach/methods)? (3) Why was it done that way (context within the field)? (4) What do the results show (figures and data tables)? (5) How did the author(s) interpret the results (interpretation/discussion)? (6) What should be done next? (Regarding this last question, the author(s) may provide some suggestions in the discussion, but the key is to ask yourself what you think should come next.)

Each of these questions can and should be asked about the complete work as well as each table, figure, or experiment within the paper. Early on, it can take a long time to read one article front to back, and this can be intimidating. Break down your understanding of each section of the work with these questions to make the effort more manageable.

Rule 4: Unpack each figure and table

Scientists write original research papers primarily to present new data that may change or reinforce the collective knowledge of a field. Therefore, the most important parts of this type of scientific paper are the data. Some people like to scrutinize the figures and tables (including legends) before reading any of the “main text”: because all of the important information should be obtained through the data. Others prefer to read through the results section while sequentially examining the figures and tables as they are addressed in the text. There is no correct or incorrect approach: Try both to see what works best for you. The key is making sure that one understands the presented data and how it was obtained.

For each figure, work to understand each x- and y-axes, color scheme, statistical approach (if one was used), and why the particular plotting approach was used. For each table, identify what experimental groups and variables are presented. Identify what is shown and how the data were collected. This is typically summarized in the legend or caption but often requires digging deeper into the methods: Do not be afraid to refer back to the methods section frequently to ensure a full understanding of how the presented data were obtained. Again, ask the questions in Rule 3 for each figure or panel and conclude with articulating the “take home” message.

Rule 5: Understand the formatting intentions

Just like the overall intent of the article (discussed in Rule 2), the intent of each section within a research article can guide your interpretation. Some sections are intended to be written as objective descriptions of the data (i.e., the Results section), whereas other sections are intended to present the author’s interpretation of the data. Remember though that even “objective” sections are written by and, therefore, influenced by the authors interpretations. Check out Table 2 to understand the intent of each section of a research article. When reading a specific paper, you can also refer to the journal’s website to understand the formatting intentions. The “For Authors” section of a website will have some nitty gritty information that is less relevant for the reader (like word counts) but will also summarize what the journal editors expect in each section. This will help to familiarize you with the goal of each article section.

https://doi.org/10.1371/journal.pcbi.1008032.t002

Rule 6: Be critical

Published papers are not truths etched in stone. Published papers in high impact journals are not truths etched in stone. Published papers by bigwigs in the field are not truths etched in stone. Published papers that seem to agree with your own hypothesis or data are not etched in stone. Published papers that seem to refute your hypothesis or data are not etched in stone.

Science is a never-ending work in progress, and it is essential that the reader pushes back against the author’s interpretation to test the strength of their conclusions. Everyone has their own perspective and may interpret the same data in different ways. Mistakes are sometimes published, but more often these apparent errors are due to other factors such as limitations of a methodology and other limits to generalizability (selection bias, unaddressed, or unappreciated confounders). When reading a paper, it is important to consider if these factors are pertinent.

Critical thinking is a tough skill to learn but ultimately boils down to evaluating data while minimizing biases. Ask yourself: Are there other, equally likely, explanations for what is observed? In addition to paying close attention to potential biases of the study or author(s), a reader should also be alert to one’s own preceding perspective (and biases). Take time to ask oneself: Do I find this paper compelling because it affirms something I already think (or wish) is true? Or am I discounting their findings because it differs from what I expect or from my own work?

The phenomenon of a self-fulfilling prophecy, or expectancy, is well studied in the psychology literature [ 2 ] and is why many studies are conducted in a “blinded” manner [ 3 ]. It refers to the idea that a person may assume something to be true and their resultant behavior aligns to make it true. In other words, as humans and scientists, we often find exactly what we are looking for. A scientist may only test their hypotheses and fail to evaluate alternative hypotheses; perhaps, a scientist may not be aware of alternative, less biased ways to test her or his hypothesis that are typically used in different fields. Individuals with different life, academic, and work experiences may think of several alternative hypotheses, all equally supported by the data.

Rule 7: Be kind

The author(s) are human too. So, whenever possible, give them the benefit of the doubt. An author may write a phrase differently than you would, forcing you to reread the sentence to understand it. Someone in your field may neglect to cite your paper because of a reference count limit. A figure panel may be misreferenced as Supplemental Fig 3E when it is obviously Supplemental Fig 4E. While these things may be frustrating, none are an indication that the quality of work is poor. Try to avoid letting these minor things influence your evaluation and interpretation of the work.

Similarly, if you intend to share your critique with others, be extra kind. An author (especially the lead author) may invest years of their time into a single paper. Hearing a kindly phrased critique can be difficult but constructive. Hearing a rude, brusque, or mean-spirited critique can be heartbreaking, especially for young scientists or those seeking to establish their place within a field and who may worry that they do not belong.

Rule 8: Be ready to go the extra mile

To truly understand a scientific work, you often will need to look up a term, dig into the supplemental materials, or read one or more of the cited references. This process takes time. Some advisors recommend reading an article three times: The first time, simply read without the pressure of understanding or critiquing the work. For the second time, aim to understand the paper. For the third read through, take notes.

Some people engage with a paper by printing it out and writing all over it. The reader might write question marks in the margins to mark parts (s)he wants to return to, circle unfamiliar terms (and then actually look them up!), highlight or underline important statements, and draw arrows linking figures and the corresponding interpretation in the discussion. Not everyone needs a paper copy to engage in the reading process but, whatever your version of “printing it out” is, do it.

Rule 9: Talk about it

Talking about an article in a journal club or more informal environment forces active reading and participation with the material. Studies show that teaching is one of the best ways to learn and that teachers learn the material even better as the teaching task becomes more complex [ 4 – 5 ]; anecdotally, such observations inspired the phrase “to teach is to learn twice.”

Beyond formal settings such as journal clubs, lab meetings, and academic classes, discuss papers with your peers, mentors, and colleagues in person or electronically. Twitter and other social media platforms have become excellent resources for discussing papers with other scientists, the public or your nonscientist friends, or even the paper’s author(s). Describing a paper can be done at multiple levels and your description can contain all of the scientific details, only the big picture summary, or perhaps the implications for the average person in your community. All of these descriptions will solidify your understanding, while highlighting gaps in your knowledge and informing those around you.

Rule 10: Build on it

One approach we like to use for communicating how we build on the scientific literature is by starting research presentations with an image depicting a wall of Lego bricks. Each brick is labeled with the reference for a paper, and the wall highlights the body of literature on which the work is built. We describe the work and conclusions of each paper represented by a labeled brick and discuss each brick and the wall as a whole. The top brick on the wall is left blank: We aspire to build on this work and label this brick with our own work. We then delve into our own research, discoveries, and the conclusions it inspires. We finish our presentations with the image of the Legos and summarize our presentation on that empty brick.

Whether you are reading an article to understand a new topic area or to move a research project forward, effective learning requires that you integrate knowledge from multiple sources (“click” those Lego bricks together) and build upwards. Leveraging published work will enable you to build a stronger and taller structure. The first row of bricks is more stable once a second row is assembled on top of it and so on and so forth. Moreover, the Lego construction will become taller and larger if you build upon the work of others, rather than using only your own bricks.

Build on the article you read by thinking about how it connects to ideas described in other papers and within own work, implementing a technique in your own research, or attempting to challenge or support the hypothesis of the author(s) with a more extensive literature review. Integrate the techniques and scientific conclusions learned from an article into your own research or perspective in the classroom or research lab. You may find that this process strengthens your understanding, leads you toward new and unexpected interests or research questions, or returns you back to the original article with new questions and critiques of the work. All of these experiences are part of the “active reading”: process and are signs of a successful reading experience.

In summary, practice these rules to learn how to read a scientific article, keeping in mind that this process will get easier (and faster) with experience. We are firm believers that an hour in the library will save a week at the bench; this diligent practice will ultimately make you both a more knowledgeable and productive scientist. As you develop the skills to read an article, try to also foster good reading and learning habits for yourself (recommendations here: [ 6 ] and [ 7 ], respectively) and in others. Good luck and happy reading!

Acknowledgments

Thank you to the mentors, teachers, and students who have shaped our thoughts on reading, learning, and what science is all about.

• 1. Brown E. The Weird Sisters. G. P. Putnam’s Sons; 2011.
• View Article
• Google Scholar
• PubMed/NCBI

Skip navigation

• Log in to UX Certification

World Leaders in Research-Based User Experience

Planning research with generative ai.

April 5, 2024 2024-04-05

• Email article
• Share on LinkedIn
• Share on Twitter

AI chatbots (like ChatGPT, Google’s Gemini, and Microsoft Copilot) can support UX researchers of all experience levels in planning their research.

In This Article:

What is a research plan, using ai chatbots to write a research plan.

Good research always starts with a plan.

A research plan is a document that outlines the research objectives and how the research will be executed.

Research plans should include:

• The research goals or questions that the research is hoping to achieve or answer
• The method to be used and a description of how it will be carried out
• The tasks or questions that will be given to study participants
• The profile of the target participants
• The screener questionnaire used to recruit participants

Creating a research plan can be time-consuming. Even with a good template, a researcher must generate research questions, select the appropriate method(s), decide how to run sessions, and often create study collateral (like screeners and tasks) from scratch. The good news is that AI can help with many, if not all, of these tasks!

It can be tempting to just ask an AI tool to give you a research plan for a project. Don’t do that.

❌ Bad Prompt : Generate a research plan for a usability test of a food-delivery app.

This kind of request results in a generic, template-like response, partly because AI lacks the context to propose a complete research plan and partly because of how the model has been trained.

To construct a useful research plan, deconstruct the plan into parts and have the AI chatbot tackle each part individually. You can assemble responses in a final research plan.

Don’t expect the AI to ask you the right questions in order to get a comprehensive outcome. View the AI tool as a UX assistant , not as a UX mentor. You need to feed into the AI all the steps and details you want it to consider.

Follow the steps below for the best results.

Step 1: Provide Context

Imagine asking an experienced colleague for recommendations on what research study to run without giving them any context on what project you’re working on and what you’re trying to achieve. They would probably say, "It depends on what you’re trying to learn.” Conversing with AI presents no exception.

Start your conversation with enough context , especially if the AI tool does not already have this information stored about you or your current project. ( ChatGPT 4 allows users to store context for each new conversation.)

The contextual information you share should include who you work for, the scope of your project, and what you’re looking to achieve. Remember that some AI tools may use your conversations to train their models, so share only information you would be comfortable with being in the open domain.

 Prompt Template: I work for [type or organization] . My company provides [what kind of product, service, or experience] . I need to run a user-research study to learn [something about users or a problem] .

Step 2: Generate Research Questions

After providing the AI tool with context, ask it to suggest specific research questions for the study. Here’s an example from a recent study on research repositories that I conducted for NN/g.

Prompt Example: I will run a research study and report on how UX practitioners across different organizations set up and use research repositories. Please generate 10 research questions that would be suitable for this sort of study.

It’s a good idea to ask generative AI tools for more options or ideas than you think you’ll need, because some will not be a good fit.

Microsoft Copilot responded to the prompt above with the 10 research questions below.

10 Research questions generated by Microsoft Copilot

Who are the primary users of the research repository within different organizations? What motivates users to engage with the research repository? How do different teams (design, product, sales, marketing) access and benefit from the research repository? What challenges do practitioners face when contributing to or retrieving information from the repository? What types of research artifacts (reports, raw data, templates) are most commonly stored in the repository? How do organizations structure their research repositories? What tools and templates are commonly used to facilitate research repository management? How do practitioners ensure the quality and accuracy of information in the repository? What impact does the research repository have on decision-making and project outcomes? What strategies can organizations implement to encourage widespread adoption and active participation in the research repository?

To broaden your pool of research questions, ask for further research questions or provide the same prompt in a new session. Repeat this process as many times as you need to generate a list that feels exhaustive of what you’d like to learn.

Some of the research questions Copilot generated for me were appropriate, and others were not. As a result, I had AI generate more research questions to choose from.

Unfortunately, most AI chatbots don’t offer an easy way to dismiss suggestions or combine specific responses and work from only these (a behavior called apple-picking ).

Pulling in generated research questions into an offline document (like a FigJam or Google Doc) allows you to easily group items, remove duplicates, or reword suggested research questions.

Begin a new chat session with your selected and refined set of research questions, so that the unwanted research questions are removed from the chat history.

Step 3: Request Methods

After sharing the context and your chosen research questions, ask the AI tool to identify suitable research methods.

Example Prompt : What study would you suggest to answer these research questions? Please be specific; cite which research questions would be answered by which research method if you suggest multiple methods.

Generative-AI advice is not always good advice. Often, these tools will suggest various methods and suggest you triangulate data from multiple sources. This approach is not always needed. Also, not all methods will be practical or the best fit for your study. Additionally, AI may suggest interviews and focus groups even for research questions better suited to a behavioral research method .

Ask AI chatbots to tell you which research methods would be suited to which research question and why. We also recommend doing some further reading on your own about any methods that are unfamiliar to you.

In response to the prompt above (and given my chosen research questions), ChatGPT recommended a survey, interviews with select UX practitioners, and case studies. These were all my chosen methods, so AI had done well here!

Step 4: Request Inclusion Criteria

AI can create inclusion criteria — a necessary component of your research plan. Do this step only after generating research questions and methods since these will inform who should participate in the research study.

Inclusion criteria (or recruitment criteria) are specific characteristics of the target population that need to be represented in your sample.

Start with inclusion criteria before asking the AI to help you write a screening questionnaire ; AI can only craft an appropriate screener after it “knows” who you’re looking to recruit.

Example Prompt: So that I recruit the right people for my interviews, help me create some inclusion criteria. What characteristics or behaviors should I recruit for?

Step 5: Request Help with Screeners, Interview Questions, and Tasks

Finally, ask the AI to put together:

• Interview questions or an interview guide (if conducting interviews)
• Tasks for a usability test
• Diary-study prompts (if relevant)
• Recruitment confirmation emails or other communication messages.

Unfortunately, there are a lot of bad examples of the above on the web. Conversational AI has been trained on all this data. Therefore, don’t be surprised if it produces poor study collateral on its first attempt! This is a major risk area for new researchers.

One way to mitigate this danger is to give the AI tool advice when crafting any of these outputs . Think of AI as a new research assistant who can learn extremely quickly.

Common mistakes that AI tools make include:

• Using words that appear in the interface in task instructions (priming)
• Creating task instructions that ask users to imagine they are someone that they are not
• Not including a goal or a call to action in the task instruction
• Not including distractor options in screening questionnaires
• Using overenthusiastic marketing language in recruitment materials

It’s not surprising that AI makes these mistakes since UX practitioners also make them!

To improve outputs, feed the AI essential tips, such as:

• When crafting tasks: Do not use the name of words or link labels in the task instruction. Find a natural-language equivalent to explain what the participant should do . (You can ask AI to “read” a website or an image.)
• When crafting recruitment materials: Use neutral and concise language in the recruitment email. Avoid using overly enthusiastic marketing language.
• When crafting a screener: Include multiple-choice questions and answer options that might disguise what the study is about and who I am looking to recruit.

Additionally, when possible, feed the AI with good examples of screener questionnaires, tasks, or interview questions, so it can follow their format or style.

Even with this advice, AI can still make mistakes. If you’re doubting its answers, check primary sources or speak with an experienced researcher for old-fashioned human guidance.

If you have ChatGPT’s Plus Plan, you can use our GPT for planning your research.

With the proper context, examples, and advice, AI tools, like ChatGPT and Microsoft Copilot, can craft helpful research questions, tasks, interview questions, and other study collateral far more quickly than you could if you started from scratch.

Research leads and ResearchOps personnel can support junior researchers and PWDRs (People Who Do Research) by providing examples and advice that can be fed to AI agents. Experienced researchers can benefit from using AI to speed up their research-planning process and obtain further inspiration.

Free Downloads

Related courses, practical ai for ux professionals.

Leverage artificial intelligence tools to enhance your UX work and save valuable time

Interaction

ResearchOps: Scaling User Research

Orchestrate and optimize research to amplify its impact

Analytics and User Experience

Study your users’ real-life behaviors and make data-informed design decisions

Related Topics

• Artificial Intelligence Artificial Intelligence
• Research Methods

Learn More:

AI on Intranets: 5 Valuable Features

Anna Kaley · 3 min

ELIZA Effect: Why We Fall in Love With AI

Caleb Sponheim · 3 min

Is Aggressive Marketing Influencing the UX of AI Agents?

Jakob Nielsen · 2 min

Related Articles:

Generative UI and Outcome-Oriented Design

Kate Moran and Sarah Gibbons · 6 min

AI Chat Is Not (Always) the Answer

Sarah Gibbons and Kate Moran · 5 min

The UX of AI: Lessons from Perplexity

Kate Moran · 5 min

Response Outlining with Generative-AI Chatbots

Tarun Mugunthan · 3 min

Sycophancy in Generative-AI Chatbots

Caleb Sponheim · 4 min

Prompt Structure in Conversations with Generative AI

Raluca Budiu, Feifei Liu, Amy Zhang, and Emma Cionca · 16 min

• Inside eLife

Publishing with eLife: Submitting with our new model

• Open access
• Copyright information

Submitting your research can leave a lot of unanswered questions, especially if it’s your first time submitting with eLife. What happens after you submit and what goes on behind the scenes? To help you better understand the submission process, we’re taking you through each step.

Before you submit

The first step is to find the right journal for your work. eLife’s aims and scope clearly outline the major subject areas we cover and link to the latest research in those fields. Another way to ensure a good fit is finding research on related topics to your work in the journal.

You should also make sure that the journal’s policies and approach to publishing align with your goals and values, as well as any institutional or funding requirements you might have. What should you expect regarding the transparency of peer review, policies around data availability, publication and research ethics, and what happens to the copyright of your work?

While the information here will focus on eLife’s submission process, Think Check Submit is a free and independent service that can help guide you through useful questions to ask when choosing where to submit.

Preprinting ahead of submission

Before submitting to a journal, you may want to share a preprint of your work.

A preprint is a version of your research that’s posted to a public server, often prior to any formal review. A preprint helps communicate your research quickly where others can start to make use of it, it establishes your priority for your findings, and it can be a way to get early feedback from your peers.

If you do decide to post a preprint it can help streamline your eLife submission.

eLife Submission checklist

When you start your submission it can help to have a few things ready. Don’t worry if not; you can always create a partial submission and come back later.

• if you have already posted a preprint, your preprint DOI
• the title and abstract of your paper
• information about each author: their name, email, institution, city, and country
• the most appropriate subject area(s) for your research
• suggestions or exclusions of Senior Editors and Reviewing Editors
• potential reviewers – we welcome a diverse set of suggestions, including people from eLife's pool of early-career reviewers
• licensing information: we use a Creative Commons Attribution license , except where otherwise noted
• who will be paying the fee – or you can request to waive the fee where necessary
• ensure your files are ready to be uploaded (ideally combined as a single PDF file in the first instance, although you can upload large supplementary files separately)

Submitting your research

When choosing to submit to eLife, it’s important to understand the experience and the outputs you will get from our new model . There is no accept/reject decision after peer review: rather, the preprints we review are published on the eLife website as a Reviewed Preprint that includes an eLife assessment , public reviews, and a provisional response from the authors (if they provide one). We publish several types of articles as Reviewed Preprints so consider which best fits your work.

Three ways to submit:

1. Transfer from bioRxiv or medRxiv You can submit your research to eLife by first sharing a preprint of your work on bioRxiv or medRxiv and transferring the work across for consideration. This can streamline the submission process, but you’ll still have a chance to add details before the submission is finalised later.

2. Submit to us directly (preprint first) If you already have a preprint on bioRxiv or medRxiv, you can enter the preprint DOI in the eLife submissions platform and use the Import link to pull in the files and key information from the preprint.

3. Submit to us directly (preprint later) You can submit to eLife directly without first having posted a preprint. We’ll collect the information we need to be able to post a preprint on your behalf if we proceed with peer review.

When you’re ready to submit you can login or sign up to our submissions platform here (or use the transfer service from bioRxiv or medRxiv).

What happens after you’ve submitted?

Typically a number of editors, who are all active researchers and experts in their field, are involved in deciding whether to review each submission. A Deputy Editor and a Senior Editor consider each submission, and in most cases a Senior Editor will consult with several Reviewing Editors. A number of submissions would be better served being peer reviewed within a more specialised community and on occasion we may not have editors who are available and interested in taking a paper through the review process.

Editorial screening and research integrity

During the submission and review process we will conduct a series of editorial and research integrity checks. This includes plagiarism detection for new submissions, and image screening for papers sent for peer review.

For medical submissions, we’ll ask for ICMJE forms; in other cases we may ask for PDB maps and coordinate files; and sometimes we’ll request an Inclusion in Global Research form. Later on, we’ll ask for things such as source data for gels and blots.

Peer review

Our editors select experts to review the work independently, and then the editors and reviewers discuss the reviews with each other in a constructive, consultative process. The editors and reviewers discuss which terms best describe the significance of the findings and strength of evidence, to help arrive at an eLife assessment.

You will receive the eLife assessment, the Public Reviews, and also recommendations that can be implemented as part of a revision. This can help you understand what changes could help improve your paper and whether it might lead to an updated eLife assessment. The letter to the authors is prepared by the Reviewing Editor, with input from the Senior Editor, staff, and a final check by a Deputy Editor.

Once you have received the eLife assessment, public reviews, you have two weeks before the Reviewed Preprint is published. You can use this time to raise concerns about any factual errors or provide a provisional author response to accompany the first version of your Reviewed Preprint. Your provisional author response can address the eLife assessment and public reviews, either in the form of a few summary paragraphs, or by addressing each comment in turn: this can outline the revisions that are planned or add points of clarification. If you would prefer to accelerate publication of the Reviewed Preprint, please let us know. Once this stage is complete, your work is published as a Reviewed Preprint.

You’ve published your Reviewed Preprint v1

This is only a step on your paper’s journey and it doesn’t need to be a final one. Your Reviewed Preprint version 1, citable, sharable and with its own DOI, is now openly available alongside trusted expert review and commentary, where it can start having an impact in your field and beyond. Most authors will later revise their work and address some or all of the reviewer comments and suggestions. This can help further refine your work, and will form part of its living record. In these cases a revised version of the Reviewed Preprint will be published after the editors and reviewers have had an opportunity to consider updating the eLife assessment and public reviews. Our approach to publishing helps create an open record of research communication that doesn’t squander research outputs when they can be of use to research communities. Your research is shared together with the valuable contributions of reviewers and editors, which can help readers better understand your preprint’s strengths and limitations, while empowering authors as they decide in what ways they revise and update their work.

We welcome comments and questions from researchers as well as other journals. Please annotate publicly on the article or contact us at hello [at] elifesciences [dot] org.

For the latest in published research sign up for our bi-weekly email alerts . You can also follow eLife on Twitter , Mastodon , Facebook or LinkedIn .

Be the first to read new articles from eLife

• Election 2024
• Entertainment
• Newsletters
• Photography
• Personal Finance
• AP Investigations
• AP Buyline Personal Finance
• Press Releases
• Israel-Hamas War
• Russia-Ukraine War
• Global elections
• Asia Pacific
• Latin America
• Middle East
• Election Results
• Delegate Tracker
• AP & Elections
• March Madness
• AP Top 25 Poll
• Movie reviews
• Book reviews
• Personal finance
• Financial Markets
• Business Highlights
• Financial wellness
• Artificial Intelligence
• Social Media

Americans think a president’s power should be checked, AP-NORC poll finds — unless their side wins

A new Associated Press-NORC poll finds that while Americans say they respect the Constitution’s checks and balances and don’t want a president to have too much power, that view shifts if the candidate of their party wins the presidency.

FILE - President Joe Biden speaks at an event in Raleigh, N.C., March. 26, 2024. A new poll from The Associated Press-NORC Center for Public Opinion Research conducted March 21-25, finds that while Americans say they respect the Constitution’s checks and balances and don’t want a president to have too much power, that view shifts if the candidate of their party wins the presidency. It’s a view held by members of both parties, though it’s especially common among Republicans. (AP Photo/Matt Kelley, File)

• Copy Link copied

FILE - Republican presidential candidate former President Donald Trump speaks April 2, 2024, at a rally in Green Bay, Wis. A new poll from The Associated Press-NORC Center for Public Opinion Research conducted March 21-25, finds that while Americans say they respect the Constitution’s checks and balances and don’t want a president to have too much power, that view shifts if the candidate of their party wins the presidency. It’s a view held by members of both parties, though it’s especially common among Republicans. (AP Photo/Mike Roemer, File)

WASHINGTON (AP) — Like many Americans, Richard Bidon says he’d like to see the U.S. government “go back to its original design” — a system of checks and balances developed nearly 240 years ago to prevent any branch, especially the presidency, from becoming too powerful.

But that’s mainly when Republicans are in power.

Bidon, an 84-year-old Democrat who lives near Los Angeles, said if President Joe Biden is reelected , he doesn’t want him to have to get the approval of a possibly Republican-controlled Congress to enact policies to slow climate change. He wants presidents to have the power to change policy unilaterally — as long as they’re from the right party.

“When a Democrat’s in, I support” a strong presidency, Bidon said. “When Republicans are in, I don’t support it that much. It’s sort of a wishy-washy thing.”

A new poll from The Associated Press-NORC Center for Public Opinion Research finds that Bidon’s view is common. Though Americans say don’t want a president to have too much power, that view shifts if the candidate of their party wins the presidency. It’s a view held by members of both parties, though it’s especially common among Republicans.

Overall, only about 2 in 10 Americans say it would be “a good thing” for the next president to be able to change policy without waiting on Congress or the courts. But nearly 6 in 10 Republicans say it would be good for a future President Donald Trump to take unilateral action, while about 4 in 10 Democrats say the same if Biden is reelected.

The sentiment comes amid escalating polarization and is a sign of the public’s willingness to push the boundaries of the political framework that has kept the U.S. a stable democracy for more than two centuries. In the poll, only 9% of Americans say the nation’s system of checks and balances is working extremely or very well. It also follows promises by Trump to “act as a dictator” on day one of a new administration to secure the border and expand oil and gas drilling.

Bob Connor, a former carpenter now on disability in Versailles, Missouri, wants that type of decisive action on the border. He’s given up hope on Congress taking action.

“From what I’ve seen, the Republicans are trying to get some stuff done, the Democrats are trying to get some other stuff done — they’re not mixing in the middle,” said Connor, 56. “We’re not getting anywhere.”

He blames the influx of migrants on Biden unilaterally revoking some of Trump’s own unilateral border security policies when he took office.

“I’m not a Trump fanatic, but what he’s saying has to get done is right,” Connor said.

Joe Titus, a 69-year-old Democrat from Austin, Texas, believes Republicans have destroyed Congress’ ability to act in its traditional legislative role and says Biden will have to step into the gap.

“There’s this so-called ‘majority’ in Congress, and they’re a bunch of whack-jobs,” Titus, a retired Air Force mechanic, said of the GOP-controlled House of Representatives. “It’s not the way this thing was set up.”

The current Congress is setting dubious records as the least productive one in the country’s history, with fewer than three dozen bills sent to Biden’s desk last year. At Trump’s urging, House Republicans have stalled aid to Ukraine and a bipartisan immigration bill .

Titus said that in general he opposes expanded presidential power but would support Biden funding more immigration judges and sending additional aid to Ukraine on his own.

“There’s certain things that it seems to me the public wants and the other party is blocking,” Titus said.

The presidency has steadily gained power in recent years as congressional deadlocks have become more common. Increasingly, the nation’s chief executive is moving to resolve issues through administrative policy or executive orders. The U.S. Supreme Court is poised to rule later this year on a case that could significantly weaken the ability of federal agencies — and thus a presidential administration — to issue regulations.

Meanwhile, conservatives are planning a takeover of the federal bureaucracy should they win the White House in November, a move that could increase the administration’s ability to make sweeping policy changes on its own.

The AP-NORC poll found that voters’ views of which institutions have too much power were colored by their own partisanship. Only 16% of Democrats, whose party currently controls the White House, say the presidency has too much power while nearly half of Republicans believe it does. In contrast, about 6 in 10 Democrats say the U.S. Supreme Court, with its 6-3 conservative majority, has too much power.

With Congress evenly divided between the two parties — the GOP has a narrow House majority, Democrats a narrow Senate one — Americans have similar views on its power regardless of party. About 4 in 10 from both major parties say it has too much power.

“I think Congress had too much power when the presidency and Congress were both ruled by Democrats, but now that Republicans are in the majority there’s an equal balance,” said John V. Mohr, a 62-year-old housecleaner in Wilmington, North Carolina.

In contrast, he complained that Biden is “sitting there writing executive orders left and right,” including his proclamation marking Transgender Day of Visibility , which fell on Easter Sunday this year.

The abstract idea of a president with nearly unchecked power remains unpopular.

Steven Otney, a retired trucker in Rock Hill, South Carolina, said major policies should be approved by Congress and gain approval from the courts. But he also said it depends on the topic. He wants to see prompt action on certain issues by the next president if he’s Trump.

“Some things need to be done immediately, like that border wall being finished,” said Otney, a Republican.

He said it’s just common sense.

“If Trump got in there and said ‘I want to bomb Iran,’ no, that’s crazy,” Otney said. “Within reason, not stupid stuff either way. Something to help the American people, not hurt us.”

The poll of 1,282 adults was conducted March 21-25, 2024, using a sample drawn from NORC’s probability-based AmeriSpeak Panel, which is designed to be representative of the U.S. population. The margin of sampling error for all respondents is plus or minus 3.8 percentage points.

Riccardi reported from Denver.

The Associated Press receives support from several private foundations to enhance its explanatory coverage of elections and democracy. See more about AP’s democracy initiative here . The AP is solely responsible for all content.

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

To Succeed with AI, Adopt a Beginner’s Mindset

• Jacqueline Carter,
• Marissa Afton,
• Paula Kelley

Letting go of ego and expertise allows for openness and curiosity.

Times of substantial tech progress and change, like the current AI revolution, create fear and anxiety. This often causes leaders to fall back on their ego and emphasize their expertise, closing their minds and negatively impacting their people and organizations. Instead, leaders need to take on a beginner’s mindset of openness and curiosity. This is not easy. The more experienced we are, the more locked-in we tend to become in our ways of thinking and doing things. But it is possible, and with employee stress at record highs, it’s necessary. And research shows that the openness that comes with a beginner’s mindset is a crucial factor in achieving better outcomes. There are a few simple questions you can ask yourself to gauge your leadership style and whether you have a beginner’s mindset.

Every day, fears about artificial intelligence (AI) run through the news cycle: Jobs will be eliminated, intellectual property misused, personal data compromised, or biases propagated.

• Jacqueline Carter is a senior partner and the North American Director of Potential Project. She has extensive experience working with senior leaders to enable them to achieve better performance while enhancing a more caring culture. She is the coauthor, with Rasmus Hougaard, of Compassionate Leadership: How to Do Hard Things in a Human Way and The Mind of the Leader – How to Lead Yourself, Your People, and Your Organization for Extraordinary Results .
• Marissa Afton is an organizational psychologist and a Partner and the head of Global Accounts at Potential Project. She is a contributing coauthor of Compassionate Leadership: How to Do Hard Things in a Human Way by Rasmus Hougaard and Jacqueline Carter.
• Paula Kelley leads Potential Project’s global marketing function and serves its financial services clients. Prior to Potential Project, Paula was a senior executive at Citigroup and a partner at Deloitte Consulting.

Partner Center