questions to ask about research articles

• Follow us on Facebook
• Follow us on Twitter
• Criminal Justice
• Environment
• Politics & Government
• Race & Gender

Expert Commentary

Eight questions to ask when interpreting academic studies: A primer for media

Scholarly research is a great source for rigorous, unbiased information, but making judgments about its quality can be difficult. Here are some important questions to ask when reading studies.

Republish this article

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License .

by Justin Feldman and John Wihbey, The Journalist's Resource March 26, 2015

This <a target="_blank" href="https://journalistsresource.org/home/interpreting-academic-studies-primer-media/">article</a> first appeared on <a target="_blank" href="https://journalistsresource.org">The Journalist's Resource</a> and is republished here under a Creative Commons license.<img src="https://journalistsresource.org/wp-content/uploads/2020/11/cropped-jr-favicon-150x150.png" style="width:1em;height:1em;margin-left:10px;">

Reading scholarly studies can help journalists integrate rigorous, unbiased sources of information into their reporting. These studies are typically carried out by professors and professional researchers — at universities, think tanks and government institutions — and are published through a peer-review process in which those familiar with the study area ensure that there are no major flaws.

Even for people who carry out research, however, interpreting scientific (and social science) studies and making judgments about their quality can be difficult tasks. In a now-famous article, Stanford professor John Ioannidis argues that “ most published research findings are false ” due to inherent limitations in how researchers design studies. (Health and medical studies can be particularly attractive to media, but be aware that there is a long history of faulty findings .) Occasionally, too, studies can be the product of outright fraud: A 1998 study falsely linking vaccines and autism is now perhaps the canonical example, as it spurred widespread and long-lasting societal damage . Journalists should also always examine the funding sources behind the study, which are frequently declared at the study’s conclusion.

Before journalists write about research and speak with authors, they should be able to both interpret a study’s results generally and understand the appropriate degree of skepticism that a given study’s findings warrant. This requires data literacy , some familiarity with statistical terms and a basic knowledge of hypothesis testing and construction of theories .

Journalists should also be well aware that most academic research contains careful qualifications about findings. The common complaint from scientists and social scientists is that news media tend to pump up findings and hype studies through catchy headlines, distorting public understanding. But landmark studies sometimes do no more than tighten the margin of error around a given measurement — not inherently flashy, but intriguing to an audience if explained with rich context and clear presentation.

Here are some important questions to ask when reading a scientific study:

1. What are the researchers’ hypotheses?

A hypothesis is a research question that a study seeks to answer. Sometimes researchers state their hypotheses explicitly, but more often their research questions are implicit. Hypotheses are testable assertions usually involving the relationship between two variables. In a study of smoking and lung cancer, the hypothesis might be that smokers develop lung cancer at a higher rate than non-smokers over a five-year period.

It is also important to note that there are formal definitions of null and alternative hypotheses for use with statistical analysis.

2. What are the independent and dependent variables?

Independent variables are factors that influence particular outcomes. Dependent variables are measures of the outcomes themselves. In the study assessing the relationship between smoking and lung cancer, smoking is the independent variable because the researcher assumes it predicts lung cancer, the dependent variable. (Some fields use related terms such as “exposure” and “outcome.”)

Pay particular attention to how the researchers define all of the variables — there can be quite a bit of nuance in the definitions. Also look at the methods by which the researchers measure the variables. Generally speaking, a variable measured using a subject’s response to a survey question is less trustworthy than one measured through more objective means — reviewing laboratory findings in their medical records, for example.

3. What is the unit of analysis?

For most studies involving human subjects, the individual person is the unit of analysis. However, studies are sometimes interested in a different level of analysis that makes comparisons between classrooms, hospitals, schools or states, for example, rather than between individuals.

4. How well does the study design address causation?

Most studies identify correlations or associations between variables, but typically the ultimate goal is to determine causation . Certain study designs are more useful than others for the purpose of determining causation.

At the most basic level, studies can be placed into one of two categories: experimental and observational . In experimental studies, the researchers decide who is exposed to the independent variable and who is not. In observational studies, the researchers do not have any control over who is exposed to the independent variable — instead they make comparisons between groups that are already different from one another. In nearly all cases, experimental studies provide stronger evidence than observational studies.

Here are descriptions of some of the most common study designs, presented along with their respective values for inferring causation:

• Randomized controlled trials (RCTs), also known as clinical trials, are experimental studies that are considered the “gold standard” in research. Out of all study designs, they have the most value for determining causation although they do have limitations. In an RCT, researchers randomly divide subjects into at least two groups: One that receives a treatment, and the other — the control group — that receives either no treatment or a simulated version of the treatment called a placebo . The independent variable in these experiments is whether or not the subject receives the real treatment. Ideally an RCT should be double-blind — the participants should not know to which treatment group they have been assigned, nor should the study staff know. This arrangement helps to avoid bias. Researchers commonly use RCTs to meet regulatory requirements, such as evaluating pharmaceuticals for the Food and Drug Administration. Due to issues of cost, logistics and ethics, RCTs are fairly uncommon for other purposes. Example: “ Short-Term Soy Isoflavone Intervention in Patients with Localized Prostate Cancer ”
• Longitudinal studies , like RCTs, follow the same subjects over a given time period. Unlike in RCTs, they are observational. Researchers do not assign the independent variable in longitudinal studies — they instead observe what happens in the real world. A longitudinal study might compare the risk for heart disease among one group of people who are exposed to high levels of air pollution to the risk of heart disease among another group exposed to low levels of air pollution. The problem is that, because there is no random assignment, the groups may differ from one another in other important ways and, as a result, we cannot completely isolate the effects of air pollution. These differences result in confounding and other forms of bias. For that reason, longitudinal studies have less validity for inferring causation than RCTs and other experimental study designs. Longitudinal studies have more validity than other kinds of observational studies, however. Example: “ Mood after Moderate and Severe Traumatic Brain Injury: A Prospective Cohort Study ”
• Case-control studies are technically a type of longitudinal study, but they are unique enough to discuss separately. Common in public health and medical research, case-control studies begin with a group of people who have already developed a particular disease and compare them to a similar but disease-free group recruited by the researchers. These studies are more likely to suffer from bias than other longitudinal studies for two reasons. First, they are always retrospective , meaning they collect data about independent variables years after the exposures of interest occurred — sometimes even after the subject has died. Second, the group of disease-free people is very likely to differ from the group that developed the disease, creating a substantial risk for confounding. Example: “ Risk Factors for Preeclampsia in Women from Colombia ”.
• Cross-sectional studies are a kind of observational study that measure both dependent and independent variables at a single point in time. Although researchers may administer the same cross-sectional survey every few years, they do not follow the same subjects over time. An important part of determining causation is establishing that the independent variable occurred for a given subject before the dependent variable occurred. But because they do not measure the variables over time, cross-sectional studies cannot determine that a hypothesized cause precedes its effect, so the design is limited to making inferences about correlations rather than causation. Example : “ Physical Predictors of Cognitive Performance in Healthy Older Adults ”
• Ecological studies are observational studies that are similar to cross-sectional studies except that they measure at least one variable on the group-level rather that the subject-level. For example, an ecological study may look at the relationship between individuals’ meat consumption and their incidence of colon cancer. But rather than using individual-level data, the study relies on national cancer rates and national averages for meat consumption. While it might seem that higher meat consumption is linked to a higher risk of cancer, there is no way to know if the individuals eating more meat within a country are the same people who are more likely to develop cancer. This means that ecological studies are not only inadequate for inferring causation, they are also inadequate for establishing a correlation. As a consequence, they should be regarded with strong skepticism. Example: “ A Multi-country Ecological Study of Cancer Incidence Rates in 2008 with Respect to Various Risk-Modifying Factors ”
• Systematic reviews are surveys of existing studies on a given topic. Investigators specify inclusion and exclusion criteria to weed out studies that are either irrelevant to their research question or poorly designed. Using keywords, they systematically search research databases, present the findings of the studies they include and draw conclusions based on their consideration of the findings. Assuming that the review includes only well-designed studies, systematic reviews are more useful for inferring causation than any single well-designed study. Example: “ Enablers and Barriers to Large-Scale Uptake of Improved Solid Fuel Stoves. ” For a sense of how systematic reviews are interpreted and used by researchers in the field, see “How to Read a Systematic Review and Meta-analysis and Apply the Results to Patient Care,” published in the Journal of the American Medical Association (JAMA.)
• Meta-analyses are similar to systematic reviews but use the original data from all included studies to create a new analysis. As a result, a meta-analysis is able to draw conclusions that are more meaningful than a systematic review. Again, a meta-analysis is more useful for inferring causation than any single study, assuming that all studies are well-designed. Example: “ Occupational Exposure to Asbestos and Ovarian Cancer ”

5. What are the study’s results?

There are several aspects involved in understanding a study’s results:

• Understand whether or not the study found statistically significant relationships between the dependent and independent variables. If the relationship is statistically significant, it means that any difference observed between groups is unlikely to be due to random chance. P-values help researchers to decide whether observed differences are simply due to chance or represent a true difference between groups.
• If the relationship is statistically significant, it is then important to determine the effect size , which is the size of the difference observed between the groups. Subjects enrolled in a weight loss program may have experienced a statistically significant reduction in weight compared to those in a control group, but is that difference one ounce, one pound or ten pounds? There are myriad ways in which studies present effect sizes — such obscure terms as regression coefficients, odds ratios, and population attributable fractions may come into play. Unfortunately, research articles sometimes fail to interpret effect sizes in words. In these cases, it may be best to consult an expert to help develop a plain-English interpretation.
• Even if there is a statistically significant difference between comparison groups, this does not mean the effect size is meaningful. A weight loss program that leads to a total weight reduction of one ounce on average or a policy that saves one life out of a billion may not be meaningful. Again, consulting an expert in the field can help to determine how meaningful an effect size is, a determination that is ultimately a subjective judgment call.

6. How generalizable are the results?

Study results are useful because they help us make inferences about the relationship between independent and dependent variables among a larger population. The subjects enrolled in the study must be similar to those in the larger population, however, in order to generalize the findings. Even a perfectly designed study may be of limited value when its results cannot be generalized. It is important to pay attention to the composition of the study sample. If the unit of analysis is the individual, important factors to consider regarding the group’s composition include age, race/ethnicity, gender, socioeconomic status, and geographic location. While some samples are deliberately constructed to be representative of a country or region, most are not.

7. What limitations do the authors note?

Within a research article, authors often state some of the study’s limitations explicitly. This information can be very helpful in determining the strength of the evidence presented in the study.

8. What conclusions do similar studies draw?

With some notable exceptions, a single study is unlikely to fundamentally change what is already known about the research question it addresses. It is important to compare a new study’s findings to existing studies that address similar research questions, particularly systematic reviews or meta-analyses if available.

Further: One hidden form of bias that is easily missed is what’s called “selecting on the dependent variable,” which is the research practice of focusing on only those areas where there are effects and ignoring ones where there are not. This can lead to exaggerated conclusions (and thereby false media narratives). For example, it is tempting to say that “science has become polarized,” as survey data suggest significant differences in public opinion on issues such as climate change, vaccinations and nuclear power. However, on most scientific issues, there is almost no public debate or controversy . Additionally, the reality of “publication bias” — academic journals have traditionally been more interested in publishing studies that show effects, rather than no effects — can create a biased incentive structure that distorts larger truths.

For an updated overview, see a 2014 paper by Stanford’s John Ioannidis, “How to Make More Published Research True.”

Keywords: training

About the Authors

Justin Feldman

John Wihbey

Child Care and Early Education Research Connections

Key questions to ask.

This section outlines key questions to ask in assessing the quality of research and describes internal validity, external validity, and construct validity.

Was the research peer reviewed?

Peer reviewed research studies have already been evaluated by experienced researchers with relevant expertise. Most journal articles, books and government reports have gone through a peer review process. Keep in mind that there are many types of peer reviews. Reports issued by the federal government have been subject to many levels of internal review and approval before being issued. Articles published in professional journals with peer review have been evaluated by researchers that are experts in the field and who can vouch for the soundness of the methodology and the analysis applied. As a result, peer-reviewed research is usually of high quality. A research consumer, however, should still critically evaluate the study's methodology and conclusions.

Can a study's quality be evaluated with the information provided?

Every study should include a description of the population of interest, an explanation of the process used to select and gather data on study subjects, definitions of key variables and concepts, descriptive statistics for main variables, and a description of the analytic techniques. Research consumers should be cautious when drawing conclusions from studies that do not provide sufficient information about these key research components.

Are there any potential threats to the study's validity?

A valid study answers research questions in a scientifically rigorous manner. Threats to a study's validity are found in three areas: 

Internal Validity

External validity, construct validity.

Internal Validity refers to whether the outcomes observed in a study are due to the independent variables or experimental manipulations investigated in the study and not to some other factor or set of factors. To determine whether a research study has internal validity, a research consumer should ask whether changes in the outcome could be attributed to alternative explanations that are not explored in the study. For example, a study may show that a new curriculum had a significant positive effect on children's reading comprehension.

The study must rule out alternative explanations for the increase in reading comprehension, such as a new teacher, in order to attribute the increase in reading comprehension to the new curriculum. Studies that specifically explain how alternative explanations were ruled out are more likely to have internal validity. Threats to a study's internal validity can compromise the confidence consumers have in the findings from a study and include:

• The introduction of events while the study is being conducted that may affect the outcome or dependent variable of the study. For example, while studying the effectiveness of children's participation in an early childhood program, the program was closed for an extended period of time due to damage from a hurricane.
• Changes in the dependent variable due to normal developmental processes in study participants. For example, young children's performance on a battery of outcome measures (e.g., reading and math assessments) may decline during the testing or observation period due to fatigue or other factors.
• The circumstances around the testing that is used to assess the dependent variable. For example, preschool children's performance on a standardized test may be questionable if test items are presented to children in unfamiliar ways or in group settings.
• Participants leaving or dropping out of the study before it is completed. This can be especially problematic if those who leave the study are different from those who stay. For example, in a longitudinal study of the effects of a school lunch program on children's academic achievement, the validity of the findings could be problematic if the most disadvantaged children in the program left the study at a higher rate than other children.
• Changes to or inconsistencies in how the dependent and independent variables were measured. For example, changing the way in which children's math skills are measured at two time points could introduce error if the two measures were developed using different assessment frameworks (i.e., they were developed to assess different math content and processes). Inconsistencies are also introduced when different staff follow different procedures when administering the same measure. For example, when administering an assessment to bilingual children, some staff give children credit for answering correctly in English or Spanish, and other staff only give credit for answering correctly in English.
• Statistical regression or regression to the mean can affect the outcome of a study. It is the movement of test scores (post-test scores) toward the mean (average score), independent of any effect of an independent variable. It is especially a concern when assessing the skills of low performing individuals and comparing their skills to those with average or above average performance. For example, kindergarten children with the weakest reading skills at the start of the school year may show the greatest gains in their skills over the school year (e.g., between fall and spring assessments) independent of the instruction they received from their teachers.

External Validity refers to the extent to which the results of a study can be generalized to other settings (ecological validity), other people (population validity) and over time (historical validity). To assess whether a study has external validity, a research consumer should ask whether the findings apply to individuals whose place and circumstances differ from those of study participants. For example, a research study shows that a new curriculum improved reading comprehension of third-grade children in Iowa. As a research consumer, you want to ask whether this new curriculum may also be effective with third graders in New York or with children in other elementary grades. Studies that randomly select participants from the most diverse and representative populations and that are conducted in natural settings are more likely to have external validity. Threats to a study's external validity come from several sources, including:

• The list of all those in the population who are eligible to be sampled is incomplete or contains duplicates. For example, in a household survey, the list of housing units from which the sample will be drawn may be missing housing units (e.g., one or the two housing units in a duplex home). Or, an address list that will be used to drawn a sample may have some households listed twice.
• Some members of the population or members of certain groups may not be adequately represented in the sample (undercoverage). For example, a survey of adult education that relies on a published list of telephone numbers to select its sample may not get an accurate estimate of the participation of adults in different education programs because young adults who have higher rates of participation are less likely to have landlines and to have numbers published.
• Not all individuals who are sampled agree to participate in the study. When those who participate are different in meaningful ways from those who do not, there is the potential for the findings from the study to be biased (nonresponse bias). That is, the findings may not represent an accurate picture of the total population.
• Selecting samples using non-probability methods (e.g., purposive sample, volunteer samples), which tend to over- or under-represent certain groups in the population. For example, volunteer surveys on controversial topics such as school vouchers and sex education are more likely to overrepresent individuals with strong opinions. And, shopping mall surveys in general only represent the small group of individuals who are shopping at a particular location and at specific times.
• The findings from one study are difficult to replicate across locations, groups, and time. Despite the best efforts, it is extremely difficult to introduce and implement a program (treatment) exactly the same way in different locations. Similarly, it is difficult to conduct a study the same way each time. While researchers have control over many features of their studies, there are factors that are beyond their control (e.g., willingness of potential subjects to participate, scheduling conflicts that could lead to cancellations of data collection activities, data collection being suspended due to natural disasters). For example, the ability to carry out a study of school-age children's reading and math achievement in one school or in one school district may be affected by teachers' willingness to surrender instructional time for students to participate in a series of standardized assessments. In some cases, modifications to the study design (e.g., shorten the assessment, limit sensitive questions on a teacher or parent survey) must be made to accommodate the concerns of school and district leaders.
• Changes in the behaviors and reported attitudes of study participants as a result of being included in a research study (Hawthorne effect). For example, parents participating in a research study on children's early development may change the ways in which they support their child's learning at home.

Construct Validity refers to the degree to which a variable, test, questionnaire or instrument measures the theoretical concept that the researcher hopes to measure. To assess whether a study has construct validity, a research consumer should ask whether the study has adequately measured the key concepts in the study. For example, a study of reading comprehension should present convincing evidence that reading tests do indeed measure reading comprehension. Studies that use measures that have been independently validated in prior studies are more likely to have construct validity.

There are many threats to construct validity. These can arise during: the planning and design stage, assessment or survey administration, and data processing and analysis. Some are attributed to researchers and others to the subjects of the research. Here are some of the more common threats:

• Poorly defined constructs are perhaps the largest threat to construct validity. This applies to constructs that are too narrowly defined as well as those that are defined too broadly.
• Validity can also be affected by the measures a researcher chooses to measure a construct. Measures that include too few items to adequately represent the construct pose a threat as do measures that include items that tap other constructs. For example, a math assessment administered to four- and five-year old children that only includes items that require children to count would not be adequate to represent their math skills. A math assessment administered to this same group of children that was made up mostly of word problems would be tapping both their math and language skills. A valid measure should cover all aspects of the theoretical construct and only aspects of the theoretical construct.
• Assessment items or survey questions that are poorly written are threats to validity. Such items would include double-barreled questions that ask multiple questions within a single item (e.g., are you happily married and do you and your spouse argue?). Other examples of poorly written questions include those that use language that is above the reading level of most respondents, use professional jargon or are written in such a way as to trigger a socially desirable response.
• The validity of an assessment is threatened if there are too many items that are outside the ability of the individual being assessed (e.g., too many very easy items and too many very difficult items). For example, an early literacy assessment that only included passages that children were asked to read and answer questions about would not result in a valid assessment of children's early literacy.
• Threats that are introduced by interviewers and assessment staff. Actions by these individuals that can affect the reliability and thus the validity of the assessment occur when they deviate from the research protocol and when they signal a correct answer to the study participant through their actions. For example, an assessment of young children's English language vocabulary may specify that only responses in English are acceptable. However, when assessing bilingual children, some assessors comply with this rule while others accept responses in English or in the child's home language (e.g., Spanish). Assessors may unintentionally signal to children the correct responses on an assessment by 'staring' at the correct response to a multiple-choice item or by smiling and giving praise only when the child answers correctly.
• Threats to validity can also be introduced by the research participants. These would include participant apprehension or anxiety that could result in poorer performance on an assessment or to incorrect or ambiguous responses to a series of interview items. These threats must be taken seriously and addressed when administering standardized assessments to young children, many of whom will have limited experience with these types of tests. The language used when administering an assessment can also threaten its validity, if subjects do not have the language skills to understand what they are being asked to do and the language skills needed to respond.
• Coding errors - Coding errors that are systematic as compared to those that are random are especially problematic.
• Poor inter-coder or inter-rater reliability - When coding responses to open-ended survey items or assigning scores to behaviors observed during a video interaction, it is important that different coders or raters assign the same code or score for the same response or behavior. That is, the goal is high inter-coder or inter-rater reliability. When inter-coder or inter-rater reliability is poor, it can have an adverse effect on the validity of a measure. For example, the construct validity of an observation measure of the quality of parent-child interactions could be compromised should individual members of a group coding a set of videotaped mother-child interactions apply different standards as to what they deem as intrusive parenting practices.
• Inconsistencies in how data are analyzed and missing data handled - Missing data may be handled in a number of different ways, and the approach that is chosen could prove to be problematic for a construct, especially when the data are not missing at random. For example, if items tapping certain math skills are missing disproportionately, the validity of the measure could be jeopardized if a researcher assigns the mean score for those items or if he simply averages the scores for the non-missing items.

Evaluating Research Articles

Understanding research statistics, critical appraisal, help us improve the libguide.

Imagine for a moment that you are trying to answer a clinical (PICO) question regarding one of your patients/clients. Do you know how to determine if a research study is of high quality? Can you tell if it is applicable to your question? In evidence based practice, there are many things to look for in an article that will reveal its quality and relevance. This guide is a collection of resources and activities that will help you learn how to evaluate articles efficiently and accurately.

Is health research new to you? Or perhaps you're a little out of practice with reading it? The following questions will help illuminate an article's strengths or shortcomings. Ask them of yourself as you are reading an article:

• Is the article peer reviewed?
• Are there any conflicts of interest based on the author's affiliation or the funding source of the research?
• Are the research questions or objectives clearly defined?
• Is the study a systematic review or meta analysis?
• Is the study design appropriate for the research question?
• Is the sample size justified? Do the authors explain how it is representative of the wider population?
• Do the researchers describe the setting of data collection?
• Does the paper clearly describe the measurements used?
• Did the researchers use appropriate statistical measures?
• Are the research questions or objectives answered?
• Did the researchers account for confounding factors?
• Have the researchers only drawn conclusions about the groups represented in the research?
• Have the authors declared any conflicts of interest?

If the answer to these questions about an article you are reading are mostly YESes , then it's likely that the article is of decent quality. If the answers are most NOs , then it may be a good idea to move on to another article. If the YESes and NOs are roughly even, you'll have to decide for yourself if the article is good enough quality for you. Some factors, like a poor literature review, are not as important as the researchers neglecting to describe the measurements they used. As you read more research, you'll be able to more easily identify research that is well done vs. that which is not well done.

Determining if a research study has used appropriate statistical measures is one of the most critical and difficult steps in evaluating an article. The following links are great, quick resources for helping to better understand how to use statistics in health research.

• How to read a paper: Statistics for the non-statistician. II: “Significant” relations and their pitfalls This article continues the checklist of questions that will help you to appraise the statistical validity of a paper. Greenhalgh Trisha. How to read a paper: Statistics for the non-statistician. II: “Significant” relations and their pitfalls BMJ 1997; 315 :422 *On the PMC PDF, you need to scroll past the first article to get to this one.*
• A consumer's guide to subgroup analysis The extent to which a clinician should believe and act on the results of subgroup analyses of data from randomized trials or meta-analyses is controversial. Guidelines are provided in this paper for making these decisions.

Statistical Versus Clinical Significance

When appraising studies, it's important to consider both the clinical and statistical significance of the research. This video offers a quick explanation of why.

If you have a little more time, this video explores statistical and clinical significance in more detail, including examples of how to calculate an effect size.

• Statistical vs. Clinical Significance Transcript Transcript document for the Statistical vs. Clinical Significance video.
• Effect Size Transcript Transcript document for the Effect Size video.
• P Values, Statistical Significance & Clinical Significance This handout also explains clinical and statistical significance.
• Absolute versus relative risk – making sense of media stories Understanding the difference between relative and absolute risk is essential to understanding statistical tests commonly found in research articles.

Critical appraisal is the process of systematically evaluating research using established and transparent methods. In critical appraisal, health professionals use validated checklists/worksheets as tools to guide their assessment of the research. It is a more advanced way of evaluating research than the more basic method explained above. To learn more about critical appraisal or to access critical appraisal tools, visit the websites below.

• Last Updated: May 15, 2024 1:10 PM
• URL: https://libguides.massgeneral.org/evaluatingarticles

Evaluating Information

• Understanding Primary and Secondary Sources
• Exploring and Evaluating Popular, Trade, and Scholarly Sources

Reading a Scholarly Article

Common components of original research articles, while you read, reading strategies, reading for citations, further reading, learning objectives.

This page was created to help you:

Identify the different parts of a scholarly article

Efficiently analyze and evaluate scholarly articles for usefulness

This page will focus on reading scholarly articles — published reports on original research in the social sciences, humanities, and STEM fields. Reading and understanding this type of article can be challenging. This guide will help you develop these skills, which can be learned and improved upon with practice.

We will go over:

There are many different types of articles that may be found in scholarly journals and other academic publications. For more, see:

• Types of Information Sources

Reading a scholarly article isn’t like reading a novel, website, or newspaper article. It’s likely you won’t read and absorb it from beginning to end, all at once.

Instead, think of scholarly reading as inquiry, i.e., asking a series of questions as you do your research or read for class. Your reading should be guided by your class topic or your own research question or thesis.

For example, as you read, you might ask yourself:

• What questions does it help to answer, or what topics does it address?
• Are these relevant or useful to me?
• Does the article offer a helpful framework for understanding my topic or question (theoretical framework)?
• Do the authors use interesting or innovative methods to conduct their research that might be relevant to me?
• Does the article contain references I might consult for further information?

In Practice

Scanning and skimming are essential when reading scholarly articles, especially at the beginning stages of your research or when you have a lot of material in front of you.

Many scholarly articles are organized to help you scan and skim efficiently. The next time you need to read an article, practice scanning the following sections (where available) and skim their contents:

• The abstract: This summary provides a birds’ eye view of the article contents.
• The introduction:  What is the topic(s) of the research article? What is its main idea or question?
• The list of keywords or descriptors
• Methods: How did the author(s) go about answering their question/collecting their data?
• Section headings:  Stop and skim those sections you may find relevant.
• Figures:  Offer lots of information in quick visual format.
• The conclusion:  What are the findings and/or conclusions of this article?

Mark Up Your Text

Read with purpose.

• Scanning and skimming with a pen in hand can help to focus your reading.
• Use color for quick reference. Try highlighters or some sticky notes. Use different colors to represent different topics.
• Write in the margins, putting down thoughts and questions about the content as you read.
• Use digital markup features available in eBook platforms or third-party solutions, like Adobe Reader or Hypothes.is.

Categorize Information

Create your own informal system of organization. It doesn’t have to be complicated — start basic, and be sure it works for you.

• Jot down a few of your own keywords for each article. These keywords may correspond with important topics being addressed in class or in your research paper.  
• Write keywords on print copies or use the built-in note taking features in reference management tools like Zotero and EndNote.  
• Your keywords and system of organization may grow more complex the deeper you get into your reading.

Highlight words, terms, phrases, acronyms, etc. that are unfamiliar to you. You can highlight on the text or make a list in a notetaking program.

• Decide if the term is essential to your understanding of the article or if you can look it up later and keep scanning.

You may scan an article and discover that it isn’t what you thought it was about. Before you close the tab or delete that PDF, consider scanning the article one more time, specifically to look for citations that might be more on-target for your topic.  

You don’t need to look at every citation in the bibliography — you can look to the literature review to identify the core references that relate to your topic. Literature reviews are typically organized by subtopic within a research question or thesis. Find the paragraph or two that are closely aligned with your topic, make note of the author names, then locate those citations in the bibliography or footnote.

See the Find Articles page for what to do next:

• Find Articles

See the Citation Searching page for more on following a citation trail:

• Citation Searching
• Taking notes effectively. [blog post] Raul Pacheco-Vega, PhD
• How to read an academic paper. [video] UBCiSchool. 2013
• How to (seriously) read a scientific paper. (2016, March 21). Science | AAAS.
• How to read a paper. S. Keshav. 2007. SIGCOMM Comput. Commun. Rev. 37, 3 (July 2007), 83–84.

This guide was designed to help you:

• << Previous: Exploring and Evaluating Popular, Trade, and Scholarly Sources
• Last Updated: Feb 16, 2024 3:55 PM
• URL: https://libguides.brown.edu/evaluate

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

Greater Good Science Center • Magazine • In Action • In Education

10 Questions to Ask about Scientific Studies

Here at Greater Good , we cover research into social and emotional well-being, and we try to help people apply findings to their personal and professional lives. We are well aware that our business is a tricky one.

Summarizing scientific studies and applying them to people’s lives isn’t just difficult for the obvious reasons, like understanding and then explaining scientific jargon or methods to non-specialists. It’s also the case that context gets lost when we translate findings into stories, tips, and tools for a more meaningful life, especially when we push it all through the nuance-squashing machine of the Internet. Many people never read past the headlines, which intrinsically aim to overgeneralize and provoke interest. Because our articles can never be as comprehensive as the original studies, they almost always omit some crucial caveats, such as limitations acknowledged by the researchers. To get those, you need access to the studies themselves.

And it’s very common for findings to seem to contradict each other. For example, we recently covered an experiment that suggests stress reduces empathy—after having previously discussed other research suggesting that stress-prone people can be more empathic. Some readers asked: Which one is correct? (You’ll find my answer here .)

But probably the most important missing piece is the future. That may sound like a funny thing to say, but, in fact, a new study is not worth the PDF it’s printed on until its findings are replicated and validated by other studies—studies that haven’t yet happened. An experiment is merely interesting until time and testing turns its finding into a fact.

Scientists know this, and they are trained to react very skeptically to every new paper. They also expect to be greeted with skepticism when they present findings. Trust is good, but science isn’t about trust. It’s about verification.

However, journalists like me, and members of the general public, are often prone to treat every new study as though it represents the last word on the question addressed. This particular issue was highlighted last week by—wait for it—a new study that tried to reproduce 100 prior psychological studies to see if their findings held up. The result of the three-year initiative is chilling: The team, led by University of Virginia psychologist Brian Nosek, got the same results in only 36 percent of the experiments they replicated. This has led to some predictably provocative, overgeneralizing headlines implying that we shouldn’t take psychology seriously.

I don’t agree.

Despite all the mistakes and overblown claims and criticism and contradictions and arguments—or perhaps because of them—our knowledge of human brains and minds has expanded dramatically during the past century. Psychology and neuroscience have documented phenomena like cognitive dissonance, identified many of the brain structures that support our emotions, and proved the placebo effect and other dimensions of the mind-body connection, among other findings that have been tested over and over again.

These discoveries have helped us understand and treat the true causes of many illnesses. I’ve heard it argued that rising rates of diagnoses of mental illness constitute evidence that psychology is failing, but in fact, the opposite is true: We’re seeing more and better diagnoses of problems that would have compelled previous generations to dismiss people as “stupid” or “crazy” or “hyper” or “blue.” The important thing to bear in mind is that it took a very, very long time for science to come to these insights and treatments, following much trial and error.

Science isn’t a faith, but rather a method that takes time to unfold. That’s why it’s equally wrong to uncritically embrace everything you read, including what you are reading on this page.

Given the complexities and ambiguities of the scientific endeavor, is it possible for a non-scientist to strike a balance between wholesale dismissal and uncritical belief? Are there red flags to look for when you read about a study on a site like Greater Good or in a popular self-help book? If you do read one of the actual studies, how should you, as a non-scientist, gauge its credibility?

I drew on my own experience as a science journalist, and surveyed my colleagues here at the UC Berkeley Greater Good Science Center. We came up 10 questions you might ask when you read about the latest scientific findings. These are also questions we ask ourselves, before we cover a study.

1. Did the study appear in a peer-reviewed journal?

Peer review—submitting papers to other experts for independent review before acceptance—remains one of the best ways we have for ascertaining the basic seriousness of the study, and many scientists describe peer review as a truly humbling crucible. If a study didn’t go through this process, for whatever reason, it should be taken with a much bigger grain of salt.

2. Who was studied, where?

Animal experiments tell scientists a lot, but their applicability to our daily human lives will be limited. Similarly, if researchers only studied men, the conclusions might not be relevant to women, and vice versa.

This was actually a huge problem with Nosek’s effort to replicate other people’s experiments. In trying to replicate one German study, for example, they had to use different maps (ones that would be familiar to University of Virginia students) and change a scale measuring aggression to reflect American norms. This kind of variance could explain the different results. It may also suggest the limits of generalizing the results from one study to other populations not included within that study.

As a matter of approach, readers must remember that many psychological studies rely on WEIRD (Western, educated, industrialized, rich and democratic) samples, mainly college students, which creates an in-built bias in the discipline’s conclusions. Does that mean you should dismiss Western psychology? Of course not. It’s just the equivalent of a “Caution” or “Yield” sign on the road to understanding.

3. How big was the sample?

In general, the more participants in a study, the more valid its results. That said, a large sample is sometimes impossible or even undesirable for certain kinds of studies. This is especially true in expensive neuroscience experiments involving functional magnetic resonance imaging, or fMRI, scans.

And many mindfulness studies have scanned the brains of people with many thousands of hours of meditation experience—a relatively small group. Even in those cases, however, a study that looks at 30 experienced meditators is probably more solid than a similar one that scanned the brains of only 15.

4. Did the researchers control for key differences?

Diversity or gender balance aren’t necessarily virtues in a research study; it’s actually a good thing when a study population is as homogenous as possible, because it allows the researchers to limit the number of differences that might affect the result. A good researcher tries to compare apples to apples, and control for as many differences as possible in her analysis.

5. Was there a control group?

One of the first things to look for in methodology is whether the sample is randomized and involved a control group; this is especially important if a study is to suggest that a certain variable might actually cause a specific outcome, rather than just be correlated with it (see next point).

For example, were some in the sample randomly assigned a specific meditation practice while others weren’t? If the sample is large enough, randomized trials can produce solid conclusions. But, sometimes, a study will not have a control group because it’s ethically impossible. (Would people still divert a trolley to kill one person in order to save five lives, if their decision killed a real person, instead of just being a thought experiment? We’ll never know for sure!)

The conclusions may still provide some insight, but they need to be kept in perspective.

6. Did the researchers establish causality, correlation, dependence, or some other kind of relationship?

I often hear “Correlation is not causation” shouted as a kind of battle cry, to try to discredit a study. But correlation—the degree to which two or more measurements seem to change at the same time—is important, and is one step in eventually finding causation—that is, establishing a change in one variable directly triggers a change in another.

The important thing is to correctly identify the relationship.

7. Is the journalist, or even the scientist, overstating the result?

Language that suggests a fact is “proven” by one study or which promotes one solution for all people is most likely overstating the case. Sweeping generalizations of any kind often indicate a lack of humility that should be a red flag to readers. A study may very well “suggest” a certain conclusion but it rarely, if ever, “proves” it.

This is why we use a lot of cautious, hedging language in Greater Good , like “might” or “implies.”

8. Is there any conflict of interest suggested by the funding or the researchers’ affiliations?

A recent study found that you could drink lots of sugary beverages without fear of getting fat, as long as you exercised. The funder? Coca Cola, which eagerly promoted the results. This doesn’t mean the results are wrong. But it does suggest you should seek a second opinion .

9. Does the researcher seem to have an agenda?

Readers could understandably be skeptical of mindfulness meditation studies promoted by practicing Buddhists or experiments on the value of prayer conducted by Christians. Again, it doesn’t automatically mean that the conclusions are wrong. It does, however, raise the bar for peer review and replication. For example, it took hundreds of experiments before we could begin saying with confidence that mindfulness can indeed reduce stress.

10. Do the researchers acknowledge limitations and entertain alternative explanations?

Is the study focused on only one side of the story or one interpretation of the data? Has it failed to consider or refute alternative explanations? Do they demonstrate awareness of which questions are answered and which aren’t by their methods?

I summarize my personal stance as a non-scientist toward scientific findings as this: Curious, but skeptical. I take it all seriously and I take it all with a grain of salt. I judge it against my experience, knowing that my experience creates bias. I try to cultivate humility, doubt, and patience. I don’t always succeed; when I fail, I try to admit fault and forgive myself. My own understanding is imperfect, and I remind myself that one study is only one step in understanding. Above all, I try to bear in mind that science is a process, and that conclusions always raise more questions for us to answer.

About the Author

Jeremy Adam Smith

Uc berkeley.

Jeremy Adam Smith edits the GGSC's online magazine, Greater Good . He is also the author or coeditor of five books, including The Daddy Shift , Are We Born Racist? , and (most recently) The Gratitude Project: How the Science of Thankfulness Can Rewire Our Brains for Resilience, Optimism, and the Greater Good . Before joining the GGSC, Jeremy was a John S. Knight Journalism Fellow at Stanford University.

You May Also Enjoy

Challenges to Gratitude Research

Three Insights from the Cutting Edge of Compassion Research

Does Empathy Reduce Prejudice—or Promote It?

What is the Relationship Between Stress and Empathy?

Gratitude Research: Why Bother?

• YJBM Updates
• Editorial Board
• Colloquium Series
• Podcast Series
• Open Access
• Call for Papers
• PubMed Publications
• News & Views
• YJBM Podcasts (by topic)
• About YJBM PodSquad
• Why Publish in YJBM?
• Types of Articles Published
• Manuscript Submission Guidelines
• YJBM Ethical Guidelines

Points to Consider When Reviewing Articles

• Writing and Submitting a Review
• Join Our Peer Reviewer Database

INFORMATION FOR

• Residents & Fellows
• Researchers

General questions that Reviewers should keep in mind when reviewing articles are the following:

• Is the article of interest to the readers of YJBM ?
• What are the strengths and weaknesses of the manuscript?
• How can the Editors work with the Authors to improve the submitted manuscripts, if the topic and scope of the manuscript is of interest to YJBM readers?

The following contains detailed descriptions as to what should be included in each particular type of article as well as points that Reviewers should keep in mind when specifically reviewing each type of article.

YJBM will ask Reviewers to Peer Review the following types of submissions:

Download PDF

Frequently asked questions.

These manuscripts should present well-rounded studies reporting innovative advances that further knowledge about a topic of importance to the fields of biology or medicine. The conclusions of the Original Research Article should clearly be supported by the results. These can be submitted as either a full-length article (no more than 6,000 words, 8 figures, and 4 tables) or a brief communication (no more than 2,500 words, 3 figures, and 2 tables). Original Research Articles contain five sections: abstract, introduction, materials and methods, results and discussion.

Reviewers should consider the following questions:

• What is the overall aim of the research being presented? Is this clearly stated?
• Have the Authors clearly stated what they have identified in their research?
• Are the aims of the manuscript and the results of the data clearly and concisely stated in the abstract?
• Does the introduction provide sufficient background information to enable readers to better understand the problem being identified by the Authors?
• Have the Authors provided sufficient evidence for the claims they are making? If not, what further experiments or data needs to be included?
• Are similar claims published elsewhere? Have the Authors acknowledged these other publications? Have the Authors made it clear how the data presented in the Author’s manuscript is different or builds upon previously published data?
• Is the data presented of high quality and has it been analyzed correctly? If the analysis is incorrect, what should the Authors do to correct this?
• Do all the figures and tables help the reader better understand the manuscript? If not, which figures or tables should be removed and should anything be presented in their place?
• Is the methodology used presented in a clear and concise manner so that someone else can repeat the same experiments? If not, what further information needs to be provided?
• Do the conclusions match the data being presented?
• Have the Authors discussed the implications of their research in the discussion? Have they presented a balanced survey of the literature and information so their data is put into context?
• Is the manuscript accessible to readers who are not familiar with the topic? If not, what further information should the Authors include to improve the accessibility of their manuscript?
• Are all abbreviations used explained? Does the author use standard scientific abbreviations?

Case reports describe an unusual disease presentation, a new treatment, an unexpected drug interaction, a new diagnostic method, or a difficult diagnosis. Case reports should include relevant positive and negative findings from history, examination and investigation, and can include clinical photographs. Additionally, the Author must make it clear what the case adds to the field of medicine and include an up-to-date review of all previous cases. These articles should be no more than 5,000 words, with no more than 6 figures and 3 tables. Case Reports contain five sections: abstract; introduction; case presentation that includes clinical presentation, observations, test results, and accompanying figures; discussion; and conclusions.

• Does the abstract clearly and concisely state the aim of the case report, the findings of the report, and its implications?
• Does the introduction provide enough details for readers who are not familiar with a particular disease/treatment/drug/diagnostic method to make the report accessible to them?
• Does the manuscript clearly state what the case presentation is and what was observed so that someone can use this description to identify similar symptoms or presentations in another patient?
• Are the figures and tables presented clearly explained and annotated? Do they provide useful information to the reader or can specific figures/tables be omitted and/or replaced by another figure/table?
• Are the data presented accurately analyzed and reported in the text? If not, how can the Author improve on this?
• Do the conclusions match the data presented?
• Does the discussion include information of similar case reports and how this current report will help with treatment of a disease/presentation/use of a particular drug?

Reviews provide a reasoned survey and examination of a particular subject of research in biology or medicine. These can be submitted as a mini-review (less than 2,500 words, 3 figures, and 1 table) or a long review (no more than 6,000 words, 6 figures, and 3 tables). They should include critical assessment of the works cited, explanations of conflicts in the literature, and analysis of the field. The conclusion must discuss in detail the limitations of current knowledge, future directions to be pursued in research, and the overall importance of the topic in medicine or biology. Reviews contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Is the review accessible to readers of YJBM who are not familiar with the topic presented?
• Does the abstract accurately summarize the contents of the review?
• Does the introduction clearly state what the focus of the review will be?
• Are the facts reported in the review accurate?
• Does the Author use the most recent literature available to put together this review?
• Is the review split up under relevant subheadings to make it easier for the readers to access the article?
• Does the Author provide balanced viewpoints on a specific topic if there is debate over the topic in the literature?
• Are the figures or tables included relevant to the review and enable the readers to better understand the manuscript? Are there further figures/tables that could be included?
• Do the conclusions and outlooks outline where further research can be done on the topic?

Perspectives provide a personal view on medical or biomedical topics in a clear narrative voice. Articles can relate personal experiences, historical perspective, or profile people or topics important to medicine and biology. Long perspectives should be no more than 6,000 words and contain no more than 2 tables. Brief opinion pieces should be no more than 2,500 words and contain no more than 2 tables. Perspectives contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Does the abstract accurately and concisely summarize the main points provided in the manuscript?
• Does the introduction provide enough information so that the reader can understand the article if he or she were not familiar with the topic?
• Are there specific areas in which the Author can provide more detail to help the reader better understand the manuscript? Or are there places where the author has provided too much detail that detracts from the main point?
• If necessary, does the Author divide the article into specific topics to help the reader better access the article? If not, how should the Author break up the article under specific topics?
• Do the conclusions follow from the information provided by the Author?
• Does the Author reflect and provide lessons learned from a specific personal experience/historical event/work of a specific person?

Analyses provide an in-depth prospective and informed analysis of a policy, major advance, or historical description of a topic related to biology or medicine. These articles should be no more than 6,000 words with no more than 3 figures and 1 table. Analyses contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Does the abstract accurately summarize the contents of the manuscript?
• Does the introduction provide enough information if the readers are not familiar with the topic being addressed?
• Are there specific areas in which the Author can provide more detail to help the reader better understand the manuscript? Or are there places where the Author has provided too much detail that detracts from the main point?

Profiles describe a notable person in the fields of science or medicine. These articles should contextualize the individual’s contributions to the field at large as well as provide some personal and historical background on the person being described. More specifically, this should be done by describing what was known at the time of the individual’s discovery/contribution and how that finding contributes to the field as it stands today. These pieces should be no more than 5,000 words, with up to 6 figures, and 3 tables. The article should include the following: abstract, introduction, topics (with headings and subheadings), and conclusions.

• Does the Author provide information about the person of interest’s background, i.e., where they are from, where they were educated, etc.?
• Does the Author indicate how the person focused on became interested or involved in the subject that he or she became famous for?
• Does the Author provide information on other people who may have helped the person in his or her achievements?
• Does the Author provide information on the history of the topic before the person became involved?
• Does the Author provide information on how the person’s findings affected the field being discussed?
• Does the introduction provide enough information to the readers, should they not be familiar with the topic being addressed?

Interviews may be presented as either a transcript of an interview with questions and answers or as a personal reflection. If the latter, the Author must indicate that the article is based on an interview given. These pieces should be no more than 5,000 words and contain no more than 3 figures and 2 tables. The articles should include: abstract, introduction, questions and answers clearly indicated by subheadings or topics (with heading and subheadings), and conclusions.

• Does the Author provide relevant information to describe who the person is whom they have chosen to interview?
• Does the Author explain why he or she has chosen the person being interviewed?
• Does the Author explain why he or she has decided to focus on a specific topic in the interview?
• Are the questions relevant? Are there more questions that the Author should have asked? Are there questions that the Author has asked that are not necessary?
• If necessary, does the Author divide the article into specific topics to help the reader better access the article? If not, how should the author break up the article under specific topics?
• Does the Author accurately summarize the contents of the interview as well as specific lesson learned, if relevant, in the conclusions?

90 Questions to Ask a Researcher

Embarking on a journey to unravel the mysteries of research can be both exhilarating and daunting. Whether you’re a budding scholar, a curious journalist, or simply an individual with a thirst for knowledge, probing the minds of those at the forefront of discovery can provide invaluable insights.

This article offers an array of thought-provoking questions meticulously crafted to delve deep into the essence of research.

Table of Contents

Understanding Research Objectives

• What inspired the research question you’re exploring?
• Can you briefly describe the main goal of your research?
• How does your research fit within the broader field of study?
• Who are the intended beneficiaries of your research findings?
• Did the initial objectives evolve as the research progressed?
• How do you ensure your research objectives are achievable?
• What is the scope of your research?
• How did you narrow down your research questions?
• What significant problem does your research seek to address?
• What are the anticipated outcomes of your research?
• How do you prioritize different research objectives?
• What hypotheses are you testing in your research?
• Can you clarify any terms specific to your research objectives?
• How do your research objectives align with current trends in the field?
• What challenges did you face in defining your research objectives?

Exploring Methodology and Design

• What research methodology did you choose, and why?
• How did you ensure the reliability and validity of your methods?
• Can you explain the process of data collection in your research?
• Did you encounter any obstacles in your research design?
• How did your methodology affect the interpretation of results?
• What tools or technologies did you use in your research?
• How did you address potential biases in your research design?
• What criteria did you use for selecting your study sample?
• How do you manage the ethical considerations in your research?
• What measures did you take to ensure data security and privacy?
• How is your approach different from existing methodologies?
• Did you require any special approvals for your research methods?
• How do you keep your methodology transparent and replicable?
• Have you piloted your research design, and what were the results?
• Can you walk me through the timeline of the research process?

Discussing Findings and Interpretations

• What are the key findings of your research?
• Were there any surprising results you encountered?
• How do your findings contribute to the field?
• What do you believe is the reason behind these outcomes?
• How do your results compare to those of similar studies?
• What limitations should we consider when interpreting your findings?
• Can you discuss any patterns or trends revealed in your research?
• How might your findings influence future research?
• How robust are your findings to different analytical methods?
• Were any of your hypotheses not supported by the data?
• How do you plan to share and publish your findings?
• What is the potential for real-world application of your research?
• How do you validate the interpretations of your data?
• In what ways have you engaged with other researchers about your findings?
• What follow-up studies would you suggest based on your results?

Assessing Impact and Relevance

• How does your research address societal or environmental needs?
• In what ways could your research potentially affect industry practices?
• What long-term changes do you hope your research will inspire?
• How relevant is your research in today’s context?
• What is the envisioned impact on policy or public discourse?
• How do you gauge the success of your research’s impact?
• Who are the stakeholders most interested in your research?
• How do you disseminate your research to maximize impact?
• Can your research findings be generalized across different contexts?
• What collaborations did you form to enhance the impact of your research?
• How do you measure the social return on investment for your research?
• What has been the feedback from the community affected by your research?
• How has your research contributed to advancements in technology or science?
• What part of your research do you think will have the greatest impact?
• Are there any unintended consequences that may arise from your research?

Evaluating Credibility and Ethics

• How do you ensure the credibility of your research findings?
• What kind of peer review or scrutiny has your research undergone?
• Have you published in open-access journals or other accessible platforms?
• Can you explain the ethical considerations relevant to your research?
• How would you address any conflicts of interest in your work?
• How do you ensure that your research benefits are shared fairly?
• What steps do you take to maintain transparency in your research?
• How have you dealt with any ethical dilemmas during your study?
• In what ways do you work to maintain participant confidentiality?
• Can you discuss the ethical governance of your research project?
• What safeguards are in place for the ethical use of data?
• How does your research comply with legal and regulatory standards?
• Are there any socio-cultural sensitivities you had to consider in your research?
• How might your research confront or challenge ethical norms?
• What is your approach to obtaining informed consent?

Personal Insights and Journey in Research

• What inspired you to become a researcher in your field?
• Can you share a pivotal moment in your research career?
• How do you stay motivated when facing research challenges?
• What are the most rewarding aspects of conducting research?
• How do you balance personal biases and professional objectivity?
• What lessons have you learned from your time in research?
• How do you approach collaborative research projects?
• What are your top strategies for overcoming research obstacles?
• Can you describe a significant breakthrough you had in your research?
• How do you keep up-to-date with advancements in your field?
• What advice would you give to aspiring researchers?
• Can you share a mistake you made and how you addressed it?
• What has been the most unexpected turn in your research journey?
• How do you handle skepticism or criticism of your research?
• What personal qualities do you think are essential for a successful researcher?

Frequently Asked Questions

Why is it important to ask about a researcher’s personal insights.

Learning about a researcher’s personal insights provides context that enriches your understanding of their work. It can reveal the human element behind the research, including motivation, challenges, and the passion driving their inquiries.

How can I probe into the relevance and real-world application of research?

Ask direct questions about how the research tackles contemporary issues, its societal benefits, and its potential for practical application. Discussing envisioned impacts on policy or technology can also highlight the research’s real-world significance.

Final Thoughts

Beyond the data and analysis, these conversations are a portal into the multifaceted world of research – a blend of empirical rigor, ethical considerations, and personal dedication.

May these engaging inquiries lead you to profound discoveries and a deeper appreciation for the painstaking yet rewarding odyssey that researchers embark upon to advance human knowledge.

How useful was this post?

Click on a star to rate it!

As you found this post useful...

Share it on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Bea Mariel Saulo

Get science-backed answers as you write with Paperpal's Research feature

How to Write a Research Question: Types and Examples 

The first step in any research project is framing the research question. It can be considered the core of any systematic investigation as the research outcomes are tied to asking the right questions. Thus, this primary interrogation point sets the pace for your research as it helps collect relevant and insightful information that ultimately influences your work.   

Typically, the research question guides the stages of inquiry, analysis, and reporting. Depending on the use of quantifiable or quantitative data, research questions are broadly categorized into quantitative or qualitative research questions. Both types of research questions can be used independently or together, considering the overall focus and objectives of your research.  

What is a research question?

A research question is a clear, focused, concise, and arguable question on which your research and writing are centered. 1 It states various aspects of the study, including the population and variables to be studied and the problem the study addresses. These questions also set the boundaries of the study, ensuring cohesion. 

Designing the research question is a dynamic process where the researcher can change or refine the research question as they review related literature and develop a framework for the study. Depending on the scale of your research, the study can include single or multiple research questions. 

A good research question has the following features: 

• It is relevant to the chosen field of study. 
• The question posed is arguable and open for debate, requiring synthesizing and analysis of ideas. 
• It is focused and concisely framed. 
• A feasible solution is possible within the given practical constraint and timeframe. 

A poorly formulated research question poses several risks. 1   

• Researchers can adopt an erroneous design. 
• It can create confusion and hinder the thought process, including developing a clear protocol.  
• It can jeopardize publication efforts.  
• It causes difficulty in determining the relevance of the study findings.  
• It causes difficulty in whether the study fulfils the inclusion criteria for systematic review and meta-analysis. This creates challenges in determining whether additional studies or data collection is needed to answer the question.  
• Readers may fail to understand the objective of the study. This reduces the likelihood of the study being cited by others. 

Now that you know “What is a research question?”, let’s look at the different types of research questions. 

Types of research questions

Depending on the type of research to be done, research questions can be classified broadly into quantitative, qualitative, or mixed-methods studies. Knowing the type of research helps determine the best type of research question that reflects the direction and epistemological underpinnings of your research. 

The structure and wording of quantitative 2 and qualitative research 3 questions differ significantly. The quantitative study looks at causal relationships, whereas the qualitative study aims at exploring a phenomenon. 

• Quantitative research questions:  
• Seeks to investigate social, familial, or educational experiences or processes in a particular context and/or location.  
• Answers ‘how,’ ‘what,’ or ‘why’ questions. 
• Investigates connections, relations, or comparisons between independent and dependent variables. 

Quantitative research questions can be further categorized into descriptive, comparative, and relationship, as explained in the Table below. 

• Qualitative research questions  

Qualitative research questions are adaptable, non-directional, and more flexible. It concerns broad areas of research or more specific areas of study to discover, explain, or explore a phenomenon. These are further classified as follows: 

• Mixed-methods studies  

Mixed-methods studies use both quantitative and qualitative research questions to answer your research question. Mixed methods provide a complete picture than standalone quantitative or qualitative research, as it integrates the benefits of both methods. Mixed methods research is often used in multidisciplinary settings and complex situational or societal research, especially in the behavioral, health, and social science fields. 

What makes a good research question

A good research question should be clear and focused to guide your research. It should synthesize multiple sources to present your unique argument, and should ideally be something that you are interested in. But avoid questions that can be answered in a few factual statements. The following are the main attributes of a good research question. 

• Specific: The research question should not be a fishing expedition performed in the hopes that some new information will be found that will benefit the researcher. The central research question should work with your research problem to keep your work focused. If using multiple questions, they should all tie back to the central aim. 
• Measurable: The research question must be answerable using quantitative and/or qualitative data or from scholarly sources to develop your research question. If such data is impossible to access, it is better to rethink your question. 
• Attainable: Ensure you have enough time and resources to do all research required to answer your question. If it seems you will not be able to gain access to the data you need, consider narrowing down your question to be more specific. 
• You have the expertise 
• You have the equipment and resources 
• Realistic: Developing your research question should be based on initial reading about your topic. It should focus on addressing a problem or gap in the existing knowledge in your field or discipline. 
• Based on some sort of rational physics 
• Can be done in a reasonable time frame 
• Timely: The research question should contribute to an existing and current debate in your field or in society at large. It should produce knowledge that future researchers or practitioners can later build on. 
• Novel 
• Based on current technologies. 
• Important to answer current problems or concerns. 
• Lead to new directions. 
• Important: Your question should have some aspect of originality. Incremental research is as important as exploring disruptive technologies. For example, you can focus on a specific location or explore a new angle. 
• Meaningful whether the answer is “Yes” or “No.” Closed-ended, yes/no questions are too simple to work as good research questions. Such questions do not provide enough scope for robust investigation and discussion. A good research question requires original data, synthesis of multiple sources, and original interpretation and argumentation before providing an answer. 

Steps for developing a good research question

The importance of research questions cannot be understated. When drafting a research question, use the following frameworks to guide the components of your question to ease the process. 4  

• Determine the requirements: Before constructing a good research question, set your research requirements. What is the purpose? Is it descriptive, comparative, or explorative research? Determining the research aim will help you choose the most appropriate topic and word your question appropriately. 
• Select a broad research topic: Identify a broader subject area of interest that requires investigation. Techniques such as brainstorming or concept mapping can help identify relevant connections and themes within a broad research topic. For example, how to learn and help students learn. 
• Perform preliminary investigation: Preliminary research is needed to obtain up-to-date and relevant knowledge on your topic. It also helps identify issues currently being discussed from which information gaps can be identified. 
• Narrow your focus: Narrow the scope and focus of your research to a specific niche. This involves focusing on gaps in existing knowledge or recent literature or extending or complementing the findings of existing literature. Another approach involves constructing strong research questions that challenge your views or knowledge of the area of study (Example: Is learning consistent with the existing learning theory and research). 
• Identify the research problem: Once the research question has been framed, one should evaluate it. This is to realize the importance of the research questions and if there is a need for more revising (Example: How do your beliefs on learning theory and research impact your instructional practices). 

How to write a research question

Those struggling to understand how to write a research question, these simple steps can help you simplify the process of writing a research question. 

Sample Research Questions

The following are some bad and good research question examples 

• Example 1 
• Example 2 

References:  

• Thabane, L., Thomas, T., Ye, C., & Paul, J. (2009). Posing the research question: not so simple.  Canadian Journal of Anesthesia/Journal canadien d’anesthésie ,  56 (1), 71-79. 
• Rutberg, S., & Bouikidis, C. D. (2018). Focusing on the fundamentals: A simplistic differentiation between qualitative and quantitative research.  Nephrology Nursing Journal ,  45 (2), 209-213. 
• Kyngäs, H. (2020). Qualitative research and content analysis.  The application of content analysis in nursing science research , 3-11. 
• Mattick, K., Johnston, J., & de la Croix, A. (2018). How to… write a good research question.  The clinical teacher ,  15 (2), 104-108. 
• Fandino, W. (2019). Formulating a good research question: Pearls and pitfalls.  Indian Journal of Anaesthesia ,  63 (8), 611. 
• Richardson, W. S., Wilson, M. C., Nishikawa, J., & Hayward, R. S. (1995). The well-built clinical question: a key to evidence-based decisions.  ACP journal club ,  123 (3), A12-A13 

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• Scientific Writing Style Guides Explained
• Ethical Research Practices For Research with Human Subjects
• 8 Most Effective Ways to Increase Motivation for Thesis Writing 
• 6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Transitive and Intransitive Verbs in the World of Research

Language and grammar rules for academic writing, you may also like, academic editing: how to self-edit academic text with..., measuring academic success: definition & strategies for excellence, phd qualifying exam: tips for success , quillbot review: features, pricing, and free alternatives, what is an academic paper types and elements , 9 steps to publish a research paper, what are the different types of research papers, how to make translating academic papers less challenging, 6 tips for post-doc researchers to take their..., presenting research data effectively through tables and figures.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Reading Research Effectively
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Reading a Scholarly Article or Research Paper

Identifying a research problem to investigate usually requires a preliminary search for and critical review of the literature in order to gain an understanding about how scholars have examined a topic. Scholars rarely structure research studies in a way that can be followed like a story; they are complex and detail-intensive and often written in a descriptive and conclusive narrative form. However, in the social and behavioral sciences, journal articles and stand-alone research reports are generally organized in a consistent format that makes it easier to compare and contrast studies and to interpret their contents.

General Reading Strategies

W hen you first read an article or research paper, focus on asking specific questions about each section. This strategy can help with overall comprehension and with understanding how the content relates [or does not relate] to the problem you want to investigate. As you review more and more studies, the process of understanding and critically evaluating the research will become easier because the content of what you review will begin to coalescence around common themes and patterns of analysis. Below are recommendations on how to read each section of a research paper effectively. Note that the sections to read are out of order from how you will find them organized in a journal article or research paper.

1.  Abstract

The abstract summarizes the background, methods, results, discussion, and conclusions of a scholarly article or research paper. Use the abstract to filter out sources that may have appeared useful when you began searching for information but, in reality, are not relevant. Questions to consider when reading the abstract are:

• Is this study related to my question or area of research?
• What is this study about and why is it being done ?
• What is the working hypothesis or underlying thesis?
• What is the primary finding of the study?
• Are there words or terminology that I can use to either narrow or broaden the parameters of my search for more information?

2.  Introduction

If, after reading the abstract, you believe the paper may be useful, focus on examining the research problem and identifying the questions the author is trying to address. This information is usually located within the first few paragraphs of the introduction or in the concluding paragraph. Look for information about how and in what way this relates to what you are investigating. In addition to the research problem, the introduction should provide the main argument and theoretical framework of the study and, in the last paragraphs of the introduction, describe what the author(s) intend to accomplish. Questions to consider when reading the introduction include:

• What is this study trying to prove or disprove?
• What is the author(s) trying to test or demonstrate?
• What do we already know about this topic and what gaps does this study try to fill or contribute a new understanding to the research problem?
• Why should I care about what is being investigated?
• Will this study tell me anything new related to the research problem I am investigating?

3.  Literature Review

The literature review describes and critically evaluates what is already known about a topic. Read the literature review to obtain a big picture perspective about how the topic has been studied and to begin the process of seeing where your potential study fits within the domain of prior research. Questions to consider when reading the literature review include:

• W hat other research has been conducted about this topic and what are the main themes that have emerged?
• What does prior research reveal about what is already known about the topic and what remains to be discovered?
• What have been the most important past findings about the research problem?
• How has prior research led the author(s) to conduct this particular study?
• Is there any prior research that is unique or groundbreaking?
• Are there any studies I could use as a model for designing and organizing my own study?

4.  Discussion/Conclusion

The discussion and conclusion are usually the last two sections of text in a scholarly article or research report. They reveal how the author(s) interpreted the findings of their research and presented recommendations or courses of action based on those findings. Often in the conclusion, the author(s) highlight recommendations for further research that can be used to develop your own study. Questions to consider when reading the discussion and conclusion sections include:

• What is the overall meaning of the study and why is this important? [i.e., how have the author(s) addressed the " So What? " question].
• What do you find to be the most important ways that the findings have been interpreted?
• What are the weaknesses in their argument?
• Do you believe conclusions about the significance of the study and its findings are valid?
• What limitations of the study do the author(s) describe and how might this help formulate my own research?
• Does the conclusion contain any recommendations for future research?

5.  Methods/Methodology

The methods section describes the materials, techniques, and procedures for gathering information used to examine the research problem. If what you have read so far closely supports your understanding of the topic, then move on to examining how the author(s) gathered information during the research process. Questions to consider when reading the methods section include:

• Did the study use qualitative [based on interviews, observations, content analysis], quantitative [based on statistical analysis], or a mixed-methods approach to examining the research problem?
• What was the type of information or data used?
• Could this method of analysis be repeated and can I adopt the same approach?
• Is enough information available to repeat the study or should new data be found to expand or improve understanding of the research problem?

6.  Results

After reading the above sections, you should have a clear understanding of the general findings of the study. Therefore, read the results section to identify how key findings were discussed in relation to the research problem. If any non-textual elements [e.g., graphs, charts, tables, etc.] are confusing, focus on the explanations about them in the text. Questions to consider when reading the results section include:

• W hat did the author(s) find and how did they find it?
• Does the author(s) highlight any findings as most significant?
• Are the results presented in a factual and unbiased way?
• Does the analysis of results in the discussion section agree with how the results are presented?
• Is all the data present and did the author(s) adequately address gaps?
• What conclusions do you formulate from this data and does it match with the author's conclusions?

7.  References

The references list the sources used by the author(s) to document what prior research and information was used when conducting the study. After reviewing the article or research paper, use the references to identify additional sources of information on the topic and to examine critically how these sources supported the overall research agenda. Questions to consider when reading the references include:

• Do the sources cited by the author(s) reflect a diversity of disciplinary viewpoints, i.e., are the sources all from a particular field of study or do the sources reflect multiple areas of study?
• Are there any unique or interesting sources that could be incorporated into my study?
• What other authors are respected in this field, i.e., who has multiple works cited or is cited most often by others?
• What other research should I review to clarify any remaining issues or that I need more information about?

NOTE :  A final strategy in reviewing research is to copy and paste the title of the source [journal article, book, research report] into Google Scholar . If it appears, look for a "cited by" followed by a hyperlinked number [e.g., Cited by 45]. This number indicates how many times the study has been subsequently cited in other, more recently published works. This strategy, known as citation tracking, can be an effective means of expanding your review of pertinent literature based on a study you have found useful and how scholars have cited it. The same strategies described above can be applied to reading articles you find in the list of cited by references.

Reading Tip

Specific Reading Strategies

Effectively reading scholarly research is an acquired skill that involves attention to detail and an ability to comprehend complex ideas, data, and theoretical concepts in a way that applies logically to the research problem you are investigating. Here are some specific reading strategies to consider.

As You are Reading

• Focus on information that is most relevant to the research problem; skim over the other parts.
• As noted above, read content out of order! This isn't a novel; you want to start with the spoiler to quickly assess the relevance of the study.
• Think critically about what you read and seek to build your own arguments; not everything may be entirely valid, examined effectively, or thoroughly investigated.
• Look up the definitions of unfamiliar words, concepts, or terminology. A good scholarly source is Credo Reference .

Taking notes as you read will save time when you go back to examine your sources. Here are some suggestions:

• Mark or highlight important text as you read [e.g., you can use the highlight text  feature in a PDF document]
• Take notes in the margins [e.g., Adobe Reader offers pop-up sticky notes].
• Highlight important quotations; consider using different colors to differentiate between quotes and other types of important text.
• Summarize key points about the study at the end of the paper. To save time, these can be in the form of a concise bulleted list of statements [e.g., intro has provides historical background; lit review has important sources; good conclusions].

Write down thoughts that come to mind that may help clarify your understanding of the research problem. Here are some examples of questions to ask yourself:

• Do I understand all of the terminology and key concepts?
• Do I understand the parts of this study most relevant to my topic?
• What specific problem does the research address and why is it important?
• Are there any issues or perspectives the author(s) did not consider?
• Do I have any reason to question the validity or reliability of this research?
• How do the findings relate to my research interests and to other works which I have read?

Adapted from text originally created by Holly Burt, Behavioral Sciences Librarian, USC Libraries, April 2018.

Another Reading Tip

When is it Important to Read the Entire Article or Research Paper

Laubepin argues, "Very few articles in a field are so important that every word needs to be read carefully." However, this implies that some studies are worth reading carefully. As painful and time-consuming as it may seem, there are valid reasons for reading a study in its entirety from beginning to end. Here are some examples:

• Studies Published Very Recently .  The author(s) of a recent, well written study will provide a survey of the most important or impactful prior research in the literature review section. This can establish an understanding of how scholars in the past addressed the research problem. In addition, the most recently published sources will highlight what is currently known and what gaps in understanding currently exist about a topic, usually in the form of the need for further research in the conclusion .
• Surveys of the Research Problem .  Some papers provide a comprehensive analytical overview of the research problem. Reading this type of study can help you understand underlying issues and discover why scholars have chosen to investigate the topic. This is particularly important if the study was published very recently because the author(s) should cite all or most of the key prior research on the topic. Note that, if it is a long-standing problem, there may be studies that specifically review the literature to identify gaps that remain. These studies often include the word review in their title [e.g., Hügel, Stephan, and Anna R. Davies. "Public Participation, Engagement, and Climate Change Adaptation: A Review of the Research Literature." Wiley Interdisciplinary Reviews: Climate Change 11 (July-August 2020): https://doi.org/10.1002/ wcc.645].
• Highly Cited .  If you keep coming across the same citation to a study while you are reviewing the literature, this implies it was foundational in establishing an understanding of the research problem or the study had a significant impact within the literature [positive or negative]. Carefully reading a highly cited source can help you understand how the topic emerged and motivated scholars to further investigate the problem. It also could be a study you need to cite as foundational in your own paper to demonstrate to the reader that you understand the roots of the problem.
• Historical Overview .  Knowing the historical background of a research problem may not be the focus of your analysis. Nevertheless, carefully reading a study that provides a thorough description and analysis of the history behind an event, issue, or phenomenon can add important context to understanding the topic and what aspect of the problem you may want to examine further.
• Innovative Methodological Design .  Some studies are significant and worth reading in their entirety because the author(s) designed a unique or innovative approach to researching the problem. This may justify reading the entire study because it can motivate you to think creatively about pursuing an alternative or non-traditional approach to examining your topic of interest. These types of studies are generally easy to identify because they are often cited in others works because of their unique approach to studying the research problem.
• Cross-disciplinary Approach .  R eviewing studies produced outside of your discipline is an essential component of investigating research problems in the social and behavioral sciences. Consider reading a study that was conducted by author(s) based in a different discipline [e.g., an anthropologist studying political cultures; a study of hiring practices in companies published in a sociology journal]. This approach can generate a new understanding or a unique perspective about the topic . If you are not sure how to search for studies published in a discipline outside of your major or of the course you are taking, contact a librarian for assistance.

Laubepin, Frederique. How to Read (and Understand) a Social Science Journal Article . Inter-University Consortium for Political and Social Research (ISPSR), 2013; Shon, Phillip Chong Ho. How to Read Journal Articles in the Social Sciences: A Very Practical Guide for Students . 2nd edition. Thousand Oaks, CA: Sage, 2015; Lockhart, Tara, and Mary Soliday. "The Critical Place of Reading in Writing Transfer (and Beyond): A Report of Student Experiences." Pedagogy 16 (2016): 23-37; Maguire, Moira, Ann Everitt Reynolds, and Brid Delahunt. "Reading to Be: The Role of Academic Reading in Emergent Academic and Professional Student Identities." Journal of University Teaching and Learning Practice 17 (2020): 5-12.

• << Previous: 1. Choosing a Research Problem
• Next: Narrowing a Topic Idea >>
• Last Updated: May 15, 2024 9:53 AM
• URL: https://libguides.usc.edu/writingguide

• Ethics & Leadership
• Fact-Checking
• Media Literacy
• The Craig Newmark Center
• Reporting & Editing
• Ethics & Trust
• Tech & Tools
• Business & Work
• Educators & Students
• Training Catalog
• Custom Teaching
• For ACES Members
• All Categories
• Broadcast & Visual Journalism
• Fact-Checking & Media Literacy
• In-newsroom
• Memphis, Tenn.
• Minneapolis, Minn.
• St. Petersburg, Fla.
• Washington, D.C.
• Poynter ACES Introductory Certificate in Editing
• Poynter ACES Intermediate Certificate in Editing
• Ethics & Trust Articles
• Get Ethics Advice
• Fact-Checking Articles
• International Fact-Checking Day
• Teen Fact-Checking Network
• International
• Media Literacy Training
• MediaWise Resources
• Ambassadors
• MediaWise in the News

Support responsible news and fact-based information today!

Questions to ask scientists about scientific research

Journalists talk to a variety of sources about scientific research. Different types of sources have different qualifications and can provide different types of information. Here are some considerations and questions when you are interviewing scientists.

To understand their academic background, check:

• Educational background
• Number of publications
• Lack of retractions on RetractionWatch.com

Questions to ask:

• Why does this study matter?
• Why did you want to do this study?
• How does this study relate to your other work?
• What was your role in the study?
• What surprised you the most?
• Did you change any of your or your family’s habits as a result of anything you learned from this study?
• What are the economic stakes of the study/topic from your perspective? Your employer’s or funder’s perspective? (i.e., whose profits / earnings might be harmed or helped by this study? Who is invested in it?)
• What comes next? (More studies? Different funders? Legislation?)

Taken from Whose Truth? Tools for Smart Science Journalism in the Digital Age , a self-directed course by Elissa Yancey at Poynter NewsU .

Take the full course

Have you missed a Coffee Break Course? Here’s our complete lineup . And follow along on Twitter at #coffeebreakcourse.

Opinion | We’re set for the presidential debates. Now what?

The first debate is set for June 27, much earlier than usual. It will appear on CNN. Jake Tapper and Dana Bash will moderate.

The World Health Organization’s pandemic plan won’t end free speech

A draft of the WHO’s pandemic accord says that the document will be used with respect to individual’s personal freedoms

Nobel Prize winner Maria Ressa, experts on democracy and AI to lead GlobalFact as keynote speakers

Other speakers at the fact-checking summit include Steve Levitsky, Nikita Roy and Craig Silverman

Opinion | Q&A: Fox News foreign correspondent Trey Yingst on the perils of covering war

Yingst, 30, has spent the past several years in some of the most dangerous places in the world.

Opinion | Planning a town hall with candidates? Try a reverse town hall instead.

They spend so much time talking. It’s their turn to listen.

Start your day informed and inspired.

Get the Poynter newsletter that's right for you.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Indian Assoc Pediatr Surg
• v.24(1); Jan-Mar 2019

Formulation of Research Question – Stepwise Approach

Simmi k. ratan.

Department of Pediatric Surgery, Maulana Azad Medical College, New Delhi, India

1 Department of Community Medicine, North Delhi Municipal Corporation Medical College, New Delhi, India

2 Department of Pediatric Surgery, Batra Hospital and Research Centre, New Delhi, India

Formulation of research question (RQ) is an essentiality before starting any research. It aims to explore an existing uncertainty in an area of concern and points to a need for deliberate investigation. It is, therefore, pertinent to formulate a good RQ. The present paper aims to discuss the process of formulation of RQ with stepwise approach. The characteristics of good RQ are expressed by acronym “FINERMAPS” expanded as feasible, interesting, novel, ethical, relevant, manageable, appropriate, potential value, publishability, and systematic. A RQ can address different formats depending on the aspect to be evaluated. Based on this, there can be different types of RQ such as based on the existence of the phenomenon, description and classification, composition, relationship, comparative, and causality. To develop a RQ, one needs to begin by identifying the subject of interest and then do preliminary research on that subject. The researcher then defines what still needs to be known in that particular subject and assesses the implied questions. After narrowing the focus and scope of the research subject, researcher frames a RQ and then evaluates it. Thus, conception to formulation of RQ is very systematic process and has to be performed meticulously as research guided by such question can have wider impact in the field of social and health research by leading to formulation of policies for the benefit of larger population.

I NTRODUCTION

A good research question (RQ) forms backbone of a good research, which in turn is vital in unraveling mysteries of nature and giving insight into a problem.[ 1 , 2 , 3 , 4 ] RQ identifies the problem to be studied and guides to the methodology. It leads to building up of an appropriate hypothesis (Hs). Hence, RQ aims to explore an existing uncertainty in an area of concern and points to a need for deliberate investigation. A good RQ helps support a focused arguable thesis and construction of a logical argument. Hence, formulation of a good RQ is undoubtedly one of the first critical steps in the research process, especially in the field of social and health research, where the systematic generation of knowledge that can be used to promote, restore, maintain, and/or protect health of individuals and populations.[ 1 , 3 , 4 ] Basically, the research can be classified as action, applied, basic, clinical, empirical, administrative, theoretical, or qualitative or quantitative research, depending on its purpose.[ 2 ]

Research plays an important role in developing clinical practices and instituting new health policies. Hence, there is a need for a logical scientific approach as research has an important goal of generating new claims.[ 1 ]

C HARACTERISTICS OF G OOD R ESEARCH Q UESTION

“The most successful research topics are narrowly focused and carefully defined but are important parts of a broad-ranging, complex problem.”

A good RQ is an asset as it:

• Details the problem statement
• Further describes and refines the issue under study
• Adds focus to the problem statement
• Guides data collection and analysis
• Sets context of research.

Hence, while writing RQ, it is important to see if it is relevant to the existing time frame and conditions. For example, the impact of “odd-even” vehicle formula in decreasing the level of air particulate pollution in various districts of Delhi.

A good research is represented by acronym FINERMAPS[ 5 ]

Interesting.

• Appropriate
• Potential value and publishability
• Systematic.

Feasibility means that it is within the ability of the investigator to carry out. It should be backed by an appropriate number of subjects and methodology as well as time and funds to reach the conclusions. One needs to be realistic about the scope and scale of the project. One has to have access to the people, gadgets, documents, statistics, etc. One should be able to relate the concepts of the RQ to the observations, phenomena, indicators, or variables that one can access. One should be clear that the collection of data and the proceedings of project can be completed within the limited time and resources available to the investigator. Sometimes, a RQ appears feasible, but when fieldwork or study gets started, it proves otherwise. In this situation, it is important to write up the problems honestly and to reflect on what has been learned. One should try to discuss with more experienced colleagues or the supervisor so as to develop a contingency plan to anticipate possible problems while working on a RQ and find possible solutions in such situations.

This is essential that one has a real grounded interest in one's RQ and one can explore this and back it up with academic and intellectual debate. This interest will motivate one to keep going with RQ.

The question should not simply copy questions investigated by other workers but should have scope to be investigated. It may aim at confirming or refuting the already established findings, establish new facts, or find new aspects of the established facts. It should show imagination of the researcher. Above all, the question has to be simple and clear. The complexity of a question can frequently hide unclear thoughts and lead to a confused research process. A very elaborate RQ, or a question which is not differentiated into different parts, may hide concepts that are contradictory or not relevant. This needs to be clear and thought-through. Having one key question with several subcomponents will guide your research.

This is the foremost requirement of any RQ and is mandatory to get clearance from appropriate authorities before stating research on the question. Further, the RQ should be such that it minimizes the risk of harm to the participants in the research, protect the privacy and maintain their confidentiality, and provide the participants right to withdraw from research. It should also guide in avoiding deceptive practices in research.

The question should of academic and intellectual interest to people in the field you have chosen to study. The question preferably should arise from issues raised in the current situation, literature, or in practice. It should establish a clear purpose for the research in relation to the chosen field. For example, filling a gap in knowledge, analyzing academic assumptions or professional practice, monitoring a development in practice, comparing different approaches, or testing theories within a specific population are some of the relevant RQs.

Manageable (M): It has the similar essence as of feasibility but mainly means that the following research can be managed by the researcher.

Appropriate (A): RQ should be appropriate logically and scientifically for the community and institution.

Potential value and publishability (P): The study can make significant health impact in clinical and community practices. Therefore, research should aim for significant economic impact to reduce unnecessary or excessive costs. Furthermore, the proposed study should exist within a clinical, consumer, or policy-making context that is amenable to evidence-based change. Above all, a good RQ must address a topic that has clear implications for resolving important dilemmas in health and health-care decisions made by one or more stakeholder groups.

Systematic (S): Research is structured with specified steps to be taken in a specified sequence in accordance with the well-defined set of rules though it does not rule out creative thinking.

Example of RQ: Would the topical skin application of oil as a skin barrier reduces hypothermia in preterm infants? This question fulfills the criteria of a good RQ, that is, feasible, interesting, novel, ethical, and relevant.

Types of research question

A RQ can address different formats depending on the aspect to be evaluated.[ 6 ] For example:

• Existence: This is designed to uphold the existence of a particular phenomenon or to rule out rival explanation, for example, can neonates perceive pain?
• Description and classification: This type of question encompasses statement of uniqueness, for example, what are characteristics and types of neuropathic bladders?
• Composition: It calls for breakdown of whole into components, for example, what are stages of reflux nephropathy?
• Relationship: Evaluate relation between variables, for example, association between tumor rupture and recurrence rates in Wilm's tumor
• Descriptive—comparative: Expected that researcher will ensure that all is same between groups except issue in question, for example, Are germ cell tumors occurring in gonads more aggressive than those occurring in extragonadal sites?
• Causality: Does deletion of p53 leads to worse outcome in patients with neuroblastoma?
• Causality—comparative: Such questions frequently aim to see effect of two rival treatments, for example, does adding surgical resection improves survival rate outcome in children with neuroblastoma than with chemotherapy alone?
• Causality–Comparative interactions: Does immunotherapy leads to better survival outcome in neuroblastoma Stage IV S than with chemotherapy in the setting of adverse genetic profile than without it? (Does X cause more changes in Y than those caused by Z under certain condition and not under other conditions).

How to develop a research question

• Begin by identifying a broader subject of interest that lends itself to investigate, for example, hormone levels among hypospadias
• Do preliminary research on the general topic to find out what research has already been done and what literature already exists.[ 7 ] Therefore, one should begin with “information gaps” (What do you already know about the problem? For example, studies with results on testosterone levels among hypospadias
• What do you still need to know? (e.g., levels of other reproductive hormones among hypospadias)
• What are the implied questions: The need to know about a problem will lead to few implied questions. Each general question should lead to more specific questions (e.g., how hormone levels differ among isolated hypospadias with respect to that in normal population)
• Narrow the scope and focus of research (e.g., assessment of reproductive hormone levels among isolated hypospadias and hypospadias those with associated anomalies)
• Is RQ clear? With so much research available on any given topic, RQs must be as clear as possible in order to be effective in helping the writer direct his or her research
• Is the RQ focused? RQs must be specific enough to be well covered in the space available
• Is the RQ complex? RQs should not be answerable with a simple “yes” or “no” or by easily found facts. They should, instead, require both research and analysis on the part of the writer
• Is the RQ one that is of interest to the researcher and potentially useful to others? Is it a new issue or problem that needs to be solved or is it attempting to shed light on previously researched topic
• Is the RQ researchable? Consider the available time frame and the required resources. Is the methodology to conduct the research feasible?
• Is the RQ measurable and will the process produce data that can be supported or contradicted?
• Is the RQ too broad or too narrow?
• Create Hs: After formulating RQ, think where research is likely to be progressing? What kind of argument is likely to be made/supported? What would it mean if the research disputed the planned argument? At this step, one can well be on the way to have a focus for the research and construction of a thesis. Hs consists of more specific predictions about the nature and direction of the relationship between two variables. It is a predictive statement about the outcome of the research, dictate the method, and design of the research[ 1 ]
• Understand implications of your research: This is important for application: whether one achieves to fill gap in knowledge and how the results of the research have practical implications, for example, to develop health policies or improve educational policies.[ 1 , 8 ]

Brainstorm/Concept map for formulating research question

• First, identify what types of studies have been done in the past?
• Is there a unique area that is yet to be investigated or is there a particular question that may be worth replicating?
• Begin to narrow the topic by asking open-ended “how” and “why” questions
• Evaluate the question
• Develop a Hypothesis (Hs)
• Write down the RQ.

Writing down the research question

• State the question in your own words
• Write down the RQ as completely as possible.

For example, Evaluation of reproductive hormonal profile in children presenting with isolated hypospadias)

• Divide your question into concepts. Narrow to two or three concepts (reproductive hormonal profile, isolated hypospadias, compare with normal/not isolated hypospadias–implied)
• Specify the population to be studied (children with isolated hypospadias)
• Refer to the exposure or intervention to be investigated, if any
• Reflect the outcome of interest (hormonal profile).

Another example of a research question

Would the topical skin application of oil as a skin barrier reduces hypothermia in preterm infants? Apart from fulfilling the criteria of a good RQ, that is, feasible, interesting, novel, ethical, and relevant, it also details about the intervention done (topical skin application of oil), rationale of intervention (as a skin barrier), population to be studied (preterm infants), and outcome (reduces hypothermia).

Other important points to be heeded to while framing research question

• Make reference to a population when a relationship is expected among a certain type of subjects
• RQs and Hs should be made as specific as possible
• Avoid words or terms that do not add to the meaning of RQs and Hs
• Stick to what will be studied, not implications
• Name the variables in the order in which they occur/will be measured
• Avoid the words significant/”prove”
• Avoid using two different terms to refer to the same variable.

Some of the other problems and their possible solutions have been discussed in Table 1 .

Potential problems and solutions while making research question

G OING B EYOND F ORMULATION OF R ESEARCH Q UESTION–THE P ATH A HEAD

Once RQ is formulated, a Hs can be developed. Hs means transformation of a RQ into an operational analog.[ 1 ] It means a statement as to what prediction one makes about the phenomenon to be examined.[ 4 ] More often, for case–control trial, null Hs is generated which is later accepted or refuted.

A strong Hs should have following characteristics:

• Give insight into a RQ
• Are testable and measurable by the proposed experiments
• Have logical basis
• Follows the most likely outcome, not the exceptional outcome.

E XAMPLES OF R ESEARCH Q UESTION AND H YPOTHESIS

Research question-1.

• Does reduced gap between the two segments of the esophagus in patients of esophageal atresia reduces the mortality and morbidity of such patients?

Hypothesis-1

• Reduced gap between the two segments of the esophagus in patients of esophageal atresia reduces the mortality and morbidity of such patients
• In pediatric patients with esophageal atresia, gap of <2 cm between two segments of the esophagus and proper mobilization of proximal pouch reduces the morbidity and mortality among such patients.

Research question-2

• Does application of mitomycin C improves the outcome in patient of corrosive esophageal strictures?

Hypothesis-2

In patients aged 2–9 years with corrosive esophageal strictures, 34 applications of mitomycin C in dosage of 0.4 mg/ml for 5 min over a period of 6 months improve the outcome in terms of symptomatic and radiological relief. Some other examples of good and bad RQs have been shown in Table 2 .

Examples of few bad (left-hand side column) and few good (right-hand side) research questions

R ESEARCH Q UESTION AND S TUDY D ESIGN

RQ determines study design, for example, the question aimed to find the incidence of a disease in population will lead to conducting a survey; to find risk factors for a disease will need case–control study or a cohort study. RQ may also culminate into clinical trial.[ 9 , 10 ] For example, effect of administration of folic acid tablet in the perinatal period in decreasing incidence of neural tube defect. Accordingly, Hs is framed.

Appropriate statistical calculations are instituted to generate sample size. The subject inclusion, exclusion criteria and time frame of research are carefully defined. The detailed subject information sheet and pro forma are carefully defined. Moreover, research is set off few examples of research methodology guided by RQ:

• Incidence of anorectal malformations among adolescent females (hospital-based survey)
• Risk factors for the development of spontaneous pneumoperitoneum in pediatric patients (case–control design and cohort study)
• Effect of technique of extramucosal ureteric reimplantation without the creation of submucosal tunnel for the preservation of upper tract in bladder exstrophy (clinical trial).

The results of the research are then be available for wider applications for health and social life

C ONCLUSION

A good RQ needs thorough literature search and deep insight into the specific area/problem to be investigated. A RQ has to be focused yet simple. Research guided by such question can have wider impact in the field of social and health research by leading to formulation of policies for the benefit of larger population.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

R EFERENCES

'I learned how to ask the right questions'

Kim Montpelier: Classics & Philosophy

A&S Communications

Kim Montpelier

Classics & Philosophy Washington, D.C.

Why did you choose Cornell?

Mostly for the ice cream and the fact that Kurt Vonnegut went here. But also because I knew that I wanted to study classics and I thought that the faculty in the classics department here were doing really exciting things (there aren't many places where you can take a class in spoken Latin!).

What is your main extracurricular activity and why is it important to you? 

I was involved in the Prison Reform and Education Project, because of which I became a tutor for people incarcerated in Westchester. I learned a lot through this experience, both about teaching and about the obstacles that people in prison face (especially obstacles to education) while they are incarcerated and after they get out of prison.

What Cornell memory do you treasure the most?         

I took a day hiking P.E. class in the spring semester one year. On the first day of class there was a big snowstorm, and I was hoping that the class would be cancelled. It wasn't. I was expecting a cold, wet, miserable day outside, but instead it was a lot of fun. It was cold but beautiful, and in a really different way than the way that other seasons are beautiful. That was my first winter in Ithaca and it taught me to love it for what it is.

What are the most valuable skills you gained from your Arts & Sciences education?      

I learned how to ask the right questions when you are confused about something. When I first started at Cornell, I found that I often had to do readings that were so hard to understand that I didn’t even know what I was confused about or what to ask to gain clarity. Throughout my time at Cornell, I was taught how to construct questions and what kinds of questions to ask to get closer to the truth.   

What have you accomplished as a Cornell student that you are most proud of?

I am pretty proud of my two theses. One is on friendship in Augustine’s Confessions and one is on materialism in the works of the 17th century philosopher Margaret Cavendish. These are the most substantial research projects that I’ve done so far, and through them I’ve learned the joy of feeling like you’ve made intellectual progress in your understanding of something after a lot of work.

How have your beliefs or perspectives changed since you first arrived at Cornell? 

Before I came here, I felt that a lot of the friends I had thought of the world in a very similar way to me. When I got here, I met people who were from a lot of different places and had a lot of different ways of thinking about the world. I realized that I shouldn’t take my approach to things for granted, even with my friends. It’s easy to assume you already know what someone’s going to say, but even people who we know really well can surprise us in really beautiful ways.  

Every year, our faculty nominate graduating Arts & Sciences students to be featured as part of our Extraordinary Journeys series.  Read more about the Class of 202 4.

More News from A&S

Committee to recommend final expressive activity policy

American Sign Language has found a growing home on the Hill

Extraordinary Journeys: The Class of 2024

BTPI will research relationship between Bitcoin and financial freedom

Featured Topics

Featured series.

A series of random questions answered by Harvard experts.

Explore the Gazette

Read the latest.

‘The scientist is not in the business of following instructions.’

Glimpse of next-generation internet

Epic science inside a cubic millimeter of brain

Venki Ramakrishnan.

Niles Singer/Harvard Staff Photographer

Science is making anti-aging progress. But do we want to live forever?

Nobel laureate details new book, which surveys research, touches on larger philosophical questions

Anne J. Manning

Harvard Staff Writer

Mayflies live for only a day. Galapagos tortoises can reach up to age 170. The Greenland shark holds the world record at over 400 years of life. 

Venki Ramakrishnan, Nobel laureate and author of the newly released “ Why We Die: The New Science of Aging and the Quest for Immortality ,” opened his packed Harvard Science Book Talk last week by noting the vast variabilities of lifespans across the natural world. Death is certain, so far as we know. But there’s no physical or chemical law that says it must happen at a fixed time, which raises other, more philosophical issues.

The “why” behind these enormous swings, and the quest to harness longevity for humans, have driven fevered attempts (and billions of dollars in research spending) to slow or stop aging. Ramakrishnan’s book is a dispassionate journey through current scientific understanding of aging and death, which basically comes down to an accumulation of chemical damage to molecules and cells.

“The question is whether we can tackle aging processes, while still keeping us who we are as humans,” said Ramakrishnan during his conversation with Antonio Regalado, a writer for the MIT Technology Review. “And whether we can do that in a safe and effective way.”

Even if immortality — or just living for a very, very long time — were theoretically possible through science, should we pursue it? Ramakrishnan likened the question to other moral ponderings.

“There’s no physical or chemical law that says we can’t colonize other galaxies, or outer space, or even Mars,” he said. “I would put it in that same category. And it would require huge breakthroughs, which we haven’t made yet.”

In fact, we’re a lot closer to big breakthroughs when it comes to chasing immortality. Ramakrishnan noted the field is moving so fast that a book like his can capture but a snippet. He then took the audience on a brief tour of some of the major directions of aging research. And much of it, he said, started in unexpected places.

Take rapamycin, a drug first isolated in the 1960s from a bacterium on Easter Island found to have antifungal, immunosuppressant, and anticancer properties. Rapamycin targets the TOR pathway, a large molecular signaling cascade within cells that regulates many functions fundamental to life. Rapamycin has garnered renewed attention for its potential to reverse the aging process by targeting cellular signaling associated with physiological changes and diseases in older adults.

Other directions include mimicking the anti-aging effects of caloric restriction shown in mice, as well as one particularly exciting area called cellular reprogramming. That means taking fully developed cells and essentially turning back the clock on their development.

The most famous foundational experiment in this area was by Kyoto University scientist and Nobel laureate Shinya Yamanaka, who showed that just four transcription factors could revert an adult cell all the way back to a pluripotent stem cell, creating what are now known as induced pluripotent stem cells.

Ramakrishnan , a scientist at England’s MRC Laboratory of Molecular Biology, won the 2009 Nobel Prize in chemistry for uncovering the structure of the ribosome. He said he felt qualified to write the book because he has “no skin in the game” of aging research. As a molecular biologist who has studied fundamental processes of how cells make proteins, he had connections in the field but wasn’t too close to any of it.

While researching the book, he took pains to avoid interviewing scientists with commercial ventures tied to aging.

The potential for conflicts of interest abound.

The world has seen an explosion in aging research in recent decades, with billions of dollars spent by government agencies and private companies . And the consumer market for products is forecast to hit $93 billion by 2027 .

As a result, false or exaggerated claims by companies promising longer life are currently on the rise, Ramakrishnan noted. He shared one example: Supplements designed to lengthen a person’s telomeres, or genetic segments that shrink with age, are available on Amazon.

“Of course, these are not FDA approved. There are no clinical trials, and it’s not clear what their basis is,” he said.

But still there appears to be some demand.

Get the best of the Gazette delivered to your inbox

By subscribing to this newsletter you’re agreeing to our privacy policy

Share this article

You might like.

George Whitesides became a giant of chemistry by keeping it simple

Physicists demo first metro-area quantum computer network in Boston

Researchers publish largest-ever dataset of neural connections

Pop star on one continent, college student on another

Model and musician Kazuma Mitchell managed to (mostly) avoid the spotlight while at Harvard

Finding right mix on campus speech policies

Legal, political scholars discuss balancing personal safety, constitutional rights, academic freedom amid roiling protests, cultural shifts

More From Forbes

Questions to ask when considering buying in an hoa.

• Share to Facebook
• Share to Twitter
• Share to Linkedin

Interior of a residence at Portico, a 112-unit luxury condominium community in North Scottsdale, ... [+] Ariz.

ecoming part of a homeowner’s association is a growing reality of the home-buying process in the U.S. Approximately 75.5 million Americans live in a community controlled by a homeowner’s association, states the Foundation for Community Association Research. More than 3,000 additional HOAs will form this year, the foundation adds. In 2022, 84% of new-construction, single-family home buyers purchased in an HOA-governed community, an almost 10% surge since 2019, according to the U.S. Census.

The Cato Institute reports homes in HOA communities are valued at 5 to 6% more than comparable homes not governed by HOAs. But that doesn’t mean everyone who lives in HOAs is delighted.

In fact, more than half (57%) of those who reside in communities governed by HOAs report not being happy, the Cato Institute has found. Ten percent have considered selling their homes to be free of HOA strictures, a Rocket Mortgage study revealed.

Folks purchasing a home in master-planned communities, condominium structures or maintenance-free residences in general will soon be forking out HOA fees in addition to their monthly mortgage payments.

Google Chrome Gets Third Emergency Update In A Week As Attacks Continue

Biden vs. trump 2024 election polls: biden losing support among key voting blocs, japanese fans are puzzled that yasuke is in assassin s creed shadows, special assessments.

One of the key factors in purchasers’ dissatisfaction with HOAs are the fees charged, which can rise with inflation. However, fees are just one of the sore points that may emerge when owners come up against HOAs.

Home purchasers should not buy before they ask a number of questions about the particular HOA into which they’re buying.

Among them: How has the HOA managed its reserves and responded to maintenance requirements? Who serves on the board, and what is the board’s tenure or turnover rate? If this is a new construction community, how will the developer ensure a smooth transition to an HOA board and management company? If an older condo community, is there a history of special assessments levied for certain needs?

If the HOA governs a condominium community, what rules does the HOA impose regarding rentals, items on balconies and alterations to common-area property? If a single-family home community, what rules are imposed regarding number of vehicles in driveways or on streets, when garbage can be put out and empties taken in, extent and frequency of exterior home maintenance and more?

And how can homeowners new to the HOA help ensure they’re up to date on rules to ensure their moves are seamless, trouble-free experiences?

According to Shruti Kumar, CPM and vice president of condominium management at Habitat, managing more than 4,400 Chicago-area units, a major issue in many HOAs is the failure of new owners to review the rules prior to move-in day. “This leads to fines and, needless to say, a negative experience,” Kumar reports.

In addition, a number of HOAs have instituted regulations that forbid or limit rentals, either with rental caps or residence requirements. “I have seen that many home buyers tend to overlook this aspect during the purchasing process,’ Kumar says.

Home buyers purchasing in new construction properties should explore not only the size of the monthly assessments, but also the ways in which the developer plans to hand over the management of the building to a newly created homeowners association. So says Liz Brooks, executive vice president of sales and marketing at condominium developer Belgravia Group. The firm has helped transition its HOAs to homeowners at dozens of properties over the past three-quarters of a century.

“One way we give a future HOA a head start is to establish and maintain a sense of community even before construction begins on our buildings,” she says. She pointed to the company’s 112-unit North Scottsdale, Ariz. luxury condominium project, Portico. There, Belgravia Group staged events for buyers that were intended to allow them to connect to future neighbors, some of whom were destined to become future HOA board members. “From these events, we often begin seeing interest in participating in the HOA emerge [among] some of the buyers,” Brooks concludes.

• Editorial Standards
• Reprints & Permissions

Join The Conversation

One Community. Many Voices. Create a free account to share your thoughts. 

Forbes Community Guidelines

Our community is about connecting people through open and thoughtful conversations. We want our readers to share their views and exchange ideas and facts in a safe space.

In order to do so, please follow the posting rules in our site's  Terms of Service.   We've summarized some of those key rules below. Simply put, keep it civil.

Your post will be rejected if we notice that it seems to contain:

• False or intentionally out-of-context or misleading information
• Insults, profanity, incoherent, obscene or inflammatory language or threats of any kind
• Attacks on the identity of other commenters or the article's author
• Content that otherwise violates our site's  terms.

User accounts will be blocked if we notice or believe that users are engaged in:

• Continuous attempts to re-post comments that have been previously moderated/rejected
• Racist, sexist, homophobic or other discriminatory comments
• Attempts or tactics that put the site security at risk
• Actions that otherwise violate our site's  terms.

So, how can you be a power user?

• Stay on topic and share your insights
• Feel free to be clear and thoughtful to get your point across
• ‘Like’ or ‘Dislike’ to show your point of view.
• Protect your community.
• Use the report tool to alert us when someone breaks the rules.

Thanks for reading our community guidelines. Please read the full list of posting rules found in our site's  Terms of Service.

• Skip to main content
• Keyboard shortcuts for audio player

• Your Health
• Treatments & Tests
• Health Inc.
• Public Health

As the FDA evaluates ecstasy treatment for PTSD, questions mount about the evidence

Research on MDMA has shown it can be effective for PTSD, but approval of the treatment isn't yet guaranteed. The Washington Post via Getty Images hide caption

Research on MDMA has shown it can be effective for PTSD, but approval of the treatment isn't yet guaranteed.

In a matter of months, the Food and Drug Administration is expected to decide whether the drug commonly known as ecstasy can be used as a treatment for post-traumatic stress disorder.

An approval by the agency would represent an enormous milestone for the movement to bring psychedelics into the mainstream of mental health care. An FDA rejection of MDMA, the abbreviation of the drug's chemical name, would deal a major setback to the effort.

Clinical trials have inspired optimism in the drug for its potential to help the millions of Americans who experience PTSD. Accounts from some of those who've participated in the trials describe the treatment as transformational.

But new and troubling questions about this research are now threatening to upset the final stretch in the drug's path to market.

The allegations surfaced in a draft report released in March by the Institute for Clinical and Economic Review , a nonprofit that evaluates clinical trials and drug prices, which found "substantial concerns about the validity of the results" of the MDMA clinical trials.

The ICER report was followed in April by a citizen petition to the FDA. In that document, a group of concerned people allege possible misconduct and ethical violations that could compromise the MDMA research. The petition asked the agency to hold a public meeting to address the concerns.

If true, the claims could jeopardize the drug's chances of receiving FDA approval, a decision that is expected to come by early August .

"There's the possibility that the data might not be representative of what's actually happened in clinical trials," says Neşe Devenot , one of the authors of the citizen petition and a senior lecturer in the writing program at Johns Hopkins University who is involved in psychedelic research. "I don't think this has been publicly reckoned with."

That may soon happen. The FDA announced Thursday that it plans to hold a public advisory committee meeting on MDMA-assisted therapy on June 4.

At the heart of the controversy are the organizations that have pioneered research into MDMA: the Multidisciplinary Association for Psychedelic Studies , and the public benefit corporation incubated by MAPS, which was recently rebranded as Lykos Therapeutics .

Lykos sponsored the clinical trials of MDMA. The results are included in the company's application to the FDA seeking approval to market the drug for therapy-assisted PTSD treatment.

Researchers and clinicians involved in the trials have pushed back strongly against the accusations that their clinical data isn't sound.

Jennifer Mitchell , lead author of the published papers from the Phase 3 trials, says she stands behind their findings.

"I didn't feel any pressure from the sponsor to come up with anything different than what the data was providing," says Mitchell, a professor of neurology and psychiatry at the University of California, San Francisco and associate chief of staff for research at the San Francisco VA Medical Center. "I wouldn't have continued to work with them if I had felt that."

Promising MDMA research for PTSD

The Phase 3 trials evaluated MDMA-assisted therapy, a protocol in which the drug is given under the supervision of two therapists.

In the second stage of the trials , 94 people with moderate and severe PTSD received either the drug or a placebo during three sessions, each spaced a month apart. There were also follow up "integration" sessions to help people process their experiences while on MDMA.

By the end of the trial, about 71% of participants in the MDMA arm no longer met the diagnostic criteria for PTSD, compared to about 48% who underwent the same therapy but took a placebo instead. Those findings built on promising results from earlier studies .

The study documented various adverse events in both groups — ranging from nausea and anxiety to heart palpitations — but none of them qualified as serious. The treatment was "generally well tolerated."

"Consistent with PTSD, suicidal ideation was observed in both groups," the authors reported in the journal Nature Medicine , "MDMA did not appear to increase this risk, and no suicidal behavior was observed."

Casey Tylek, a participant in the Phase 3 trials, says he had no experience with the drug prior to the study.

Tylek was in the placebo group, but was later given the opportunity to undergo the therapy with MDMA.

"It was incredibly powerful," says Tylek, a veteran who lives in Massachusetts, "I truly don't know if I would be alive today if I hadn't gone through that trial."

ICER report raises concerns

In its report , ICER acknowledges that the MDMA data suggest it would be an "important addition to treatment options for PTSD," but it questions whether the published findings tell the full story.

Among the concerns, the ICER report details a well-known challenge in psychedelic research around how to make sure study participants don't know if they got the experimental treatment or a placebo. Most of those in the MDMA group were able to identify they had received the drug. It also suggests the method used to assess PTSD — considered the gold-standard — showed improvements in symptoms after the treatment, even though some people were worse overall.

Beyond that, however, the report brings up the possibility that "very strong prior beliefs" among therapists, investigators and patients influenced the results.

"Concerns have been raised by some that therapists encouraged favorable reports by patients and discouraged negative reports by patients including discouraging reports of substantial harms, potentially biasing the recording of benefits and harms," the report states.

ICER does not identify the sources who were interviewed, although it did include two trial participants, a "trial therapist" and those who worked on a podcast called Cover Story , says Dr. David Rind , the chief medical officer for ICER.

"This was a very unusual review," says Rind.

The podcast, produced by New York Magazine and the nonprofit media organization Psymposia , brought to light claims by a participant named Meaghan Buisson, who appeared in a video of two therapists , a married couple, engaged in what Buisson described as inappropriate physical contact while she was under the influence of MDMA at a Phase 2 trial site in Canada.

Enlighten Me with Rachel Martin

This psychedelics researcher approached his death with calm and curiosity.

MAPS determined the therapists "substantially deviated" from the treatment manual. The organization also barred the two therapists from becoming providers of MDMA-assisted therapy in affiliation with MAPS, and health authorities were notified in Canada and the U.S.

The podcast also interviewed two people (their full names were not revealed) who said they received MDMA in the large-scale, or Phase 3, trials and experienced feelings of suicidality and other distress after the studies.

The ICER report is yet to be finalized, but Rind says their analysis showed "there's still a lot of uncertainty" about the treatment.

"You have a group of people who are very upset about how these trials went," he says. "We couldn't tell, even though we talked with people where this happened, whether that represents a tiny fraction of bad events or a number of bad events large enough to have rendered the trial just not believable."

Pushback against the allegations

According to Rind, MAPS and Lykos had "very little" engagement with ICER on the draft report.

But since then, a group of more than 70 clinicians, investigators and others involved in the Phase 3 MDMA trials have published a detailed response , saying that certain aspects of the trials were "misrepresented" and that a number of assertions amount to "hearsay."

Willa Hall , a clinical psychologist in the Phase 3 trials, says she and her colleagues were shocked by how ICER described their work.

"I didn't recognize the study that they were talking about," Hall told NPR. "I think a lot of us felt quite insulted actually by that characterization. I saw nothing like that. I only saw professionalism ."

In their response, Hall and her colleagues write that "[ICER] does not note the many measures taken to train, support, and evaluate therapists on those trials—measures that met, and in some cases exceeded, the accepted standards in the field of psychotherapy research."

They also take issue with ICER relying on "a small number of undisclosed study participants and unnamed 'experts' rather than validated research outcomes." The critiques that participants knew they received the treatment or that the measure of PTSD symptoms might not capture someone's overall condition would also apply to other clinical trials, unrelated to MDMA, they say.

UCSF's Jennifer Mitchell says the clinical trial was designed to safeguard against bias.

Therapists on site were not collecting key data from participants about their PTSD symptoms following the sessions. Instead, that was being done online by "independent assessors" who didn't know who had received the treatment or a placebo.

Hall says therapists "meticulously" captured any adverse events. "We encouraged our participants to be very honest," she says. "We're all invested in knowing how it works and what are the risks for people."

Still, Mitchell acknowledges she doesn't have full insight into what was going on at each trial site on a given day.

"This is my own frustration," she says. "I can't attest to what was happening at one of the sites in Israel on a day to day basis, or on one of the sites in Canada."

But she contends that ICER tried to conduct an investigation without access to the full data.

Luminous: A Series About Psychedelics from 'To The Best Of Our Knowledge'

The FDA granted MDMA "breakthrough therapy" status, she says, which means the agency was involved in the study design and "many aspects" of the trial.

"So there's no keeping things from the FDA," Mitchell says.

The ICER report points out that therapists and participants in the study were "pulled heavily from the existing community of those interested and involved in the use of psychedelics for possible psychological benefits."

For her part though, Mitchell says she's not what some would call a "true believer."

"My personal feeling is that psychedelics are complicated compounds and they do not work for everyone," she says.

Petition adds to controversy ahead of the FDA meeting

On the heels of the ICER report, Neşe Devenot and four others, including Meaghan Buisson, submitted the citizen petition to the FDA calling for a public advisory meeting about the Lykos' application for MDMA.

In it, they ask for extended time for stakeholders who are concerned about the "risks and shortcomings" of the research.

"Evidence from multiple sources indicates that the sponsor has engaged in a pattern of systematic and deliberate omission of adverse events from the public record while minimizing documented harms," the petition states.

It continues: "We cannot rule out the possibility that MAPS/Lykos manipulated clinical trial data to hide adverse events from regulatory agencies."

The petition cites media reports and public statements from figures at Lykos and MAPS — and disclosures from a former employee of the MAPS public benefit corporation "who prefers to maintain anonymity at this juncture."

In addition, the petition alleges that clinical trial investigators would phone MAPS in the event of an incident during the trial to see whether that should be reported as an adverse event and that a suicide attempt during a dosing session was discouraged from being reported.

In an email to NPR, a spokesperson for MAPS rejects the claim.

It's not clear if the FDA's decision to hold a meeting was influenced by Devenot's petition, which has over 80 signatures .

Alaina Jaster , who has a doctorate in pharmacology and toxicology, is another author of the petition.

"We need to listen to people [in the trials] who are having these experiences, instead of telling them that they are liars and that they're going to ruin the psychedelic renaissance," says Jaster who hosts a podcast on psychedelics.

"None of us are against this as a useful tool, or none of us are against treating mental health. We don't have any monetary interests in this not going through," she says.

Neşe Devenot and Brian Pace, another author of the petition, are affiliated with Psymposia , the media organization that produced the podcast, but Devenot says they were not involved in the podcast and are unpaid board members.

In response to the petition's call for a public meeting, a spokesperson for Lykos sent NPR a statement in April saying the company supports holding an advisory meeting. "The voice of the PTSD patient is incredibly important," the email reads.

Shots - Health News

Psychedelic drugs may launch a new era in psychiatric treatment, brain scientists say.

MAPS "remains fully supportive of comprehensive, high-quality research; careful analysis of safety and efficacy; and stringent regulatory oversight of any psychedelic-assisted therapy research or delivery," according to the statement it also sent to NPR in April. "We stand behind Lykos' execution of the clinical program and support the clinical results."

One of those who signed the petition after seeing it posted online is Dr. Boris Heifets , an anesthesiologist at Stanford University whose lab studies psychedelics, including MDMA.

"I don't know if the allegations are true, it just makes me deeply sad if there was actually malfeasance for such an important trial," says Heifets. "The MDMA Phase 3 trials were very important for mental health, important for a lot of people who may benefit from this therapy."

Learning about MAPS several decades ago was, in part, what inspired Heifets to get involved in this type of research.

He says he donates $100 a year to MAPS and that they have supplied his lab with MDMA. He also consults for one company that's developing a derivative of MDMA.

Heifets says the petition contains some "very strong allegations," particularly the claim that certain adverse events were not disclosed.

"I want to hear MAPS respond," he says. " I would really like to understand the risk profile of this drug before it's approved, not after."

• psychedelic drugs
• psychedelics
• clinical trials
• mdma therapy

I worked at Amazon, Tesla, SAP, Salesforce, and Meta. Here's what I do 24 hours before a big interview.

• Hemant Pandey, a Meta senior software engineer, suggests exploring the job market every two years.
• Pandey's pre-interview prep includes reading up on past interviews and preparing good questions.
• His strategy not only helps him stand out at interviews but also provides insights into company culture.

This as-told-to essay is based on a conversation with Hemant Pandey , a senior software engineer at Meta in California. It has been edited for length and clarity. Business Insider has verified his employment history.

I graduated with a master's in computer science in 2017. I was a few months into my first job at Tesla when I was laid off due to budget cuts.

Things worked out well, and I landed on my feet within a few weeks, but the experience left a mark on me.

I realized that the relationship between an employer and employee is transactional, and there will always be ups and downs.

Since then, I have made it a practice to interview and explore the job market every two years even if I don't plan to switch jobs, just to get an idea of my demand in the market and what employers are paying.

Over the last seven years, I have interned at Amazon and worked full time at Tesla, SAP, Salesforce, and Meta.

At various points in my career, I have also received offers from LinkedIn, TikTok, Square, and Splunk. Over time, I have solidified an interview preparation strategy that has worked for me and one I share with juniors I mentor.

This is what 24 hours before a Big Tech interview look like for me:

Read up on past interview questions

I have found that Big Tech interviews largely follow a set pattern of processes. It is common for the recruiter to explain all the steps in the first call — including the types and number of interviews. This information is also easily accessible on online forums.

I have a list of technical topics that I revise a week before the interview so that I am relaxed on the last day. On the day before, I go online and look up the experiences of people who have recently interviewed at the company.

Related stories

For example, if I am applying for Google, I'll go to a coding practice website called Leetcode and click on the "discuss" tab. Here, people share their company-specific interview experiences. I look at what popular questions are and if the user has any advice on how to pass them.

I mostly use it as a checklist to ensure I am comfortable answering those questions. If I spot anything new, I look into it.

I keep the the last day light, because I don't want to be stressed during the interview or even the day before. Interviews require you to be good at communication and time management and pressur iz ing yourself on the last day might cause more harm than good.

Prepare questions to ask

On the day before a big interview, I focus on planning what I want to ask the hiring managers at the end of our conversation.

This step is important for not only doing well in the interview but also for analyzing the company.

I usually ask hiring managers these three questions:

• What advice do they have for someone who wants to succeed in the company?
• What has their growth been in the company — what level did they join, and where are they at now?
• I do my research on the company's upcoming projects or the systems they use and bring it up in the form of a question.

Answers to these questions give me more information about the company and play a role in my decision if I am comparing multiple offers.

For the second question, if someone says they joined the company as a fresh graduate and moved to a staff engineer role in three years, it tells me the company rewards top performers. I try to ask multiple people this question and look for a pattern. A few people saying they are at the same level they joined a couple of years ago makes the company culture less appealing to me.

Besides helping me, these questions tell the hiring team that I am someone who has done their homework about the company and is interested in the work they do.

As someone who is now on the other side of the interview panel, I love when candidates ask me about my career growth in the company, or specific questions like why Meta is focusing on AI. This shows me that they keep up-to-date with tech and are passionate about my company.

Do you work in tech, finance, or consulting and have tips to share about your interview strategy? Email this reporter at [email protected] .

Watch: Marketing leaders from Amazon, LinkedIn, Lego Group and more tell Insider what pandemic-fueled business changes are likely to stick around

• Main content