questions to ask about a research article

Evaluating Information

• Understanding Primary and Secondary Sources
• Exploring and Evaluating Popular, Trade, and Scholarly Sources

Reading a Scholarly Article

Common components of original research articles, while you read, reading strategies, reading for citations, further reading, learning objectives.

This page was created to help you:

Identify the different parts of a scholarly article

Efficiently analyze and evaluate scholarly articles for usefulness

This page will focus on reading scholarly articles — published reports on original research in the social sciences, humanities, and STEM fields. Reading and understanding this type of article can be challenging. This guide will help you develop these skills, which can be learned and improved upon with practice.

We will go over:

There are many different types of articles that may be found in scholarly journals and other academic publications. For more, see:

• Types of Information Sources

Reading a scholarly article isn’t like reading a novel, website, or newspaper article. It’s likely you won’t read and absorb it from beginning to end, all at once.

Instead, think of scholarly reading as inquiry, i.e., asking a series of questions as you do your research or read for class. Your reading should be guided by your class topic or your own research question or thesis.

For example, as you read, you might ask yourself:

• What questions does it help to answer, or what topics does it address?
• Are these relevant or useful to me?
• Does the article offer a helpful framework for understanding my topic or question (theoretical framework)?
• Do the authors use interesting or innovative methods to conduct their research that might be relevant to me?
• Does the article contain references I might consult for further information?

In Practice

Scanning and skimming are essential when reading scholarly articles, especially at the beginning stages of your research or when you have a lot of material in front of you.

Many scholarly articles are organized to help you scan and skim efficiently. The next time you need to read an article, practice scanning the following sections (where available) and skim their contents:

• The abstract: This summary provides a birds’ eye view of the article contents.
• The introduction:  What is the topic(s) of the research article? What is its main idea or question?
• The list of keywords or descriptors
• Methods: How did the author(s) go about answering their question/collecting their data?
• Section headings:  Stop and skim those sections you may find relevant.
• Figures:  Offer lots of information in quick visual format.
• The conclusion:  What are the findings and/or conclusions of this article?

Mark Up Your Text

Read with purpose.

• Scanning and skimming with a pen in hand can help to focus your reading.
• Use color for quick reference. Try highlighters or some sticky notes. Use different colors to represent different topics.
• Write in the margins, putting down thoughts and questions about the content as you read.
• Use digital markup features available in eBook platforms or third-party solutions, like Adobe Reader or Hypothes.is.

Categorize Information

Create your own informal system of organization. It doesn’t have to be complicated — start basic, and be sure it works for you.

• Jot down a few of your own keywords for each article. These keywords may correspond with important topics being addressed in class or in your research paper.  
• Write keywords on print copies or use the built-in note taking features in reference management tools like Zotero and EndNote.  
• Your keywords and system of organization may grow more complex the deeper you get into your reading.

Highlight words, terms, phrases, acronyms, etc. that are unfamiliar to you. You can highlight on the text or make a list in a notetaking program.

• Decide if the term is essential to your understanding of the article or if you can look it up later and keep scanning.

You may scan an article and discover that it isn’t what you thought it was about. Before you close the tab or delete that PDF, consider scanning the article one more time, specifically to look for citations that might be more on-target for your topic.  

You don’t need to look at every citation in the bibliography — you can look to the literature review to identify the core references that relate to your topic. Literature reviews are typically organized by subtopic within a research question or thesis. Find the paragraph or two that are closely aligned with your topic, make note of the author names, then locate those citations in the bibliography or footnote.

See the Find Articles page for what to do next:

• Find Articles

See the Citation Searching page for more on following a citation trail:

• Citation Searching
• Taking notes effectively. [blog post] Raul Pacheco-Vega, PhD
• How to read an academic paper. [video] UBCiSchool. 2013
• How to (seriously) read a scientific paper. (2016, March 21). Science | AAAS.
• How to read a paper. S. Keshav. 2007. SIGCOMM Comput. Commun. Rev. 37, 3 (July 2007), 83–84.

This guide was designed to help you:

• << Previous: Exploring and Evaluating Popular, Trade, and Scholarly Sources
• Last Updated: Feb 16, 2024 3:55 PM
• URL: https://libguides.brown.edu/evaluate

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

IOE Writing Centre

Critical Reading Questions

On one level, reading critically simply means asking questions and evaluating the claims, and not simply accepting what you read.

However, the types of questions you ask, and the types of issues you prioritise in your evaluation, can vary considerably. 

You can do it in a relatively 'logical' way, thinking about the reasoning used, the claims made based on the evidence, etc. You can also do it in a more 'political' way, where the social implications are taken into account. 

You can also ask your tutor for examples, to find out how they understand the concept of criticality. This may help to understand what they are expecting in your writing.  

We might ask some of the questions below when reading a text.

Look at the questions carefully, and check that you understand what they are asking. You do not need to use all of these questions every time you read. Choose two or three which make the most sense to you, and start there.

Questions about the overall text

(a) What is the purpose/aim of this text? How do you know? How might this influence the way it is written?

(b) Can you see any justification (direct or implied) for the research decisions? Do the justifications seem reasonable?

Questions about the truth claims made within the text

(c) Are any assumptions being made in this text? Assumptions might include: _______________________ is important. _______________________ is possible. _______________________ might influence _______________________. _______________________ is a positive thing. _______________________ is a negative thing.

(d) Do these assumptions seem reasonable in this context? Why or why not?

(e) Are any generalisations being made? Are these generalisations reasonable here?

(f) Do any claims seem too certain?

(g) Are there suitable examples?

(h) Are there claims which are based on authority for support? What kind of authority is it? Does this seem reasonable?

(i) Are there claims which are based on evidence for support? What kind of evidence is it? Does this seem reasonable?

(j) Are any concepts being conflated?

Questions about how the text could be different

(k) What is missing from the text?

(l) How could the text be not like this / different?

(m) Is anything being used out of context in the text?

Political Questions

(n) Is there anything problematic in the text?

(o) Are any groups being excluded or marginalized in the text or in the implications of the claims?

(p) Is there any exclusionary language used in the text?

(q) What would the implications be, if we were to take the claims seriously? i.e. What would happen next?

Personal Engagement

(r) How does this text relate to my personal experience?

(s) How does my personal knowledge and experience affect  the way I read the text?

(t) Can my personal experience help me to evaluate the  claims?

(u) What status does my personal experience have, in relation to the published research?

(v) Can I find anything in the literature to help me  relate this to my personal experience?

Further Critical Questions

What else? (Can you think of further critical questions? Do you have a favourite question?)

^Back to top

IOE Writing Centre Online

Self-access resources from the Academic Writing Centre at the UCL Institute of Education.

Anonymous Suggestions Box

Information for Staff

Academic Writing Centre

Academic Writing Centre, UCL Institute of Education [email protected] Twitter:   @AWC_IOE Skype:   awc.ioe

Greater Good Science Center • Magazine • In Action • In Education

Big Ideas Articles & More

10 questions to ask about scientific studies, never take a study at face value, including one you read about in greater good .

Here at Greater Good , we cover research into social and emotional well-being, and we try to help people apply findings to their personal and professional lives. We are well aware that our business is a tricky one.

Summarizing scientific studies and applying them to people’s lives isn’t just difficult for the obvious reasons, like understanding and then explaining scientific jargon or methods to non-specialists. It’s also the case that context gets lost when we translate findings into stories, tips, and tools for a more meaningful life, especially when we push it all through the nuance-squashing machine of the Internet. Many people never read past the headlines, which intrinsically aim to overgeneralize and provoke interest. Because our articles can never be as comprehensive as the original studies, they almost always omit some crucial caveats, such as limitations acknowledged by the researchers. To get those, you need access to the studies themselves.

And it’s very common for findings to seem to contradict each other. For example, we recently covered an experiment that suggests stress reduces empathy—after having previously discussed other research suggesting that stress-prone people can be more empathic. Some readers asked: Which one is correct? (You’ll find my answer here .)

But probably the most important missing piece is the future. That may sound like a funny thing to say, but, in fact, a new study is not worth the PDF it’s printed on until its findings are replicated and validated by other studies—studies that haven’t yet happened. An experiment is merely interesting until time and testing turns its finding into a fact.

Scientists know this, and they are trained to react very skeptically to every new paper. They also expect to be greeted with skepticism when they present findings. Trust is good, but science isn’t about trust. It’s about verification.

However, journalists like me, and members of the general public, are often prone to treat every new study as though it represents the last word on the question addressed. This particular issue was highlighted last week by—wait for it—a new study that tried to reproduce 100 prior psychological studies to see if their findings held up. The result of the three-year initiative is chilling: The team, led by University of Virginia psychologist Brian Nosek, got the same results in only 36 percent of the experiments they replicated. This has led to some predictably provocative, overgeneralizing headlines implying that we shouldn’t take psychology seriously.

I don’t agree.

Despite all the mistakes and overblown claims and criticism and contradictions and arguments—or perhaps because of them—our knowledge of human brains and minds has expanded dramatically during the past century. Psychology and neuroscience have documented phenomena like cognitive dissonance, identified many of the brain structures that support our emotions, and proved the placebo effect and other dimensions of the mind-body connection, among other findings that have been tested over and over again.

These discoveries have helped us understand and treat the true causes of many illnesses. I’ve heard it argued that rising rates of diagnoses of mental illness constitute evidence that psychology is failing, but in fact, the opposite is true: We’re seeing more and better diagnoses of problems that would have compelled previous generations to dismiss people as “stupid” or “crazy” or “hyper” or “blue.” The important thing to bear in mind is that it took a very, very long time for science to come to these insights and treatments, following much trial and error.

Science isn’t a faith, but rather a method that takes time to unfold. That’s why it’s equally wrong to uncritically embrace everything you read, including what you are reading on this page.

Given the complexities and ambiguities of the scientific endeavor, is it possible for a non-scientist to strike a balance between wholesale dismissal and uncritical belief? Are there red flags to look for when you read about a study on a site like Greater Good or in a popular self-help book? If you do read one of the actual studies, how should you, as a non-scientist, gauge its credibility?

I drew on my own experience as a science journalist, and surveyed my colleagues here at the UC Berkeley Greater Good Science Center. We came up 10 questions you might ask when you read about the latest scientific findings. These are also questions we ask ourselves, before we cover a study.

1. Did the study appear in a peer-reviewed journal?

Peer review—submitting papers to other experts for independent review before acceptance—remains one of the best ways we have for ascertaining the basic seriousness of the study, and many scientists describe peer review as a truly humbling crucible. If a study didn’t go through this process, for whatever reason, it should be taken with a much bigger grain of salt.

2. Who was studied, where?

Animal experiments tell scientists a lot, but their applicability to our daily human lives will be limited. Similarly, if researchers only studied men, the conclusions might not be relevant to women, and vice versa.

This was actually a huge problem with Nosek’s effort to replicate other people’s experiments. In trying to replicate one German study, for example, they had to use different maps (ones that would be familiar to University of Virginia students) and change a scale measuring aggression to reflect American norms. This kind of variance could explain the different results. It may also suggest the limits of generalizing the results from one study to other populations not included within that study.

As a matter of approach, readers must remember that many psychological studies rely on WEIRD (Western, educated, industrialized, rich and democratic) samples, mainly college students, which creates an in-built bias in the discipline’s conclusions. Does that mean you should dismiss Western psychology? Of course not. It’s just the equivalent of a “Caution” or “Yield” sign on the road to understanding.

3. How big was the sample?

In general, the more participants in a study, the more valid its results. That said, a large sample is sometimes impossible or even undesirable for certain kinds of studies. This is especially true in expensive neuroscience experiments involving functional magnetic resonance imaging, or fMRI, scans.

And many mindfulness studies have scanned the brains of people with many thousands of hours of meditation experience—a relatively small group. Even in those cases, however, a study that looks at 30 experienced meditators is probably more solid than a similar one that scanned the brains of only 15.

4. Did the researchers control for key differences?

Diversity or gender balance aren’t necessarily virtues in a research study; it’s actually a good thing when a study population is as homogenous as possible, because it allows the researchers to limit the number of differences that might affect the result. A good researcher tries to compare apples to apples, and control for as many differences as possible in her analysis.

5. Was there a control group?

One of the first things to look for in methodology is whether the sample is randomized and involved a control group; this is especially important if a study is to suggest that a certain variable might actually cause a specific outcome, rather than just be correlated with it (see next point).

For example, were some in the sample randomly assigned a specific meditation practice while others weren’t? If the sample is large enough, randomized trials can produce solid conclusions. But, sometimes, a study will not have a control group because it’s ethically impossible. (Would people still divert a trolley to kill one person in order to save five lives, if their decision killed a real person, instead of just being a thought experiment? We’ll never know for sure!)

The conclusions may still provide some insight, but they need to be kept in perspective.

6. Did the researchers establish causality, correlation, dependence, or some other kind of relationship?

I often hear “Correlation is not causation” shouted as a kind of battle cry, to try to discredit a study. But correlation—the degree to which two or more measurements seem to change at the same time—is important, and is one step in eventually finding causation—that is, establishing a change in one variable directly triggers a change in another.

The important thing is to correctly identify the relationship.

7. Is the journalist, or even the scientist, overstating the result?

Language that suggests a fact is “proven” by one study or which promotes one solution for all people is most likely overstating the case. Sweeping generalizations of any kind often indicate a lack of humility that should be a red flag to readers. A study may very well “suggest” a certain conclusion but it rarely, if ever, “proves” it.

This is why we use a lot of cautious, hedging language in Greater Good , like “might” or “implies.”

8. Is there any conflict of interest suggested by the funding or the researchers’ affiliations?

A recent study found that you could drink lots of sugary beverages without fear of getting fat, as long as you exercised. The funder? Coca Cola, which eagerly promoted the results. This doesn’t mean the results are wrong. But it does suggest you should seek a second opinion .

9. Does the researcher seem to have an agenda?

Readers could understandably be skeptical of mindfulness meditation studies promoted by practicing Buddhists or experiments on the value of prayer conducted by Christians. Again, it doesn’t automatically mean that the conclusions are wrong. It does, however, raise the bar for peer review and replication. For example, it took hundreds of experiments before we could begin saying with confidence that mindfulness can indeed reduce stress.

10. Do the researchers acknowledge limitations and entertain alternative explanations?

Is the study focused on only one side of the story or one interpretation of the data? Has it failed to consider or refute alternative explanations? Do they demonstrate awareness of which questions are answered and which aren’t by their methods?

I summarize my personal stance as a non-scientist toward scientific findings as this: Curious, but skeptical. I take it all seriously and I take it all with a grain of salt. I judge it against my experience, knowing that my experience creates bias. I try to cultivate humility, doubt, and patience. I don’t always succeed; when I fail, I try to admit fault and forgive myself. My own understanding is imperfect, and I remind myself that one study is only one step in understanding. Above all, I try to bear in mind that science is a process, and that conclusions always raise more questions for us to answer.

About the Author

Jeremy Adam Smith

Uc berkeley.

Jeremy Adam Smith edits the GGSC's online magazine, Greater Good . He is also the author or coeditor of five books, including The Daddy Shift , Are We Born Racist? , and (most recently) The Gratitude Project: How the Science of Thankfulness Can Rewire Our Brains for Resilience, Optimism, and the Greater Good . Before joining the GGSC, Jeremy was a John S. Knight Journalism Fellow at Stanford University.

You May Also Enjoy

This article — and everything on this site — is funded by readers like you.

Become a subscribing member today. Help us continue to bring “the science of a meaningful life” to you and to millions around the globe.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Reading Research Effectively
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Reading a Scholarly Article or Research Paper

Identifying a research problem to investigate usually requires a preliminary search for and critical review of the literature in order to gain an understanding about how scholars have examined a topic. Scholars rarely structure research studies in a way that can be followed like a story; they are complex and detail-intensive and often written in a descriptive and conclusive narrative form. However, in the social and behavioral sciences, journal articles and stand-alone research reports are generally organized in a consistent format that makes it easier to compare and contrast studies and to interpret their contents.

General Reading Strategies

W hen you first read an article or research paper, focus on asking specific questions about each section. This strategy can help with overall comprehension and with understanding how the content relates [or does not relate] to the problem you want to investigate. As you review more and more studies, the process of understanding and critically evaluating the research will become easier because the content of what you review will begin to coalescence around common themes and patterns of analysis. Below are recommendations on how to read each section of a research paper effectively. Note that the sections to read are out of order from how you will find them organized in a journal article or research paper.

1.  Abstract

The abstract summarizes the background, methods, results, discussion, and conclusions of a scholarly article or research paper. Use the abstract to filter out sources that may have appeared useful when you began searching for information but, in reality, are not relevant. Questions to consider when reading the abstract are:

• Is this study related to my question or area of research?
• What is this study about and why is it being done ?
• What is the working hypothesis or underlying thesis?
• What is the primary finding of the study?
• Are there words or terminology that I can use to either narrow or broaden the parameters of my search for more information?

2.  Introduction

If, after reading the abstract, you believe the paper may be useful, focus on examining the research problem and identifying the questions the author is trying to address. This information is usually located within the first few paragraphs of the introduction or in the concluding paragraph. Look for information about how and in what way this relates to what you are investigating. In addition to the research problem, the introduction should provide the main argument and theoretical framework of the study and, in the last paragraphs of the introduction, describe what the author(s) intend to accomplish. Questions to consider when reading the introduction include:

• What is this study trying to prove or disprove?
• What is the author(s) trying to test or demonstrate?
• What do we already know about this topic and what gaps does this study try to fill or contribute a new understanding to the research problem?
• Why should I care about what is being investigated?
• Will this study tell me anything new related to the research problem I am investigating?

3.  Literature Review

The literature review describes and critically evaluates what is already known about a topic. Read the literature review to obtain a big picture perspective about how the topic has been studied and to begin the process of seeing where your potential study fits within the domain of prior research. Questions to consider when reading the literature review include:

• W hat other research has been conducted about this topic and what are the main themes that have emerged?
• What does prior research reveal about what is already known about the topic and what remains to be discovered?
• What have been the most important past findings about the research problem?
• How has prior research led the author(s) to conduct this particular study?
• Is there any prior research that is unique or groundbreaking?
• Are there any studies I could use as a model for designing and organizing my own study?

4.  Discussion/Conclusion

The discussion and conclusion are usually the last two sections of text in a scholarly article or research report. They reveal how the author(s) interpreted the findings of their research and presented recommendations or courses of action based on those findings. Often in the conclusion, the author(s) highlight recommendations for further research that can be used to develop your own study. Questions to consider when reading the discussion and conclusion sections include:

• What is the overall meaning of the study and why is this important? [i.e., how have the author(s) addressed the " So What? " question].
• What do you find to be the most important ways that the findings have been interpreted?
• What are the weaknesses in their argument?
• Do you believe conclusions about the significance of the study and its findings are valid?
• What limitations of the study do the author(s) describe and how might this help formulate my own research?
• Does the conclusion contain any recommendations for future research?

5.  Methods/Methodology

The methods section describes the materials, techniques, and procedures for gathering information used to examine the research problem. If what you have read so far closely supports your understanding of the topic, then move on to examining how the author(s) gathered information during the research process. Questions to consider when reading the methods section include:

• Did the study use qualitative [based on interviews, observations, content analysis], quantitative [based on statistical analysis], or a mixed-methods approach to examining the research problem?
• What was the type of information or data used?
• Could this method of analysis be repeated and can I adopt the same approach?
• Is enough information available to repeat the study or should new data be found to expand or improve understanding of the research problem?

6.  Results

After reading the above sections, you should have a clear understanding of the general findings of the study. Therefore, read the results section to identify how key findings were discussed in relation to the research problem. If any non-textual elements [e.g., graphs, charts, tables, etc.] are confusing, focus on the explanations about them in the text. Questions to consider when reading the results section include:

• W hat did the author(s) find and how did they find it?
• Does the author(s) highlight any findings as most significant?
• Are the results presented in a factual and unbiased way?
• Does the analysis of results in the discussion section agree with how the results are presented?
• Is all the data present and did the author(s) adequately address gaps?
• What conclusions do you formulate from this data and does it match with the author's conclusions?

7.  References

The references list the sources used by the author(s) to document what prior research and information was used when conducting the study. After reviewing the article or research paper, use the references to identify additional sources of information on the topic and to examine critically how these sources supported the overall research agenda. Questions to consider when reading the references include:

• Do the sources cited by the author(s) reflect a diversity of disciplinary viewpoints, i.e., are the sources all from a particular field of study or do the sources reflect multiple areas of study?
• Are there any unique or interesting sources that could be incorporated into my study?
• What other authors are respected in this field, i.e., who has multiple works cited or is cited most often by others?
• What other research should I review to clarify any remaining issues or that I need more information about?

NOTE :  A final strategy in reviewing research is to copy and paste the title of the source [journal article, book, research report] into Google Scholar . If it appears, look for a "cited by" followed by a hyperlinked number [e.g., Cited by 45]. This number indicates how many times the study has been subsequently cited in other, more recently published works. This strategy, known as citation tracking, can be an effective means of expanding your review of pertinent literature based on a study you have found useful and how scholars have cited it. The same strategies described above can be applied to reading articles you find in the list of cited by references.

Reading Tip

Specific Reading Strategies

Effectively reading scholarly research is an acquired skill that involves attention to detail and an ability to comprehend complex ideas, data, and theoretical concepts in a way that applies logically to the research problem you are investigating. Here are some specific reading strategies to consider.

As You are Reading

• Focus on information that is most relevant to the research problem; skim over the other parts.
• As noted above, read content out of order! This isn't a novel; you want to start with the spoiler to quickly assess the relevance of the study.
• Think critically about what you read and seek to build your own arguments; not everything may be entirely valid, examined effectively, or thoroughly investigated.
• Look up the definitions of unfamiliar words, concepts, or terminology. A good scholarly source is Credo Reference .

Taking notes as you read will save time when you go back to examine your sources. Here are some suggestions:

• Mark or highlight important text as you read [e.g., you can use the highlight text  feature in a PDF document]
• Take notes in the margins [e.g., Adobe Reader offers pop-up sticky notes].
• Highlight important quotations; consider using different colors to differentiate between quotes and other types of important text.
• Summarize key points about the study at the end of the paper. To save time, these can be in the form of a concise bulleted list of statements [e.g., intro has provides historical background; lit review has important sources; good conclusions].

Write down thoughts that come to mind that may help clarify your understanding of the research problem. Here are some examples of questions to ask yourself:

• Do I understand all of the terminology and key concepts?
• Do I understand the parts of this study most relevant to my topic?
• What specific problem does the research address and why is it important?
• Are there any issues or perspectives the author(s) did not consider?
• Do I have any reason to question the validity or reliability of this research?
• How do the findings relate to my research interests and to other works which I have read?

Adapted from text originally created by Holly Burt, Behavioral Sciences Librarian, USC Libraries, April 2018.

Another Reading Tip

When is it Important to Read the Entire Article or Research Paper

Laubepin argues, "Very few articles in a field are so important that every word needs to be read carefully." However, this implies that some studies are worth reading carefully. As painful and time-consuming as it may seem, there are valid reasons for reading a study in its entirety from beginning to end. Here are some examples:

• Studies Published Very Recently .  The author(s) of a recent, well written study will provide a survey of the most important or impactful prior research in the literature review section. This can establish an understanding of how scholars in the past addressed the research problem. In addition, the most recently published sources will highlight what is currently known and what gaps in understanding currently exist about a topic, usually in the form of the need for further research in the conclusion .
• Surveys of the Research Problem .  Some papers provide a comprehensive analytical overview of the research problem. Reading this type of study can help you understand underlying issues and discover why scholars have chosen to investigate the topic. This is particularly important if the study was published very recently because the author(s) should cite all or most of the key prior research on the topic. Note that, if it is a long-standing problem, there may be studies that specifically review the literature to identify gaps that remain. These studies often include the word review in their title [e.g., Hügel, Stephan, and Anna R. Davies. "Public Participation, Engagement, and Climate Change Adaptation: A Review of the Research Literature." Wiley Interdisciplinary Reviews: Climate Change 11 (July-August 2020): https://doi.org/10.1002/ wcc.645].
• Highly Cited .  If you keep coming across the same citation to a study while you are reviewing the literature, this implies it was foundational in establishing an understanding of the research problem or the study had a significant impact within the literature [positive or negative]. Carefully reading a highly cited source can help you understand how the topic emerged and motivated scholars to further investigate the problem. It also could be a study you need to cite as foundational in your own paper to demonstrate to the reader that you understand the roots of the problem.
• Historical Overview .  Knowing the historical background of a research problem may not be the focus of your analysis. Nevertheless, carefully reading a study that provides a thorough description and analysis of the history behind an event, issue, or phenomenon can add important context to understanding the topic and what aspect of the problem you may want to examine further.
• Innovative Methodological Design .  Some studies are significant and worth reading in their entirety because the author(s) designed a unique or innovative approach to researching the problem. This may justify reading the entire study because it can motivate you to think creatively about pursuing an alternative or non-traditional approach to examining your topic of interest. These types of studies are generally easy to identify because they are often cited in others works because of their unique approach to studying the research problem.
• Cross-disciplinary Approach .  R eviewing studies produced outside of your discipline is an essential component of investigating research problems in the social and behavioral sciences. Consider reading a study that was conducted by author(s) based in a different discipline [e.g., an anthropologist studying political cultures; a study of hiring practices in companies published in a sociology journal]. This approach can generate a new understanding or a unique perspective about the topic . If you are not sure how to search for studies published in a discipline outside of your major or of the course you are taking, contact a librarian for assistance.

Laubepin, Frederique. How to Read (and Understand) a Social Science Journal Article . Inter-University Consortium for Political and Social Research (ISPSR), 2013; Shon, Phillip Chong Ho. How to Read Journal Articles in the Social Sciences: A Very Practical Guide for Students . 2nd edition. Thousand Oaks, CA: Sage, 2015; Lockhart, Tara, and Mary Soliday. "The Critical Place of Reading in Writing Transfer (and Beyond): A Report of Student Experiences." Pedagogy 16 (2016): 23-37; Maguire, Moira, Ann Everitt Reynolds, and Brid Delahunt. "Reading to Be: The Role of Academic Reading in Emergent Academic and Professional Student Identities." Journal of University Teaching and Learning Practice 17 (2020): 5-12.

• << Previous: 1. Choosing a Research Problem
• Next: Narrowing a Topic Idea >>
• Last Updated: May 1, 2024 9:25 AM
• URL: https://libguides.usc.edu/writingguide

Asking Scholarly Questions with JSTOR Daily

Help students develop analytic and scholarly questioning skills using a quick activity built on JSTOR Daily roundups and syllabi.

One of the basic skills in my introductory United States History course that students often struggle with is asking scholarly questions. They usually understand that things like “who,” “what,” “where,” and “when” aren’t sufficiently scholarly or higher-level questions to ask, but they often assume that beginning a question with “why” or “how” means it will automatically rise to the level required in our course.

My first order of business, then, is to help them understand that the key to good scholarship is a good question—a scholarly question. We sometimes discuss how fictive detectives such as Sherlock Holmes and Benoit Blanc use a simple foundational query to form more complex lines of questioning based on types of logic. Other times, great research questions come to scholars as they’re reading or thinking of an entirely unrelated topic. Even these seemingly “aha!” moments are the result of training the mind to think like a scholar—to ask scholarly questions.

Weekly Newsletter

Get your fix of JSTOR Daily’s best stories in your inbox each Thursday.

Privacy Policy   Contact Us You may unsubscribe at any time by clicking on the provided link on any marketing message.

One of the ways I teach this is by assigning podcasts from historians who model this kind of questioning in their discussions. Another is by asking them to perform the task themselves using JSTOR Daily syllabi and roundups . The remainder of this article includes the specifics of this assignment—the instructions, rubric, and an example. Instructors are encouraged to modify the assignment to fit the needs of their course and student needs.

Scholarly Questions Assignment

Introduction.

Every day, scholars ask questions to help guide their work. But how do we know the difference between a normal question and a “scholarly” one? Often, when we think about asking questions in school and for research, we start with the basics:

These questions can be a great place to start when it comes to finding basic information. But when we want to dive deeper into the kinds of questions that scholars ask, we need to broaden our approach a little bit. Here are some characteristics of scholarly questions.

Scholarly questions:

• Make connections across time, place, or circumstances.
• Attempt to challenge or confirm previously held assumptions.
• Ask us to look at the way we view concepts already accepted as “fact.”

Here are some examples:

Instructions

1. Read at least three articles from one of the syllabi below:

• Moral Panics: A Syllabus
• Security Studies: A Syllabus
• Voting in American Politics: A Syllabus
• Charlottesville Syllabus: Readings on the History of Hate in America

2. In separate bullet points, identify three specific things that you learned, found interesting, or that contradicted your previously held knowledge/beliefs. These should be explained (specifically) in no less than three sentences each. This is not simply an account of what the article says—don’t plagiarize the article (or ask AI to summarize it!).

3. Write two scholarly questions that you want to ask as a result of reading the articles or that you wish you could ask the authors. The questions can refer to something from one article specifically, or they can make connections across multiple articles.

4.  Identify which syllabus you used in the title of your assignment. For any specific quotes or paraphrases, cite the article title in parentheses.

Support JSTOR Daily! Join our membership program on Patreon today.

Get Our Newsletter

More stories.

• The Fashionable Tour : or, The First American Tourist Guidebook

• Celebrating Asian American and Pacific Islander Heritage Month

The Metaphysical Story of Chiropractic

A Bodhisattva for Japanese Women

Recent posts.

• The Development of Central American Film
• Remembering Maud Lewis
• Rice, Famine, and the Seven Wonders of the World

Support JSTOR Daily

Sign up for our weekly newsletter.

25 Qualitative market research questions (and how to write your own)

25 examples of qualitative research questions, how to write your own insightful qualitative market research questions, ask the right qualitative market research questions to the correct audience.

There’s something very satisfying about being asked a great question that really gets you thinking. And in qualitative market research, it’s especially valuable.

If you ask the right person the right question, you’ll be able to uncover next steps — both small ones and big leaps — that will lead you to a better brand.

If you approach qualitative research right, you can get rich and valuable insights into your customers’ behaviors, and how to play into them.

You’ll learn about how customers interact, their motivations, and how to be there when they need you. And, you’ll uncover things about your brand that are difficult to find out from the inside.

We’re about to show you 25 qualitative research questions across six categories, that will allow you to take a deep dive into your target customers’ brain. These research questions are perfect to use in focus groups or with Attest’s Video Responses feature .

Qualitative research questions come in all shapes and sizes. We’ve split them up in several categories to inspire you to mix it up in your next survey or interview and make them work for your choice of qualitative research question types and methods.

Descriptive qualitative research questions

Descriptive questions are effective qualitative research questions that allow participants to describe experiences, opinions and more.

• Describe how this product/service has changed the way you approach [specific task/activity]. This question digs into the tangible impacts of your product on daily life, revealing how it reshapes routines or approaches to tasks.It’s a great way to highlight the practical value and possibly discover unexpected benefits that your product brings to the table.
• If you were to introduce this product/service to a friend, what would you say? Asking this encourages users to put their experience into their words, almost like a personal pitch. It’s a fun and low barrier approach to find out what stands out to them and what they value most about your offering.
• What three words would you use to describe this product/service after your first use? If you’re looking for immediate, instinctive reactions, qualitative research questions like these work best.They allow the user to give a quick snapshot and not have to think long and hard about an answer. Encourage them to respond with the first thing that comes to mind — no wrong answers.
• What aspect of this product/service do you think is underrated? This seeks to uncover hidden gems within your product that may not be getting the spotlight they deserve — even internally.It’s a clever way to find out about features or benefits that might be flying under the radar but have the potential to be major selling points.

Understand the nuance from in your audience’s behaviors

Getting that nuance through qual research will help you explain thy why to your quant ‘whats’, and give you much-needed inspiration during ideation

Comparative qualitative research questions

Using comparative qualitative research questions you can invite respondents to talk about your brand, product or services in comparison to others. It can help you understand the differences between you and your competitors, from your consumers’ perspective.

These qualitative research questions are a great addition to numbers, scores and other numerical data derived from quantitative research questions in a quantitative study.

• What differences do you notice between this brand and its competitors in terms of value provided? This question invites customers to think critically about the unique advantages or shortcomings of your product compared to the competition.It’s insightful because it can highlight what customers value most about your brand and whether you are doubling down on the right USPs according to them.
• In what situation would you prefer this competitors’ product/service over ours? Asking this might seem a bit daring, but it’s a golden opportunity to gain honest feedback on where your product may fall short for certain users or use cases.Research questions like this can uncover specific features, price points, or scenarios where competitors have an edge, offering you clear directions for strategic improvements or innovations.
• How does the ease of use of this product/service compare to others you have used in the past? This question zeros in on usability, a crucial aspect of customer satisfaction. It offers direct feedback on how user-friendly your product is compared to others, highlighting areas where you excel or need improvement.
• When choosing between this product/service and others, what factor weighs most heavily on your decision? Understanding the key factors that influence choice can help you fine-tune your offerings and marketing messages to better meet customer needs and preferences.If you look at the answers and compare the marketing efforts of your own brand and main competitors, you’ll be able to spot where you could make improvements.
• Can you identify a feature in a competing product/service that you wish ours had? Sometimes asking what feature they’d love is tricky: it might be hard to dream up. But if you give users the opportunity to shop from your competitors’ features, it might be easier.Qualitative research questions like these are therefore a smart and straightforward way to identify gaps in your product from a user perspective.

Exploratory qualitative research questions

Exploratory qualitative research questions are used in qualitative methods to tap into potential opportunities, and uncover insights that haven’t been previously considered. Add these research questions to your qualitative research studies if you’re on the hunt for new ideas.

• What challenges are you currently facing that this product/service does not address? This question is a gem in qualitative studies because it shines a light on the gaps between what your product offers and what your users actually need.By understanding these challenges, you’re not just guessing; you’re directly addressing the needs that matter most to your users, making every feature more aligned with their real-world problems.
• If you could add any premium features to this product/service, big or small, while the price remains the same, what would it be? Research questions like these open up a playground for users’ imaginations, allowing you to peek into their deepest wishes.It’s a creative way to use qualitative research to uncover independent variables (new features) that could make your product indispensable.
• What would make you stop using this product/service tomorrow? This one might sound a bit scary, but it’s crucial. It helps you pinpoint the deal breakers that could push your users away.Think of it as a preventive measure; by understanding these thresholds, you can steer clear of them in your future updates or service improvements. This question is a cornerstone in crafting a research design that seeks to minimize risks and maximize satisfaction.
• What’s a feature you never knew you needed until you started using this product/service? These insights are gold for marketing and product development, revealing the unexpected delights that can turn casual users into loyal fans.Plus, it’s a great way to highlight the qualitative words or phrases that resonate most with your audience, giving you a direct line to what makes your product stand out.
• If this product/service no longer existed, what would be the biggest gap in your routine or activities? This qualitative question helps to understand the role your product plays in users’ lives, emphasizing its importance and potential areas for highlighting in marketing efforts.Knowing what would replace you also tells you a great deal about the value your product offers.

Experience-based qualitative research questions

These qualitative research questions focus on the personal experiences of your users, and try to understand their journey and interactions with the product or service deeply.

• Describe a situation where this product/service met or exceeded your expectations. The feedback from this research question can reveal the “wow” factors that differentiate your offering in the market. It’s a great way to identify the elements of your product or service that surprise and delight customers.These qualitative questions will also highlight the specific words they use for this will also be great to fine-tune your communications and choice of words. You might be describing the right benefits already, but maybe not in the words they relate to most.
• What’s missing from your experience with this product/service? This research question is a direct line to understanding your customers’ unmet needs and desires. It encourages them to share their thoughts on how your product or service could be more useful, enjoyable, or relevant to their lives.
• What was your initial impression of this product/service, and how has it evolved? A classic, but nonetheless a valuable qualitative research question. If peoples’ experiences with your product change their impression of it over time, it’s crucial you dig into what those experiences are, to better match your marketing to the real world.Especially if impressions tend to take a more negative turn after some experiences, but also when it’s the other way around — don’t undersell your product!

Behavioral qualitative research questions

Behavioral qualitative research questions seek to understand the actions and behaviors of consumers, particularly in relation to your product or service. Adding these to your qualitative study will make it more relevant to daily life applications.

• Have you been using products/services like ours in ways that you didn’t think you would initially? This is a good qualitative research question to learn about unconventional or alternative use cases of your product. Of course, it doesn’t mean you immediately need to pivot, but it can help you map out uncharted or ignored territory and find fans in niche parts of your market.
• Has this product/service replaced something else you used to rely on? If so, what? We’re going there: ask about the ”ex”. Knowing who or what came before you and why things didn’t work out will help you be better in many ways. So, make sure to follow up this question with another one digging into the reasons for the break-up.
• What activity or task do you most frequently pair with this product/service? This might not seem immediately relevant, but it can tell you a great deal about your customer’s behavior. Knowing what place you have in their routine or what products they combine yours with can help you uncover big possibilities for innovations or even partnerships.
• How has this product/service influenced your daily habits or routines? This question doesn’t just focus on the functional benefits of your product, but also how those manifest in someone’s daily life. Do people highlight time they won back, or pleasure gained? Have they made any other changes that are relevant to you? There’s a lot to learn from small habit changes!

Emotional qualitative research questions

These qualitative questions explore the emotional connections and reactions participants have towards a particular topic, product, service, or brand. The qualitative questions examples below specifically bring a human side to quantitative research.

• How does this product/service fit into the moments that matter most to you? This might not be interesting for every product or brand, but if your brand is aiming to significantly impact people’s lives and important moments, this is a must-ask. Are they taking your products along to big moments in their lives? Does it provide them with comfort, confidence or something else when they need it? Research questions like this go way beyond functionality and tap into emotional significance — which is great for brands who really want to integrate with people’s lives.
• How does using this product/service make you feel compared to not using it at all? Are people frustrated when they run out of your product? Sad? Do they miss it at all? This question can reveal some powerful feelings around your product.
• How does this product/service affect your mood? This is a fun question to ask and can give you a great insight into what emotions your product evokes in general. Maybe some people don’t think about how they feel with your product, but others might get a confidence boost out of it, or chuckle every time they read your product copy. This question can reveal teeny tiny details that could matter a lot.

How this team aced their pitch with qual insights

Qual research helped Barrows understand glasses wearers’ pain points and wow a prospective client.

Got a burning question? Here’s how to make it part of a successful qualitative research project.

1. Set clear objectives

Knowing why you’re asking something is what helps you ask it the right way. Ask yourself what your general research objective is and how each of your qualitative research questions helps you get to the main answer.

For every qualitative research question you ask, find out IF it leads you closer to your goals, and make sure you can explain HOW it does so.

2. Ask open-ended questions the right way

There’s an art in asking great open-ended questions. Here’s an example of both seemingly similar open-ended, qualitative research questions, but one’s good, and one’s not:

“Why do you think people say this new smartwatch is better than others on the market?”

“What feedback are you hearing about this smartwatch in comparison to other smartwatches you know?”

The first one already works with the assumption and bias that your product is great. You’re practically putting words into your respondent’s mouth/answer box. The next one lets them come up with their own, unbiased response.

Good qualitative research questions should be:

• Unbiased: avoid qualitative research questions that are leading or show any form of bias whatsoever.
• Clear: only ask one thing at a time, and make it clear what that actually is.
• Relevant: make sure your question makes sense. Not just in the whole qualitative research, but also in the place it has in your survey or interview.
• Truly open: some quantitative research questions are sometimes disguised as qualitative research questions. Make sure yours is truly open and qualitative.

If you tick these boxes, your respondents will feel encouraged to express their opinions and motivations freely, which in turn will add depth and context to your research findings.

3. Balance between structured questions and flexibility

Let’s talk about flow. Imagine the panic that would set in if your first question in a job interview would be ”and how do you tackle problems with coworkers?”

Timing matters. Mixing up the question matters. This will create a great flow and keep respondents engaged and enthusiastic, and will avoid confusion.

Make sure to give respondents space to add comments or feedback where needed, but do so in a structured way, so your data remains easy to analyze.

4. Take measures to avoid survey bias

We’re circling back to bias for a second, but just because there’s more to be said and done. Avoiding bias in your surveys and qualitative research questions isn’t just about avoiding certain words or biased language, it also helps to choose a well-mixed and representative audience. On top of that, make sure your survey churns out high-quality data, not just high-volume. Read more about how we work on keeping your data in great shape.

5. Conduct pilot testing before launching large surveys

Do a mic-check before you send your qualitative research survey out to thousands of people. With pilot testing, you make sure your survey’s research questions are as-to-the-point as you hoped it to be. Send it to a small slice of your total audience, but make sure that the pilot group is just as representative as the total one will be.

When you hit that sweet spot of the right qualitative research questions and a perfectly represented audience , the feedback you receive from qualitative research isn’t just data—it’s a roadmap to deeper understanding and connection with your audience. For those looking to dive into the rich world of conducting qualitative research, Attest offers the market research tools and audience reach you need to make every question count. Check out how Attest can help bring your qualitative research to life.

VP Customer Success 

Sam joined Attest in 2019 and leads the Customer Research Team. Sam and her team support brands through their market research journey, helping them carry out effective research and uncover insights to unlock new areas for growth.

Related articles

4 positioning strategies to grow and market your brand, attest sponsors burberry british diversity awards for third year running, product webinar: getting better insights with enhanced segments, subscribe to our newsletter.

Fill in your email and we’ll drop fresh insights and events info into your inbox each week.

* I agree to receive communications from Attest. Privacy Policy .

You're now subscribed to our mailing list to receive exciting news, reports, and other updates!

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Writing Survey Questions

Perhaps the most important part of the survey process is the creation of questions that accurately measure the opinions, experiences and behaviors of the public. Accurate random sampling will be wasted if the information gathered is built on a shaky foundation of ambiguous or biased questions. Creating good measures involves both writing good questions and organizing them to form the questionnaire.

Questionnaire design is a multistage process that requires attention to many details at once. Designing the questionnaire is complicated because surveys can ask about topics in varying degrees of detail, questions can be asked in different ways, and questions asked earlier in a survey may influence how people respond to later questions. Researchers are also often interested in measuring change over time and therefore must be attentive to how opinions or behaviors have been measured in prior surveys.

Surveyors may conduct pilot tests or focus groups in the early stages of questionnaire development in order to better understand how people think about an issue or comprehend a question. Pretesting a survey is an essential step in the questionnaire design process to evaluate how people respond to the overall questionnaire and specific questions, especially when questions are being introduced for the first time.

For many years, surveyors approached questionnaire design as an art, but substantial research over the past forty years has demonstrated that there is a lot of science involved in crafting a good survey questionnaire. Here, we discuss the pitfalls and best practices of designing questionnaires.

Question development

There are several steps involved in developing a survey questionnaire. The first is identifying what topics will be covered in the survey. For Pew Research Center surveys, this involves thinking about what is happening in our nation and the world and what will be relevant to the public, policymakers and the media. We also track opinion on a variety of issues over time so we often ensure that we update these trends on a regular basis to better understand whether people’s opinions are changing.

At Pew Research Center, questionnaire development is a collaborative and iterative process where staff meet to discuss drafts of the questionnaire several times over the course of its development. We frequently test new survey questions ahead of time through qualitative research methods such as  focus groups , cognitive interviews, pretesting (often using an  online, opt-in sample ), or a combination of these approaches. Researchers use insights from this testing to refine questions before they are asked in a production survey, such as on the ATP.

Measuring change over time

Many surveyors want to track changes over time in people’s attitudes, opinions and behaviors. To measure change, questions are asked at two or more points in time. A cross-sectional design surveys different people in the same population at multiple points in time. A panel, such as the ATP, surveys the same people over time. However, it is common for the set of people in survey panels to change over time as new panelists are added and some prior panelists drop out. Many of the questions in Pew Research Center surveys have been asked in prior polls. Asking the same questions at different points in time allows us to report on changes in the overall views of the general public (or a subset of the public, such as registered voters, men or Black Americans), or what we call “trending the data”.

When measuring change over time, it is important to use the same question wording and to be sensitive to where the question is asked in the questionnaire to maintain a similar context as when the question was asked previously (see  question wording  and  question order  for further information). All of our survey reports include a topline questionnaire that provides the exact question wording and sequencing, along with results from the current survey and previous surveys in which we asked the question.

The Center’s transition from conducting U.S. surveys by live telephone interviewing to an online panel (around 2014 to 2020) complicated some opinion trends, but not others. Opinion trends that ask about sensitive topics (e.g., personal finances or attending religious services ) or that elicited volunteered answers (e.g., “neither” or “don’t know”) over the phone tended to show larger differences than other trends when shifting from phone polls to the online ATP. The Center adopted several strategies for coping with changes to data trends that may be related to this change in methodology. If there is evidence suggesting that a change in a trend stems from switching from phone to online measurement, Center reports flag that possibility for readers to try to head off confusion or erroneous conclusions.

Open- and closed-ended questions

One of the most significant decisions that can affect how people answer questions is whether the question is posed as an open-ended question, where respondents provide a response in their own words, or a closed-ended question, where they are asked to choose from a list of answer choices.

For example, in a poll conducted after the 2008 presidential election, people responded very differently to two versions of the question: “What one issue mattered most to you in deciding how you voted for president?” One was closed-ended and the other open-ended. In the closed-ended version, respondents were provided five options and could volunteer an option not on the list.

When explicitly offered the economy as a response, more than half of respondents (58%) chose this answer; only 35% of those who responded to the open-ended version volunteered the economy. Moreover, among those asked the closed-ended version, fewer than one-in-ten (8%) provided a response other than the five they were read. By contrast, fully 43% of those asked the open-ended version provided a response not listed in the closed-ended version of the question. All of the other issues were chosen at least slightly more often when explicitly offered in the closed-ended version than in the open-ended version. (Also see  “High Marks for the Campaign, a High Bar for Obama”  for more information.)

Researchers will sometimes conduct a pilot study using open-ended questions to discover which answers are most common. They will then develop closed-ended questions based off that pilot study that include the most common responses as answer choices. In this way, the questions may better reflect what the public is thinking, how they view a particular issue, or bring certain issues to light that the researchers may not have been aware of.

When asking closed-ended questions, the choice of options provided, how each option is described, the number of response options offered, and the order in which options are read can all influence how people respond. One example of the impact of how categories are defined can be found in a Pew Research Center poll conducted in January 2002. When half of the sample was asked whether it was “more important for President Bush to focus on domestic policy or foreign policy,” 52% chose domestic policy while only 34% said foreign policy. When the category “foreign policy” was narrowed to a specific aspect – “the war on terrorism” – far more people chose it; only 33% chose domestic policy while 52% chose the war on terrorism.

In most circumstances, the number of answer choices should be kept to a relatively small number – just four or perhaps five at most – especially in telephone surveys. Psychological research indicates that people have a hard time keeping more than this number of choices in mind at one time. When the question is asking about an objective fact and/or demographics, such as the religious affiliation of the respondent, more categories can be used. In fact, they are encouraged to ensure inclusivity. For example, Pew Research Center’s standard religion questions include more than 12 different categories, beginning with the most common affiliations (Protestant and Catholic). Most respondents have no trouble with this question because they can expect to see their religious group within that list in a self-administered survey.

In addition to the number and choice of response options offered, the order of answer categories can influence how people respond to closed-ended questions. Research suggests that in telephone surveys respondents more frequently choose items heard later in a list (a “recency effect”), and in self-administered surveys, they tend to choose items at the top of the list (a “primacy” effect).

Because of concerns about the effects of category order on responses to closed-ended questions, many sets of response options in Pew Research Center’s surveys are programmed to be randomized to ensure that the options are not asked in the same order for each respondent. Rotating or randomizing means that questions or items in a list are not asked in the same order to each respondent. Answers to questions are sometimes affected by questions that precede them. By presenting questions in a different order to each respondent, we ensure that each question gets asked in the same context as every other question the same number of times (e.g., first, last or any position in between). This does not eliminate the potential impact of previous questions on the current question, but it does ensure that this bias is spread randomly across all of the questions or items in the list. For instance, in the example discussed above about what issue mattered most in people’s vote, the order of the five issues in the closed-ended version of the question was randomized so that no one issue appeared early or late in the list for all respondents. Randomization of response items does not eliminate order effects, but it does ensure that this type of bias is spread randomly.

Questions with ordinal response categories – those with an underlying order (e.g., excellent, good, only fair, poor OR very favorable, mostly favorable, mostly unfavorable, very unfavorable) – are generally not randomized because the order of the categories conveys important information to help respondents answer the question. Generally, these types of scales should be presented in order so respondents can easily place their responses along the continuum, but the order can be reversed for some respondents. For example, in one of Pew Research Center’s questions about abortion, half of the sample is asked whether abortion should be “legal in all cases, legal in most cases, illegal in most cases, illegal in all cases,” while the other half of the sample is asked the same question with the response categories read in reverse order, starting with “illegal in all cases.” Again, reversing the order does not eliminate the recency effect but distributes it randomly across the population.

Question wording

The choice of words and phrases in a question is critical in expressing the meaning and intent of the question to the respondent and ensuring that all respondents interpret the question the same way. Even small wording differences can substantially affect the answers people provide.

[View more Methods 101 Videos ]

An example of a wording difference that had a significant impact on responses comes from a January 2003 Pew Research Center survey. When people were asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule,” 68% said they favored military action while 25% said they opposed military action. However, when asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule  even if it meant that U.S. forces might suffer thousands of casualties, ” responses were dramatically different; only 43% said they favored military action, while 48% said they opposed it. The introduction of U.S. casualties altered the context of the question and influenced whether people favored or opposed military action in Iraq.

There has been a substantial amount of research to gauge the impact of different ways of asking questions and how to minimize differences in the way respondents interpret what is being asked. The issues related to question wording are more numerous than can be treated adequately in this short space, but below are a few of the important things to consider:

First, it is important to ask questions that are clear and specific and that each respondent will be able to answer. If a question is open-ended, it should be evident to respondents that they can answer in their own words and what type of response they should provide (an issue or problem, a month, number of days, etc.). Closed-ended questions should include all reasonable responses (i.e., the list of options is exhaustive) and the response categories should not overlap (i.e., response options should be mutually exclusive). Further, it is important to discern when it is best to use forced-choice close-ended questions (often denoted with a radio button in online surveys) versus “select-all-that-apply” lists (or check-all boxes). A 2019 Center study found that forced-choice questions tend to yield more accurate responses, especially for sensitive questions.  Based on that research, the Center generally avoids using select-all-that-apply questions.

It is also important to ask only one question at a time. Questions that ask respondents to evaluate more than one concept (known as double-barreled questions) – such as “How much confidence do you have in President Obama to handle domestic and foreign policy?” – are difficult for respondents to answer and often lead to responses that are difficult to interpret. In this example, it would be more effective to ask two separate questions, one about domestic policy and another about foreign policy.

In general, questions that use simple and concrete language are more easily understood by respondents. It is especially important to consider the education level of the survey population when thinking about how easy it will be for respondents to interpret and answer a question. Double negatives (e.g., do you favor or oppose  not  allowing gays and lesbians to legally marry) or unfamiliar abbreviations or jargon (e.g., ANWR instead of Arctic National Wildlife Refuge) can result in respondent confusion and should be avoided.

Similarly, it is important to consider whether certain words may be viewed as biased or potentially offensive to some respondents, as well as the emotional reaction that some words may provoke. For example, in a 2005 Pew Research Center survey, 51% of respondents said they favored “making it legal for doctors to give terminally ill patients the means to end their lives,” but only 44% said they favored “making it legal for doctors to assist terminally ill patients in committing suicide.” Although both versions of the question are asking about the same thing, the reaction of respondents was different. In another example, respondents have reacted differently to questions using the word “welfare” as opposed to the more generic “assistance to the poor.” Several experiments have shown that there is much greater public support for expanding “assistance to the poor” than for expanding “welfare.”

We often write two versions of a question and ask half of the survey sample one version of the question and the other half the second version. Thus, we say we have two  forms  of the questionnaire. Respondents are assigned randomly to receive either form, so we can assume that the two groups of respondents are essentially identical. On questions where two versions are used, significant differences in the answers between the two forms tell us that the difference is a result of the way we worded the two versions.

One of the most common formats used in survey questions is the “agree-disagree” format. In this type of question, respondents are asked whether they agree or disagree with a particular statement. Research has shown that, compared with the better educated and better informed, less educated and less informed respondents have a greater tendency to agree with such statements. This is sometimes called an “acquiescence bias” (since some kinds of respondents are more likely to acquiesce to the assertion than are others). This behavior is even more pronounced when there’s an interviewer present, rather than when the survey is self-administered. A better practice is to offer respondents a choice between alternative statements. A Pew Research Center experiment with one of its routinely asked values questions illustrates the difference that question format can make. Not only does the forced choice format yield a very different result overall from the agree-disagree format, but the pattern of answers between respondents with more or less formal education also tends to be very different.

One other challenge in developing questionnaires is what is called “social desirability bias.” People have a natural tendency to want to be accepted and liked, and this may lead people to provide inaccurate answers to questions that deal with sensitive subjects. Research has shown that respondents understate alcohol and drug use, tax evasion and racial bias. They also may overstate church attendance, charitable contributions and the likelihood that they will vote in an election. Researchers attempt to account for this potential bias in crafting questions about these topics. For instance, when Pew Research Center surveys ask about past voting behavior, it is important to note that circumstances may have prevented the respondent from voting: “In the 2012 presidential election between Barack Obama and Mitt Romney, did things come up that kept you from voting, or did you happen to vote?” The choice of response options can also make it easier for people to be honest. For example, a question about church attendance might include three of six response options that indicate infrequent attendance. Research has also shown that social desirability bias can be greater when an interviewer is present (e.g., telephone and face-to-face surveys) than when respondents complete the survey themselves (e.g., paper and web surveys).

Lastly, because slight modifications in question wording can affect responses, identical question wording should be used when the intention is to compare results to those from earlier surveys. Similarly, because question wording and responses can vary based on the mode used to survey respondents, researchers should carefully evaluate the likely effects on trend measurements if a different survey mode will be used to assess change in opinion over time.

Question order

Once the survey questions are developed, particular attention should be paid to how they are ordered in the questionnaire. Surveyors must be attentive to how questions early in a questionnaire may have unintended effects on how respondents answer subsequent questions. Researchers have demonstrated that the order in which questions are asked can influence how people respond; earlier questions can unintentionally provide context for the questions that follow (these effects are called “order effects”).

One kind of order effect can be seen in responses to open-ended questions. Pew Research Center surveys generally ask open-ended questions about national problems, opinions about leaders and similar topics near the beginning of the questionnaire. If closed-ended questions that relate to the topic are placed before the open-ended question, respondents are much more likely to mention concepts or considerations raised in those earlier questions when responding to the open-ended question.

For closed-ended opinion questions, there are two main types of order effects: contrast effects ( where the order results in greater differences in responses), and assimilation effects (where responses are more similar as a result of their order).

An example of a contrast effect can be seen in a Pew Research Center poll conducted in October 2003, a dozen years before same-sex marriage was legalized in the U.S. That poll found that people were more likely to favor allowing gays and lesbians to enter into legal agreements that give them the same rights as married couples when this question was asked after one about whether they favored or opposed allowing gays and lesbians to marry (45% favored legal agreements when asked after the marriage question, but 37% favored legal agreements without the immediate preceding context of a question about same-sex marriage). Responses to the question about same-sex marriage, meanwhile, were not significantly affected by its placement before or after the legal agreements question.

Another experiment embedded in a December 2008 Pew Research Center poll also resulted in a contrast effect. When people were asked “All in all, are you satisfied or dissatisfied with the way things are going in this country today?” immediately after having been asked “Do you approve or disapprove of the way George W. Bush is handling his job as president?”; 88% said they were dissatisfied, compared with only 78% without the context of the prior question.

Responses to presidential approval remained relatively unchanged whether national satisfaction was asked before or after it. A similar finding occurred in December 2004 when both satisfaction and presidential approval were much higher (57% were dissatisfied when Bush approval was asked first vs. 51% when general satisfaction was asked first).

Several studies also have shown that asking a more specific question before a more general question (e.g., asking about happiness with one’s marriage before asking about one’s overall happiness) can result in a contrast effect. Although some exceptions have been found, people tend to avoid redundancy by excluding the more specific question from the general rating.

Assimilation effects occur when responses to two questions are more consistent or closer together because of their placement in the questionnaire. We found an example of an assimilation effect in a Pew Research Center poll conducted in November 2008 when we asked whether Republican leaders should work with Obama or stand up to him on important issues and whether Democratic leaders should work with Republican leaders or stand up to them on important issues. People were more likely to say that Republican leaders should work with Obama when the question was preceded by the one asking what Democratic leaders should do in working with Republican leaders (81% vs. 66%). However, when people were first asked about Republican leaders working with Obama, fewer said that Democratic leaders should work with Republican leaders (71% vs. 82%).

The order questions are asked is of particular importance when tracking trends over time. As a result, care should be taken to ensure that the context is similar each time a question is asked. Modifying the context of the question could call into question any observed changes over time (see  measuring change over time  for more information).

A questionnaire, like a conversation, should be grouped by topic and unfold in a logical order. It is often helpful to begin the survey with simple questions that respondents will find interesting and engaging. Throughout the survey, an effort should be made to keep the survey interesting and not overburden respondents with several difficult questions right after one another. Demographic questions such as income, education or age should not be asked near the beginning of a survey unless they are needed to determine eligibility for the survey or for routing respondents through particular sections of the questionnaire. Even then, it is best to precede such items with more interesting and engaging questions. One virtue of survey panels like the ATP is that demographic questions usually only need to be asked once a year, not in each survey.

U.S. Surveys

Other research methods, sign up for our weekly newsletter.

Fresh data delivered Saturday mornings

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

32 Questions to Ask on a College Visit

Students should feel free to ask questions during an information session or on tour.

Questions to Ask on a College Visit

Getty Images

Prospective students should conduct at least basic research to facilitate questions to ask during the information session or on tour, experts say.

Key Takeaways

• Before a campus visit, students should do basic research on the school.
• Students and their families have various opportunities to ask questions.
• No question is dumb.

College visits, whether in person or virtual, can help give prospective students a better feel of campus life.

Contrary to popular belief, however, students don’t need to have that “a-ha” moment when they eventually find the campus where they belong, says Thyra Briggs, vice president for admission and financial aid at Harvey Mudd College in California.

“I just don't think that happens for most students,” she says. “I don't want students to walk away from a visit where that didn't happen thinking, ‘Oh, this is not the place for me.’ This is a long-term relationship. It's not necessarily love at first sight. … In this age of instant gratification, I think it's an important thing to give a school a chance to affect you in a different way.”

For an in-person visit, families should prepare ahead of time by checking the weather and dressing comfortably as tours are mostly held outside.

"Leave plenty of time at an individual campus and allow yourself to enjoy the experience, be present in the moment and (don't) feel rushed because that could also skew your perception of things," says Bryan Gross, vice president for enrollment management at Hartwick College in New York.

It’s also important, experts say, to conduct at least basic research on the institution – even if it’s just looking at their social media accounts – to help facilitate questions to ask during the information session or on tour.

"We know that for some of you, this may be the first time you are going through this," Briggs says. "For others, it's a different student (going through the process) than the student you had who's older. So there’s no bad questions. ... I would hope that any college would welcome any question a student would ask.”

Here are 32 example questions, collected from college admissions and enrollment professions, that students don't always think to ask on college visits. These questions – edited for length or clarity – were provided by Briggs, Gross and Brian Lindeman, assistant vice president of admissions and financial aid at Macalester College  in Minnesota.

Questions About Admissions

• Does this school consider demonstrated interest?
• Is there an opportunity for prospective students to sit in on a class to experience a real lecture?
• Are there options to receive a lunch or dinner pass at the dining hall to try the food?

Questions About Academics

• Where do students typically study?
• How does advising work?
• What are the academic strengths of this school?
• What opportunities are there for study abroad and exchange programs?
• If available, are these global programs directly run by this school – where faculty members travel with students – or are these study abroad programs outsourced to a third-party company?
• Are these study abroad experiences built into the tuition or are there additional fees to participate?

Questions About Financial Aid

• What is this school's average financial aid package?
• What is the average net cost when students enroll?
• What is the current level of funding with endowed scholarships – how much are donors contributing to scholarships?
• Do you offer merit aid ? If so, what are you looking for in a candidate?

Questions About Campus Housing and Community

• What are the housing options?
• What are the fee structures for these different options?
• Are students required to live on campus ?
• How does your campus define diversity, equity, inclusion and belonging?

Questions to Ask Your Tour Guide to Gauge Campus Life

• What surprised you about this school? What's something you didn't expect?
• What keeps you coming back to this school each year?
• Have we seen your favorite place on campus?
• What event on campus gets the biggest turnout every year?
• If you were struggling with an issue, would you know who to turn to? Who would that be?

Questions About Work and Research Opportunities

• What are the opportunities for undergraduate research on campus?
• How do those research opportunities give students valuable hands-on experiences that enhance their resumes?
• What are some specific ways this school helps students gain hands-on experience through internships ?

Questions About Student and Career Outcomes

• What is the retention rate from freshman to sophomore year?
• What is the five-year graduation rate?
• What is the job-attainment rate of graduates within six months of graduating?
• What percent of students are going on to graduate school ?
• What percent of students are intentionally taking time off post-graduation compared to those who are not able to find jobs?
• What size is the alumni network?
• How are alumni actively engaging with recent graduates to help connect them specifically to opportunities in their fields?

Searching for a college? Get our  complete rankings  of Best Colleges.

Unique College Campus Visits

Tags: colleges , education , campus life , college applications , students

Ask an Alum: Making the Most Out of College

You May Also Like

Nationwide campus protests escalate.

Laura Mannweiler April 30, 2024

Best Colleges Surveys Are Out

Eric Brooks April 30, 2024

Columbia Gives Ultimatum to Protesters

Lauren Camera April 29, 2024

Photos: Pro-Palestinian Student Protests

Aneeta Mathur-Ashton and Avi Gupta April 26, 2024

How to Win a Fulbright Scholarship

Cole Claybourn and Ilana Kowarski April 26, 2024

Honors Colleges and Programs

Sarah Wood April 26, 2024

Find a Job in the Age of AI

Angie Kamath April 25, 2024

Protests Boil Over on College Campuses

Lauren Camera April 22, 2024

Supporting Low-Income College Applicants

Shavar Jeffries April 16, 2024

Supporting Black Women in Higher Ed

Zainab Okolo April 15, 2024

3 questions to ask yourself if you want to cut down on ultra-processed foods, according to a dietitian

• Ultra-processed foods are linked to health problems such as cancer, type 2 diabetes, and obesity.
• About 73% of the US food supply is ultra-processed, making reducing your intake tricky.
• A dietitian shared questions that will help make realistic changes easier.

Ultra-processed foods have become a buzzy topic as people worry how products that they rely on daily might be affecting their health.

Around 73% of the US food supply is ultra-processed, according to a 2024 research paper by Northeastern University's Network Science Institute, which hasn't been peer-reviewed. And more than 60% of calories in the average American's diet come from UPFs, according to one 2019 study.

Ultra-processed foods tend to be made from ingredients you wouldn't find in a regular kitchen and are manufactured using industrial techniques. Classic examples include candy, chicken nuggets, and soda. But packaged whole wheat bread and fruit yogurt can also fall into this category.

The pervasiveness of UPFs can make cutting down feel overwhelming.

To help, Linia Patel, a registered dietitian, British Dietetic Association spokesperson, and public health researcher based at the University of Milan, shared three questions to ask yourself if you want to cut down on UPFs , including how many you're eating.

As growing research suggests that eating a diet high in UPFs can lead to potential health problems such as cardiovascular disease, type 2 diabetes, obesity, and certain types of cancer, experts recommend people try to minimize their intake without fixating on eliminating them entirely.

"I think we're getting too bogged down with the term ultra-processed food, we just need to make it a bit simpler," Patel told Business Insider.

Think about overall dietary patterns rather than the one-off treats we all indulge in, she said, adding: "70, 80% I'm eating whole food, but 20% I honestly do not care what I eat, and I grab whatever I want and fast."

1) How much of your diet is ultra-processed?

Official bodies like the FDA provide basic nutritional guidelines, and seeing how your diet compares is a good place to start, Patel said.

Related stories

For example, we should be eating at least 28 grams of fiber and no more than 50g of added sugar daily.

Many UPFs are low in fiber and high in sugar, fat, and salt because they're ultra-palatable, meaning they contain the perfect combination of ingredients to make them irresistible and moreish.

It's also very important to look at food labels, Patel said, so that you actually know what you're eating.

UPFs come in all shapes and sizes, and some are certainly more nutrient-dense than others, she said. Even the same product made by a different manufacturer could contain more additives than another.

2) Are you snacking smart?

If you are a big snacker, it's worth taking a closer look at what you tend to eat, as items such as potato chips and cookies tend to be ultra-processed.

"What we are really bad at doing in the UK and in the USA is eating on the go, and that means that those eating-on-the-go moments are where we reach out for ultra-processed food," Patel said.

But she sees this as an opportunity to reduce UPF consumption by " snacking smarter ." "It could be an apple with a handful of nuts. That's something easy and portable that is going to be nutritious and balanced," she said.

If you really need your chocolate fix, you could eat a piece of fruit and then have some chocolate , which will probably mean you eat less of the processed snack overall, she said.

3) How well do you know yourself?

There's also a behavior change involved here, the same as any new habit, Patel said. So it's helpful to understand what works for you.

Some people like to dive into the deep end and might find going cold turkey a great approach. But for others, making small, incremental changes might be better, she said.

"Start with something that you feel is doable and manageable, and then you build on that," she said. You might identify that most of your UPFs are coming from snacks, so you start there. Then with time, you target breakfast, and later lunch.

Watch: All the differences between Lays chips in the US and India

• Main content