is a research based on people's real world experience

Qualitative Study

Affiliations.

• 1 University of Nebraska Medical Center
• 2 GDB Research and Statistical Consulting
• 3 GDB Research and Statistical Consulting/McLaren Macomb Hospital
• PMID: 29262162
• Bookshelf ID: NBK470395

Qualitative research is a type of research that explores and provides deeper insights into real-world problems. Instead of collecting numerical data points or intervene or introduce treatments just like in quantitative research, qualitative research helps generate hypotheses as well as further investigate and understand quantitative data. Qualitative research gathers participants' experiences, perceptions, and behavior. It answers the hows and whys instead of how many or how much. It could be structured as a stand-alone study, purely relying on qualitative data or it could be part of mixed-methods research that combines qualitative and quantitative data. This review introduces the readers to some basic concepts, definitions, terminology, and application of qualitative research.

Qualitative research at its core, ask open-ended questions whose answers are not easily put into numbers such as ‘how’ and ‘why’. Due to the open-ended nature of the research questions at hand, qualitative research design is often not linear in the same way quantitative design is. One of the strengths of qualitative research is its ability to explain processes and patterns of human behavior that can be difficult to quantify. Phenomena such as experiences, attitudes, and behaviors can be difficult to accurately capture quantitatively, whereas a qualitative approach allows participants themselves to explain how, why, or what they were thinking, feeling, and experiencing at a certain time or during an event of interest. Quantifying qualitative data certainly is possible, but at its core, qualitative data is looking for themes and patterns that can be difficult to quantify and it is important to ensure that the context and narrative of qualitative work are not lost by trying to quantify something that is not meant to be quantified.

However, while qualitative research is sometimes placed in opposition to quantitative research, where they are necessarily opposites and therefore ‘compete’ against each other and the philosophical paradigms associated with each, qualitative and quantitative work are not necessarily opposites nor are they incompatible. While qualitative and quantitative approaches are different, they are not necessarily opposites, and they are certainly not mutually exclusive. For instance, qualitative research can help expand and deepen understanding of data or results obtained from quantitative analysis. For example, say a quantitative analysis has determined that there is a correlation between length of stay and level of patient satisfaction, but why does this correlation exist? This dual-focus scenario shows one way in which qualitative and quantitative research could be integrated together.

Examples of Qualitative Research Approaches

Ethnography

Ethnography as a research design has its origins in social and cultural anthropology, and involves the researcher being directly immersed in the participant’s environment. Through this immersion, the ethnographer can use a variety of data collection techniques with the aim of being able to produce a comprehensive account of the social phenomena that occurred during the research period. That is to say, the researcher’s aim with ethnography is to immerse themselves into the research population and come out of it with accounts of actions, behaviors, events, etc. through the eyes of someone involved in the population. Direct involvement of the researcher with the target population is one benefit of ethnographic research because it can then be possible to find data that is otherwise very difficult to extract and record.

Grounded Theory

Grounded Theory is the “generation of a theoretical model through the experience of observing a study population and developing a comparative analysis of their speech and behavior.” As opposed to quantitative research which is deductive and tests or verifies an existing theory, grounded theory research is inductive and therefore lends itself to research that is aiming to study social interactions or experiences. In essence, Grounded Theory’s goal is to explain for example how and why an event occurs or how and why people might behave a certain way. Through observing the population, a researcher using the Grounded Theory approach can then develop a theory to explain the phenomena of interest.

Phenomenology

Phenomenology is defined as the “study of the meaning of phenomena or the study of the particular”. At first glance, it might seem that Grounded Theory and Phenomenology are quite similar, but upon careful examination, the differences can be seen. At its core, phenomenology looks to investigate experiences from the perspective of the individual. Phenomenology is essentially looking into the ‘lived experiences’ of the participants and aims to examine how and why participants behaved a certain way, from their perspective . Herein lies one of the main differences between Grounded Theory and Phenomenology. Grounded Theory aims to develop a theory for social phenomena through an examination of various data sources whereas Phenomenology focuses on describing and explaining an event or phenomena from the perspective of those who have experienced it.

Narrative Research

One of qualitative research’s strengths lies in its ability to tell a story, often from the perspective of those directly involved in it. Reporting on qualitative research involves including details and descriptions of the setting involved and quotes from participants. This detail is called ‘thick’ or ‘rich’ description and is a strength of qualitative research. Narrative research is rife with the possibilities of ‘thick’ description as this approach weaves together a sequence of events, usually from just one or two individuals, in the hopes of creating a cohesive story, or narrative. While it might seem like a waste of time to focus on such a specific, individual level, understanding one or two people’s narratives for an event or phenomenon can help to inform researchers about the influences that helped shape that narrative. The tension or conflict of differing narratives can be “opportunities for innovation”.

Research Paradigm

Research paradigms are the assumptions, norms, and standards that underpin different approaches to research. Essentially, research paradigms are the ‘worldview’ that inform research. It is valuable for researchers, both qualitative and quantitative, to understand what paradigm they are working within because understanding the theoretical basis of research paradigms allows researchers to understand the strengths and weaknesses of the approach being used and adjust accordingly. Different paradigms have different ontology and epistemologies . Ontology is defined as the "assumptions about the nature of reality” whereas epistemology is defined as the “assumptions about the nature of knowledge” that inform the work researchers do. It is important to understand the ontological and epistemological foundations of the research paradigm researchers are working within to allow for a full understanding of the approach being used and the assumptions that underpin the approach as a whole. Further, it is crucial that researchers understand their own ontological and epistemological assumptions about the world in general because their assumptions about the world will necessarily impact how they interact with research. A discussion of the research paradigm is not complete without describing positivist, postpositivist, and constructivist philosophies.

Positivist vs Postpositivist

To further understand qualitative research, we need to discuss positivist and postpositivist frameworks. Positivism is a philosophy that the scientific method can and should be applied to social as well as natural sciences. Essentially, positivist thinking insists that the social sciences should use natural science methods in its research which stems from positivist ontology that there is an objective reality that exists that is fully independent of our perception of the world as individuals. Quantitative research is rooted in positivist philosophy, which can be seen in the value it places on concepts such as causality, generalizability, and replicability.

Conversely, postpositivists argue that social reality can never be one hundred percent explained but it could be approximated. Indeed, qualitative researchers have been insisting that there are “fundamental limits to the extent to which the methods and procedures of the natural sciences could be applied to the social world” and therefore postpositivist philosophy is often associated with qualitative research. An example of positivist versus postpositivist values in research might be that positivist philosophies value hypothesis-testing, whereas postpositivist philosophies value the ability to formulate a substantive theory.

Constructivist

Constructivism is a subcategory of postpositivism. Most researchers invested in postpositivist research are constructivist as well, meaning they think there is no objective external reality that exists but rather that reality is constructed. Constructivism is a theoretical lens that emphasizes the dynamic nature of our world. “Constructivism contends that individuals’ views are directly influenced by their experiences, and it is these individual experiences and views that shape their perspective of reality”. Essentially, Constructivist thought focuses on how ‘reality’ is not a fixed certainty and experiences, interactions, and backgrounds give people a unique view of the world. Constructivism contends, unlike in positivist views, that there is not necessarily an ‘objective’ reality we all experience. This is the ‘relativist’ ontological view that reality and the world we live in are dynamic and socially constructed. Therefore, qualitative scientific knowledge can be inductive as well as deductive.”

So why is it important to understand the differences in assumptions that different philosophies and approaches to research have? Fundamentally, the assumptions underpinning the research tools a researcher selects provide an overall base for the assumptions the rest of the research will have and can even change the role of the researcher themselves. For example, is the researcher an ‘objective’ observer such as in positivist quantitative work? Or is the researcher an active participant in the research itself, as in postpositivist qualitative work? Understanding the philosophical base of the research undertaken allows researchers to fully understand the implications of their work and their role within the research, as well as reflect on their own positionality and bias as it pertains to the research they are conducting.

Data Sampling

The better the sample represents the intended study population, the more likely the researcher is to encompass the varying factors at play. The following are examples of participant sampling and selection:

Purposive sampling- selection based on the researcher’s rationale in terms of being the most informative.

Criterion sampling-selection based on pre-identified factors.

Convenience sampling- selection based on availability.

Snowball sampling- the selection is by referral from other participants or people who know potential participants.

Extreme case sampling- targeted selection of rare cases.

Typical case sampling-selection based on regular or average participants.

Data Collection and Analysis

Qualitative research uses several techniques including interviews, focus groups, and observation. [1] [2] [3] Interviews may be unstructured, with open-ended questions on a topic and the interviewer adapts to the responses. Structured interviews have a predetermined number of questions that every participant is asked. It is usually one on one and is appropriate for sensitive topics or topics needing an in-depth exploration. Focus groups are often held with 8-12 target participants and are used when group dynamics and collective views on a topic are desired. Researchers can be a participant-observer to share the experiences of the subject or a non-participant or detached observer.

While quantitative research design prescribes a controlled environment for data collection, qualitative data collection may be in a central location or in the environment of the participants, depending on the study goals and design. Qualitative research could amount to a large amount of data. Data is transcribed which may then be coded manually or with the use of Computer Assisted Qualitative Data Analysis Software or CAQDAS such as ATLAS.ti or NVivo.

After the coding process, qualitative research results could be in various formats. It could be a synthesis and interpretation presented with excerpts from the data. Results also could be in the form of themes and theory or model development.

Dissemination

To standardize and facilitate the dissemination of qualitative research outcomes, the healthcare team can use two reporting standards. The Consolidated Criteria for Reporting Qualitative Research or COREQ is a 32-item checklist for interviews and focus groups. The Standards for Reporting Qualitative Research (SRQR) is a checklist covering a wider range of qualitative research.

Examples of Application

Many times a research question will start with qualitative research. The qualitative research will help generate the research hypothesis which can be tested with quantitative methods. After the data is collected and analyzed with quantitative methods, a set of qualitative methods can be used to dive deeper into the data for a better understanding of what the numbers truly mean and their implications. The qualitative methods can then help clarify the quantitative data and also help refine the hypothesis for future research. Furthermore, with qualitative research researchers can explore subjects that are poorly studied with quantitative methods. These include opinions, individual's actions, and social science research.

A good qualitative study design starts with a goal or objective. This should be clearly defined or stated. The target population needs to be specified. A method for obtaining information from the study population must be carefully detailed to ensure there are no omissions of part of the target population. A proper collection method should be selected which will help obtain the desired information without overly limiting the collected data because many times, the information sought is not well compartmentalized or obtained. Finally, the design should ensure adequate methods for analyzing the data. An example may help better clarify some of the various aspects of qualitative research.

A researcher wants to decrease the number of teenagers who smoke in their community. The researcher could begin by asking current teen smokers why they started smoking through structured or unstructured interviews (qualitative research). The researcher can also get together a group of current teenage smokers and conduct a focus group to help brainstorm factors that may have prevented them from starting to smoke (qualitative research).

In this example, the researcher has used qualitative research methods (interviews and focus groups) to generate a list of ideas of both why teens start to smoke as well as factors that may have prevented them from starting to smoke. Next, the researcher compiles this data. The research found that, hypothetically, peer pressure, health issues, cost, being considered “cool,” and rebellious behavior all might increase or decrease the likelihood of teens starting to smoke.

The researcher creates a survey asking teen participants to rank how important each of the above factors is in either starting smoking (for current smokers) or not smoking (for current non-smokers). This survey provides specific numbers (ranked importance of each factor) and is thus a quantitative research tool.

The researcher can use the results of the survey to focus efforts on the one or two highest-ranked factors. Let us say the researcher found that health was the major factor that keeps teens from starting to smoke, and peer pressure was the major factor that contributed to teens to start smoking. The researcher can go back to qualitative research methods to dive deeper into each of these for more information. The researcher wants to focus on how to keep teens from starting to smoke, so they focus on the peer pressure aspect.

The researcher can conduct interviews and/or focus groups (qualitative research) about what types and forms of peer pressure are commonly encountered, where the peer pressure comes from, and where smoking first starts. The researcher hypothetically finds that peer pressure often occurs after school at the local teen hangouts, mostly the local park. The researcher also hypothetically finds that peer pressure comes from older, current smokers who provide the cigarettes.

The researcher could further explore this observation made at the local teen hangouts (qualitative research) and take notes regarding who is smoking, who is not, and what observable factors are at play for peer pressure of smoking. The researcher finds a local park where many local teenagers hang out and see that a shady, overgrown area of the park is where the smokers tend to hang out. The researcher notes the smoking teenagers buy their cigarettes from a local convenience store adjacent to the park where the clerk does not check identification before selling cigarettes. These observations fall under qualitative research.

If the researcher returns to the park and counts how many individuals smoke in each region of the park, this numerical data would be quantitative research. Based on the researcher's efforts thus far, they conclude that local teen smoking and teenagers who start to smoke may decrease if there are fewer overgrown areas of the park and the local convenience store does not sell cigarettes to underage individuals.

The researcher could try to have the parks department reassess the shady areas to make them less conducive to the smokers or identify how to limit the sales of cigarettes to underage individuals by the convenience store. The researcher would then cycle back to qualitative methods of asking at-risk population their perceptions of the changes, what factors are still at play, as well as quantitative research that includes teen smoking rates in the community, the incidence of new teen smokers, among others.

Copyright © 2024, StatPearls Publishing LLC.

• Introduction
• Issues of Concern
• Clinical Significance
• Enhancing Healthcare Team Outcomes
• Review Questions

Publication types

• Study Guide

Why Schools Need to Change Learning for Success in the Real World

Amanda Avallone (she/her/hers) Learning Officer (ret.) Next Generation Learning Challenges in Portland, Maine

Today’s learners face an uncertain present and a rapidly changing future that demand far different skills and knowledge than were needed in the 20th century. We also know so much more about enabling deep, powerful learning than we ever did before. Our collective future depends on how well young people prepare for the challenges and opportunities of 21st-century life.

Practitioner's Guide to Next Gen Learning

Preparing young people for life-long success through real-world learning

All students need to leave school—frequently, regularly, and of course, temporarily… To accomplish this, schools must take down the walls that separate the learning that students do, and could do, in school from the learning they do and could do, outside. —Elliot Washor and Charles Mojkowski, Big Picture Learning

In 2017 NGLC published the MyWays Student Success Series , a collection of reports, resources, and tools to support educators and communities to reimagine traditional definitions of learner success and redesign learning to help all students prepare for their futures. Inspired by the practices and valuable feedback from schools and districts in the NGLC network, the MyWays project has continued to evolve and expand to meet the needs of practitioners and their communities.

One area of particular interest for educators who are adopting broader definitions of success, like the MyWays Student Success Framework , is providing all students with authentic, real-world learning opportunities. Grace Belfiore, one of the principal researchers and authors of the MyWays report series, observes that, “One of the most striking implications of the MyWays’ research was the realization that it is difficult, if not impossible, to help learners develop these competencies and skills without going outside the classroom walls.” She also notes that, although some educators in the NGLC community have made real-world learning the cornerstone of their model, many others who have embraced next gen learning are newcomers to accessing the world outside of school as a rich source of student learning.

School lasts for 12 years, but students’ lives last a lot longer. What are we doing to develop people ?

For this edition of Friday Focus: Practitioner’s Guide to Next Gen Learning , I spoke to a variety of experts who are partnering with their communities to provide relevant and authentic student learning. This edition will also introduce practitioners to a new NGLC resource for this work, the MyWays Real-World Learning Toolkit . In particular, I’ll share:

• Why real-world learning is essential for student success
• Expert advice and resources to incorporate real-world learning in your classroom or school

Skills and Abilities for Lifelong Success

Early access to real-world projects is a springboard to seeing what success looks like outside of the classroom. —Natasha Morrison, director of real world learning at Da Vinci Schools.

The MyWays Student Success Framework and other 21st-century definitions of success share a fundamental understanding that the world has changed and continues to change at an accelerating rate. In addition to Content Knowledge, learners also need competencies within the MyWays domains of Habits of Success, Creative Know How, and Wayfinding Abilities to navigate through learning, career, and life. Young people need to experience learning that does more than prepare them to take a test or pass into the next grade.

Lindsey Stutheit, workforce partnership liaison at Laramie County School District #1 in Cheyenne, WY, expresses it this way: “For a long time, college- and career-readiness has really meant college, and that, in my experience, has meant preparing for the ACT or other state-mandated standardized tests. We spend a lot of resources and time preparing students for these tests, something fleeting and momentary. These tests are valid measures, but they aren’t geared around careers, personal skills, or adult skills. ” Lindsey and other leaders in her district recognize that young people need to develop capabilities that are life-long, life-wide, and life-worthy. “School lasts for 12 years,” she notes, “but students’ lives last a lot longer. What are we doing to develop people ?”

Casey Lamb, chief operating officer at Schools That Can , a network of schools that have embraced real-world learning, also argues for a more holistic notion of education and preparation. Real-world learning is, she says, “not just about the partnerships but about the outcomes, which move beyond what is usually tested in schools. It’s about developing whole humans.”

However, expecting schools to do this work alone, our experts tell us, is neither reasonable nor possible. According to Casey, one reason young people need access to learning experiences outside of school is that “the majority of teachers went to school to be teachers. They should not be expected to know everything about all of the myriad fields in order to prepare students for every career and opportunity. Real-world learning connects both students and teachers to people in the community who are working in different environments and have different perspectives, as well as to the skills adults use on a daily basis. ”

In this way, Casey explains, real-world experiences “ground students’ learning in the paths they will pursue after graduation. It better prepares them for those pathways, offers them choices, and gives them the ability to be successful there.”

Equitable Access to the World of Adults

Offering opportunities for young people to connect with the world outside of school is not new. Common activities like field trips, science fairs, model UN, theater productions, and extracurricular clubs can provide learners with access to adults in the community. Yet, as Lindsey observes, supporting young people to work side by side with adult professionals “is often teacher dependent and based on the connections individual teachers have. You also see the same students participating over and over.”

According to Lindsey, schools need to ensure that all students have access to support for wayfinding through education, work, and life, including meaningful connections to a wide range of adults from different industries and careers . This kind of social capital “is an equity issue for us and one reason it’s facilitated at the district level—to improve student exposure and reach.”

As described in MyWays Report 4: 5 Essentials in Building Social Capital , social capital is a developmental system of human relationships, including caring adults, mentors and coaches, and professional networks. According to the MyWays research, social capital plays two important roles in the life of young people: “as support in times of need and as social leverage to get ahead.” Significant disparities exist, however, between students of varying socioeconomic backgrounds in the opportunities to build social capital. Because both social capital roles are manifested in well-designed real-world learning, the MyWays Real-World Learning Toolkit includes a Social Capital Tool , which supports educators to design learning for social capital benefits for learners across the socioeconomic spectrum.

Real-World Contexts for “Whitewater Learning”

Real-world learning shares characteristics and benefits with other types of authentic, active learning. For example, both school-based and real-world projects can help learners develop Habits of Success , from initiative and perseverance to time management. However, real-world learning also provides the unique benefits, motivations, and opportunities of tackling real problems in the environments in which they occur, in all their messiness and urgency . As explored in the MyWays Key Elements of Real-World Learning Tool , well-designed outside-of-school experiences give learners the opportunity to encounter and respond to “whitewater conditions”—diverse challenges, complexities, and unfamiliar circumstances like those they will encounter as adults.

Skills like project management, collaboration, and problem-solving must be learned in a hands-on, high-stakes way.

Natasha explains that classroom experiences that simulate real-world learning are not enough, especially when it comes to preparing young people for future careers. “You can’t simulate social interaction. It’s not the same as an airline pilot using a flight simulator. There’s no substitute for engagement with the professional world. Attempting to develop career skills with virtually no real-world practice or feedback doesn’t achieve ideal outcomes . Partnership with industry professionals on real-world projects allows students to hone the skills they’ll actually use in the modern workforce. Skills like project management, collaboration, and problem-solving must be learned in a hands-on, high-stakes way.”

In addition to teaching a fuller range of competencies and facilitating connections to a wide array of adults, meaningful real-world learning provides learners with opportunities to make choices and practice self-direction. According to Grace, “ Experience in real and diverse situations is key to agency, and to help achieve growth, educators and youth advocates need to help students access and utilize a range of real-world situations .” Real-world learning can also foster identity development, she says, offering young people a vision of what’s possible for their futures.

Connections to the Wider Learning Ecosystem

Redesigning learning to be more like an apprenticeship for adult life can seem overwhelming. In part, that’s because educators have traditionally shouldered so much of this responsibility. However, schools and educators have numerous assets and partners for this work . Accessing the Wider Learning Ecosystem means leveraging the vast network of resources and experiences, formal and informal, that exist outside of school, including higher education, the workplace, and community assets like museums and libraries, as well as school-based extracurricular activities. Tapping into these assets is the focus of the MyWays 5 Zones of the Wider Learning Ecosystem Tool .

The 5 Zones of the Wider Learning Ecosystem

As featured in a Practitioner’s Guide about community partnerships , Natasha and her colleagues at Da Vinci Schools provide high school students with numerous real-world learning experiences throughout the Wider Learning Ecosystem. Partnerships with colleges, universities, and local businesses open up opportunities for apprentice adults to participate in dual or concurrent enrollment in higher education, job-shadowing, internships, career-focused boot camps, and other industry-based experiences. Educators at both Da Vinci Schools and in Laramie County stress that these experiences need to be well designed to achieve the desired impact. According to Lindsey, “Earning college credits should not be random. Credits should lead to students’ goals for after graduation.” Similarly, workplace experiences “can’t just be a job. They have to build career awareness and help students prepare for their futures.”

Though the kinds of experiences mentioned above are most common at the secondary level, building learner skills to be successful in a real-world learning context can occur at any grade , and educators can gradually integrate real-world learning elements over time. Casey defines real-world learning this way: “In its simplest form, it’s learning that is hands on, engaging, and connected with the world beyond school walls.”

When deeply implemented, real-world learning looks more like this: “All students access multiple opportunities and teachers are actively involved. Teachers have built relationships with the community and moved beyond offering a one-off project,” Casey explains. For instance, at the “leading level” of the Schools That Can Real-World Learning Rubric , “Real-world experiences are ongoing throughout the course of the year. Real-world learning is part of the fabric of the school and everyone benefits.”

For schools and educators new to real-world learning, Casey offers several ways to get started, such as by designing learning projects and experiences that are hands on and involve learning by doing . Maker education is one example. She also recommends inquiry-based approaches like Self-Organized Learning Environments (SOLE), in which educators present learners with real-world problems and frame projects “as a big question to research and explore and build solutions to. Ideally, learning should be interdisciplinary as well because that’s the way the real world works.”

As a natural next step for engaging with the world outside of school, Casey suggests “having students present to authentic audiences outside of school or bringing experts or professionals into the school. They can be judges on work and help kids get a different kind of feedback. It’s an easy way to start those partnerships and build a mutually beneficial relationship.”

The MyWays Readiness and Preparation Tool acknowledges that many of the attributes of real-world learning are very different from the kind of teaching and learning that happens in more traditional schools. In addition to building educator capacity for designing authentic, hands-on, and student-driven learning, this tool recognizes that students themselves and members of the community will likely need assistance as they shift their mindsets and think about learning in new ways . Lindsey’s experiences bear this reality out. She recounts, for example, “When I tell a local business leader that simply coming in and talking about your business might be hard for a teacher to justify without an educational goal in mind, it can be really eye-opening. People from industry and the community may know little about education, but it’s not insurmountable.”

For that reason, the Readiness and Preparation Tool helps schools gauge the readiness of educators, learners, and external partners from the Wider Learning Ecosystem. It also provides suggestions and resources to prepare all three partners to co-create powerful and successful real-world learning experiences.

• The MyWays Real-World Learning Toolkit provides educators and their community partners with four tools to support the design of powerful and successful real-world learning experiences.
• The Schools That Can Real-World Learning Rubric (free to download with registration) serves as a guide to help K-12 schools reflect, set goals, and drive improvements around RWL.
• Part one and part two of the Practitioner’s Guide series "It Takes a Village" shares practices and resources from three school systems deeply engaged in real-world learning: Da Vinci Schools (CA), St. Vrain Valley Schools (CO), and Vista Unified School District (CA).
• “ Who You Know: Building Students’ Social Capital ” explores in greater depth why social capital is an essential part of young people’s preparation for life and how schools and educators can support students in acquiring it.
• MyWays Report 11: Learning Design for Broader, Deeper Competencies presents research, design principles, and case studies on key practices, like real-world learning, that support student development of agency, social capital, and competencies for success.

Photo at top, courtesy of NGLC: Students at Vista High School engage in real-world learning.

Amanda Avallone (she/her/hers)

Learning officer (ret.), next generation learning challenges.

Amanda retired from Next Generation Learning Challenges in 2022. As a Learning Officer for NGLC, she collaborated with pioneering educators and their communities to design authentic, powerful learning experiences for young people. She created educator professional learning experiences that exemplify the kind of learning we want for our students and she supported, connected, and celebrated, through storytelling, the educators who are already doing the challenging work of transforming learning every day.

Read More About Why Schools Need to Change

Bring Your Vision for Student Success to Life with NGLC and Bravely

March 13, 2024

For Ethical AI, Listen to Teachers

Jason Wilmot

October 23, 2023

Turning School Libraries into Discipline Centers Is Not the Answer to Disruptive Classroom Behavior

Stephanie McGary

October 4, 2023

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

2.5: Conducting Psychology Research in the Real World

• Last updated
• Save as PDF
• Page ID 10603

• Matthias R. Mehl
• https://nobaproject.com/ via The Noba Project

University of Arizona

Because of its ability to determine cause-and-effect relationships, the laboratory experiment is traditionally considered the method of choice for psychological science. One downside, however, is that as it carefully controls conditions and their effects, it can yield findings that are out of touch with reality and have limited use when trying to understand real-world behavior. This module highlights the importance of also conducting research outside the psychology laboratory, within participants’ natural, everyday environments, and reviews existing methodologies for studying daily life.

learning objectives

• Identify limitations of the traditional laboratory experiment.
• Explain ways in which daily life research can further psychological science.
• Know what methods exist for conducting psychological research in the real world.

Introduction

The laboratory experiment is traditionally considered the “gold standard” in psychology research. This is because only laboratory experiments can clearly separate cause from effect and therefore establish causality. Despite this unique strength, it is also clear that a scientific field that is mainly based on controlled laboratory studies ends up lopsided. Specifically, it accumulates a lot of knowledge on what can happen—under carefully isolated and controlled circumstances—but it has little to say about what actually does happen under the circumstances that people actually encounter in their daily lives.

For example, imagine you are a participant in an experiment that looks at the effect of being in a good mood on generosity, a topic that may have a good deal of practical application. Researchers create an internally-valid, carefully-controlled experiment where they randomly assign you to watch either a happy movie or a neutral movie, and then you are given the opportunity to help the researcher out by staying longer and participating in another study. If people in a good mood are more willing to stay and help out, the researchers can feel confident that – since everything else was held constant – your positive mood led you to be more helpful. However, what does this tell us about helping behaviors in the real world? Does it generalize to other kinds of helping, such as donating money to a charitable cause? Would all kinds of happy movies produce this behavior, or only this one? What about other positive experiences that might boost mood, like receiving a compliment or a good grade? And what if you were watching the movie with friends, in a crowded theatre, rather than in a sterile research lab? Taking research out into the real world can help answer some of these sorts of important questions.

As one of the founding fathers of social psychology remarked, “Experimentation in the laboratory occurs, socially speaking, on an island quite isolated from the life of society” (Lewin, 1944, p. 286). This module highlights the importance of going beyond experimentation and also conducting research outside the laboratory (Reis & Gosling, 2010), directly within participants’ natural environments, and reviews existing methodologies for studying daily life.

Rationale for Conducting Psychology Research in the Real World

One important challenge researchers face when designing a study is to find the right balance between ensuring internal validity , or the degree to which a study allows unambiguous causal inferences, and external validity , or the degree to which a study ensures that potential findings apply to settings and samples other than the ones being studied (Brewer, 2000). Unfortunately, these two kinds of validity tend to be difficult to achieve at the same time, in one study. This is because creating a controlled setting, in which all potentially influential factors (other than the experimentally-manipulated variable) are controlled, is bound to create an environment that is quite different from what people naturally encounter (e.g., using a happy movie clip to promote helpful behavior). However, it is the degree to which an experimental situation is comparable to the corresponding real-world situation of interest that determines how generalizable potential findings will be. In other words, if an experiment is very far-off from what a person might normally experience in everyday life, you might reasonably question just how useful its findings are.

Because of the incompatibility of the two types of validity, one is often—by design—prioritized over the other. Due to the importance of identifying true causal relationships, psychology has traditionally emphasized internal over external validity. However, in order to make claims about human behavior that apply across populations and environments, researchers complement traditional laboratory research, where participants are brought into the lab, with field research where, in essence, the psychological laboratory is brought to participants. Field studies allow for the important test of how psychological variables and processes of interest “behave” under real-world circumstances (i.e., what actually does happen rather than what can happen ). They can also facilitate “downstream” operationalizations of constructs that measure life outcomes of interest directly rather than indirectly.

Take, for example, the fascinating field of psychoneuroimmunology, where the goal is to understand the interplay of psychological factors - such as personality traits or one’s stress level - and the immune system. Highly sophisticated and carefully controlled experiments offer ways to isolate the variety of neural, hormonal, and cellular mechanisms that link psychological variables such as chronic stress to biological outcomes such as immunosuppression (a state of impaired immune functioning; Sapolsky, 2004). Although these studies demonstrate impressively how psychological factors can affect health-relevant biological processes, they—because of their research design—remain mute about the degree to which these factors actually do undermine people’s everyday health in real life. It is certainly important to show that laboratory stress can alter the number of natural killer cells in the blood. But it is equally important to test to what extent the levels of stress that people experience on a day-to-day basis result in them catching a cold more often or taking longer to recover from one. The goal for researchers, therefore, must be to complement traditional laboratory experiments with less controlled studies under real-world circumstances. The term ecological validity is used to refer the degree to which an effect has been obtained under conditions that are typical for what happens in everyday life (Brewer, 2000). In this example, then, people might keep a careful daily log of how much stress they are under as well as noting physical symptoms such as headaches or nausea. Although many factors beyond stress level may be responsible for these symptoms, this more correlational approach can shed light on how the relationship between stress and health plays out outside of the laboratory.

An Overview of Research Methods for Studying Daily Life

Capturing “life as it is lived” has been a strong goal for some researchers for a long time. Wilhelm and his colleagues recently published a comprehensive review of early attempts to systematically document daily life (Wilhelm, Perrez, & Pawlik, 2012). Building onto these original methods, researchers have, over the past decades, developed a broad toolbox for measuring experiences, behavior, and physiology directly in participants’ daily lives (Mehl & Conner, 2012). Figure 1 provides a schematic overview of the methodologies described below.

Studying Daily Experiences

Starting in the mid-1970s, motivated by a growing skepticism toward highly-controlled laboratory studies, a few groups of researchers developed a set of new methods that are now commonly known as the experience-sampling method (Hektner, Schmidt, & Csikszentmihalyi, 2007), ecological momentary assessment (Stone & Shiffman, 1994), or the diary method (Bolger & Rafaeli, 2003). Although variations within this set of methods exist, the basic idea behind all of them is to collect in-the-moment (or, close-to-the-moment) self-report data directly from people as they go about their daily lives. This is typically accomplished by asking participants’ repeatedly (e.g., five times per day) over a period of time (e.g., a week) to report on their current thoughts and feelings. The momentary questionnaires often ask about their location (e.g., “Where are you now?”), social environment (e.g., “With whom are you now?”), activity (e.g., “What are you currently doing?”), and experiences (e.g., “How are you feeling?”). That way, researchers get a snapshot of what was going on in participants’ lives at the time at which they were asked to report.

Technology has made this sort of research possible, and recent technological advances have altered the different tools researchers are able to easily use. Initially, participants wore electronic wristwatches that beeped at preprogrammed but seemingly random times, at which they completed one of a stack of provided paper questionnaires. With the mobile computing revolution, both the prompting and the questionnaire completion were gradually replaced by handheld devices such as smartphones. Being able to collect the momentary questionnaires digitally and time-stamped (i.e., having a record of exactly when participants responded) had major methodological and practical advantages and contributed to experience sampling going mainstream (Conner, Tennen, Fleeson, & Barrett, 2009).

Over time, experience sampling and related momentary self-report methods have become very popular, and, by now, they are effectively the gold standard for studying daily life. They have helped make progress in almost all areas of psychology (Mehl & Conner, 2012). These methods ensure receiving many measurements from many participants, and has further inspired the development of novel statistical methods (Bolger & Laurenceau, 2013). Finally, and maybe most importantly, they accomplished what they sought out to accomplish: to bring attention to what psychology ultimately wants and needs to know about, namely “what people actually do, think, and feel in the various contexts of their lives” (Funder, 2001, p. 213). In short, these approaches have allowed researchers to do research that is more externally valid, or more generalizable to real life, than the traditional laboratory experiment.

To illustrate these techniques, consider a classic study, Stone, Reed, and Neale (1987), who tracked positive and negative experiences surrounding a respiratory infection using daily experience sampling. They found that undesirable experiences peaked and desirable ones dipped about four to five days prior to participants coming down with the cold. More recently, Killingsworth and Gilbert (2010) collected momentary self-reports from more than 2,000 participants via a smartphone app. They found that participants were less happy when their mind was in an idling, mind-wandering state, such as surfing the Internet or multitasking at work, than when it was in an engaged, task-focused one, such as working diligently on a paper. These are just two examples that illustrate how experience-sampling studies have yielded findings that could not be obtained with traditional laboratory methods.

Recently, the day reconstruction method (DRM) (Kahneman, Krueger, Schkade, Schwarz, & Stone, 2004) has been developed to obtain information about a person’s daily experiences without going through the burden of collecting momentary experience-sampling data. In the DRM, participants report their experiences of a given day retrospectively after engaging in a systematic, experiential reconstruction of the day on the following day. As a participant in this type of study, you might look back on yesterday, divide it up into a series of episodes such as “made breakfast,” “drove to work,” “had a meeting,” etc. You might then report who you were with in each episode and how you felt in each. This approach has shed light on what situations lead to moments of positive and negative mood throughout the course of a normal day.

Studying Daily Behavior

Experience sampling is often used to study everyday behavior (i.e., daily social interactions and activities). In the laboratory, behavior is best studied using direct behavioral observation (e.g., video recordings). In the real world, this is, of course, much more difficult. As Funder put it, it seems it would require a “detective’s report [that] would specify in exact detail everything the participant said and did, and with whom, in all of the contexts of the participant’s life” (Funder, 2007, p. 41).

As difficult as this may seem, Mehl and colleagues have developed a naturalistic observation methodology that is similar in spirit. Rather than following participants—like a detective—with a video camera (see Craik, 2000), they equip participants with a portable audio recorder that is programmed to periodically record brief snippets of ambient sounds (e.g., 30 seconds every 12 minutes). Participants carry the recorder (originally a microcassette recorder, now a smartphone app) on them as they go about their days and return it at the end of the study. The recorder provides researchers with a series of sound bites that, together, amount to an acoustic diary of participants’ days as they naturally unfold—and that constitute a representative sample of their daily activities and social encounters. Because it is somewhat similar to having the researcher’s ear at the participant’s lapel, they called their method the electronically activated recorder, or EAR (Mehl, Pennebaker, Crow, Dabbs, & Price, 2001). The ambient sound recordings can be coded for many things, including participants’ locations (e.g., at school, in a coffee shop), activities (e.g., watching TV, eating), interactions (e.g., in a group, on the phone), and emotional expressions (e.g., laughing, sighing). As unnatural or intrusive as it might seem, participants report that they quickly grow accustomed to the EAR and say they soon find themselves behaving as they normally would.

In a cross-cultural study, Ramírez-Esparza and her colleagues used the EAR method to study sociability in the United States and Mexico. Interestingly, they found that although American participants rated themselves significantly higher than Mexicans on the question, “I see myself as a person who is talkative,” they actually spent almost 10 percent less time talking than Mexicans did (Ramírez-Esparza, Mehl, Álvarez Bermúdez, & Pennebaker, 2009). In a similar way, Mehl and his colleagues used the EAR method to debunk the long-standing myth that women are considerably more talkative than men. Using data from six different studies, they showed that both sexes use on average about 16,000 words per day. The estimated sex difference of 546 words was trivial compared to the immense range of more than 46,000 words between the least and most talkative individual (695 versus 47,016 words; Mehl, Vazire, Ramírez-Esparza, Slatcher, & Pennebaker, 2007). Together, these studies demonstrate how naturalistic observation can be used to study objective aspects of daily behavior and how it can yield findings quite different from what other methods yield (Mehl, Robbins, & Deters, 2012).

A series of other methods and creative ways for assessing behavior directly and unobtrusively in the real world are described in a seminal book on real-world, subtle measures (Webb, Campbell, Schwartz, Sechrest, & Grove, 1981). For example, researchers have used time-lapse photography to study the flow of people and the use of space in urban public places (Whyte, 1980). More recently, they have observed people’s personal (e.g., dorm rooms) and professional (e.g., offices) spaces to understand how personality is expressed and detected in everyday environments (Gosling, Ko, Mannarelli, & Morris, 2002). They have even systematically collected and analyzed people’s garbage to measure what people actually consume (e.g., empty alcohol bottles or cigarette boxes) rather than what they say they consume (Rathje & Murphy, 2001). Because people often cannot and sometimes may not want to accurately report what they do, the direct—and ideally nonreactive—assessment of real-world behavior is of high importance for psychological research (Baumeister, Vohs, & Funder, 2007).

Studying Daily Physiology

In addition to studying how people think, feel, and behave in the real world, researchers are also interested in how our bodies respond to the fluctuating demands of our lives. What are the daily experiences that make our “blood boil”? How do our neurotransmitters and hormones respond to the stressors we encounter in our lives? What physiological reactions do we show to being loved—or getting ostracized? You can see how studying these powerful experiences in real life, as they actually happen, may provide more rich and informative data than one might obtain in an artificial laboratory setting that merely mimics these experiences.

Also, in pursuing these questions, it is important to keep in mind that what is stressful, engaging, or boring for one person might not be so for another. It is, in part, for this reason that researchers have found only limited correspondence between how people respond physiologically to a standardized laboratory stressor (e.g., giving a speech) and how they respond to stressful experiences in their lives. To give an example, Wilhelm and Grossman (2010) describe a participant who showed rather minimal heart rate increases in response to a laboratory stressor (about five to 10 beats per minute) but quite dramatic increases (almost 50 beats per minute) later in the afternoon while watching a soccer game. Of course, the reverse pattern can happen as well, such as when patients have high blood pressure in the doctor’s office but not in their home environment—the so-called white coat hypertension (White, Schulman, McCabe, & Dey, 1989).

Ambulatory physiological monitoring – that is, monitoring physiological reactions as people go about their daily lives - has a long history in biomedical research and an array of monitoring devices exist (Fahrenberg & Myrtek, 1996). Among the biological signals that can now be measured in daily life with portable signal recording devices are the electrocardiogram (ECG), blood pressure, electrodermal activity (or “sweat response”), body temperature, and even the electroencephalogram (EEG) (Wilhelm & Grossman, 2010). Most recently, researchers have added ambulatory assessment of hormones (e.g., cortisol) and other biomarkers (e.g., immune markers) to the list (Schlotz, 2012). The development of ever more sophisticated ways to track what goes on underneath our skins as we go about our lives is a fascinating and rapidly advancing field.

In a recent study, Lane, Zareba, Reis, Peterson, and Moss (2011) used experience sampling combined with ambulatory electrocardiography (a so-called Holter monitor) to study how emotional experiences can alter cardiac function in patients with a congenital heart abnormality (e.g., long QT syndrome). Consistent with the idea that emotions may, in some cases, be able to trigger a cardiac event, they found that typical—in most cases even relatively low intensity— daily emotions had a measurable effect on ventricular repolarization, an important cardiac indicator that, in these patients, is linked to risk of a cardiac event. In another study, Smyth and colleagues (1998) combined experience sampling with momentary assessment of cortisol, a stress hormone. They found that momentary reports of current or even anticipated stress predicted increased cortisol secretion 20 minutes later. Further, and independent of that, the experience of other kinds of negative affect (e.g., anger, frustration) also predicted higher levels of cortisol and the experience of positive affect (e.g., happy, joyful) predicted lower levels of this important stress hormone. Taken together, these studies illustrate how researchers can use ambulatory physiological monitoring to study how the little—and seemingly trivial or inconsequential—experiences in our lives leave objective, measurable traces in our bodily systems.

Studying Online Behavior

Another domain of daily life that has only recently emerged is virtual daily behavior or how people act and interact with others on the Internet. Irrespective of whether social media will turn out to be humanity’s blessing or curse (both scientists and laypeople are currently divided over this question), the fact is that people are spending an ever increasing amount of time online. In light of that, researchers are beginning to think of virtual behavior as being as serious as “actual” behavior and seek to make it a legitimate target of their investigations (Gosling & Johnson, 2010).

One way to study virtual behavior is to make use of the fact that most of what people do on the Web—emailing, chatting, tweeting, blogging, posting— leaves direct (and permanent) verbal traces. For example, differences in the ways in which people use words (e.g., subtle preferences in word choice) have been found to carry a lot of psychological information (Pennebaker, Mehl, & Niederhoffer, 2003). Therefore, a good way to study virtual social behavior is to study virtual language behavior. Researchers can download people’s—often public—verbal expressions and communications and analyze them using modern text analysis programs (e.g., Pennebaker, Booth, & Francis, 2007).

For example, Cohn, Mehl, and Pennebaker (2004) downloaded blogs of more than a thousand users of lifejournal.com, one of the first Internet blogging sites, to study how people responded socially and emotionally to the attacks of September 11, 2001. In going “the online route,” they could bypass a critical limitation of coping research, the inability to obtain baseline information; that is, how people were doing before the traumatic event occurred. Through access to the database of public blogs, they downloaded entries from two months prior to two months after the attacks. Their linguistic analyses revealed that in the first days after the attacks, participants expectedly expressed more negative emotions and were more cognitively and socially engaged, asking questions and sending messages of support. Already after two weeks, though, their moods and social engagement returned to baseline, and, interestingly, their use of cognitive-analytic words (e.g., “think,” “question”) even dropped below their normal level. Over the next six weeks, their mood hovered around their pre-9/11 baseline, but both their social engagement and cognitive-analytic processing stayed remarkably low. This suggests a social and cognitive weariness in the aftermath of the attacks. In using virtual verbal behavior as a marker of psychological functioning, this study was able to draw a fine timeline of how humans cope with disasters.

Reflecting their rapidly growing real-world importance, researchers are now beginning to investigate behavior on social networking sites such as Facebook (Wilson, Gosling, & Graham, 2012). Most research looks at psychological correlates of online behavior such as personality traits and the quality of one’s social life but, importantly, there are also first attempts to export traditional experimental research designs into an online setting. In a pioneering study of online social influence, Bond and colleagues (2012) experimentally tested the effects that peer feedback has on voting behavior. Remarkably, their sample consisted of 16 million (!) Facebook users. They found that online political-mobilization messages (e.g., “I voted” accompanied by selected pictures of their Facebook friends) influenced real-world voting behavior. This was true not just for users who saw the messages but also for their friends and friends of their friends. Although the intervention effect on a single user was very small, through the enormous number of users and indirect social contagion effects, it resulted cumulatively in an estimated 340,000 additional votes—enough to tilt a close election. In short, although still in its infancy, research on virtual daily behavior is bound to change social science, and it has already helped us better understand both virtual and “actual” behavior.

“Smartphone Psychology”?

A review of research methods for studying daily life would not be complete without a vision of “what’s next.” Given how common they have become, it is safe to predict that smartphones will not just remain devices for everyday online communication but will also become devices for scientific data collection and intervention (Kaplan & Stone, 2013; Yarkoni, 2012). These devices automatically store vast amounts of real-world user interaction data, and, in addition, they are equipped with sensors to track the physical (e. g., location, position) and social (e.g., wireless connections around the phone) context of these interactions. Miller (2012, p. 234) states, “The question is not whether smartphones will revolutionize psychology but how, when, and where the revolution will happen.” Obviously, their immense potential for data collection also brings with it big new challenges for researchers (e.g., privacy protection, data analysis, and synthesis). Yet it is clear that many of the methods described in this module—and many still to be developed ways of collecting real-world data—will, in the future, become integrated into the devices that people naturally and happily carry with them from the moment they get up in the morning to the moment they go to bed.

This module sought to make a case for psychology research conducted outside the lab. If the ultimate goal of the social and behavioral sciences is to explain human behavior, then researchers must also—in addition to conducting carefully controlled lab studies—deal with the “messy” real world and find ways to capture life as it naturally happens.

Mortensen and Cialdini (2010) refer to the dynamic give-and-take between laboratory and field research as “ full-cycle psychology ”. Going full cycle, they suggest, means that “researchers use naturalistic observation to determine an effect’s presence in the real world, theory to determine what processes underlie the effect, experimentation to verify the effect and its underlying processes, and a return to the natural environment to corroborate the experimental findings” (Mortensen & Cialdini, 2010, p. 53). To accomplish this, researchers have access to a toolbox of research methods for studying daily life that is now more diverse and more versatile than it has ever been before. So, all it takes is to go ahead and—literally—bring science to life.

Outside Resources

Discussion questions.

• What do you think about the tradeoff between unambiguously establishing cause and effect (internal validity) and ensuring that research findings apply to people’s everyday lives (external validity)? Which one of these would you prioritize as a researcher? Why?
• What challenges do you see that daily-life researchers may face in their studies? How can they be overcome?
• What ethical issues can come up in daily-life studies? How can (or should) they be addressed?
• How do you think smartphones and other mobile electronic devices will change psychological research? What are their promises for the field? And what are their pitfalls?
• Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2 , 396–403.
• Bolger, N., & Laurenceau, J-P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research . New York, NY: Guilford Press.
• Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616.
• Bond, R. M., Jones, J. J., Kramer, A. D., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61 million-person experiment in social influence and political mobilization. Nature , 489, 295–298.
• Brewer, M. B. (2000). Research design and issues of validity. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology (pp. 3–16). New York, NY: Cambridge University Press.
• Cohn, M. A., Mehl, M. R., & Pennebaker, J. W. (2004). Linguistic indicators of psychological change after September 11, 2001. Psychological Science, 15, 687–693.
• Conner, T. S., Tennen, H., Fleeson, W., & Barrett, L. F. (2009). Experience sampling methods: A modern idiographic approach to personality research. Social and Personality Psychology Compass, 3 , 292–313.
• Craik, K. H. (2000). The lived day of an individual: A person-environment perspective. In W. B. Walsh, K. H. Craik, & R. H. Price (Eds.), Person-environment psychology: New directions and perspectives (pp. 233–266). Mahwah, NJ: Lawrence Erlbaum Associates.
• Fahrenberg, J., &. Myrtek, M. (Eds.) (1996). Ambulatory assessment: Computer-assisted psychological and psychophysiological methods in monitoring and field studies . Seattle, WA: Hogrefe & Huber.
• Funder, D. C. (2007). The personality puzzle . New York, NY: W. W. Norton & Co.
• Funder, D. C. (2001). Personality. Review of Psychology, 52, 197–221.
• Gosling, S. D., & Johnson, J. A. (2010). Advanced methods for conducting online behavioral research . Washington, DC: American Psychological Association.
• Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: Personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82 , 379–398.
• Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life . Thousand Oaks, CA: Sage.
• Kahneman, D., Krueger, A., Schkade, D., Schwarz, N., and Stone, A. (2004). A survey method for characterizing daily life experience: The Day Reconstruction Method. Science , 306, 1776–780.
• Kaplan, R. M., & Stone A. A. (2013). Bringing the laboratory and clinic to the community: Mobile technologies for health promotion and disease prevention. Annual Review of Psychology, 64 , 471-498.
• Killingsworth, M. A., & Gilbert, D. T. (2010). A wandering mind is an unhappy mind. Science , 330, 932.
• Lane, R. D., Zareba, W., Reis, H., Peterson, D., &, Moss, A. (2011). Changes in ventricular repolarization duration during typical daily emotion in patients with Long QT Syndrome. Psychosomatic Medicine, 73 , 98–105.
• Lewin, K. (1944) Constructs in psychology and psychological ecology . University of Iowa Studies in Child Welfare, 20, 23–27.
• Mehl, M. R., & Conner, T. S. (Eds.) (2012). Handbook of research methods for studying daily life . New York, NY: Guilford Press.
• Mehl, M. R., Pennebaker, J. W., Crow, M., Dabbs, J., & Price, J. (2001). The electronically activated recorder (EAR): A device for sampling naturalistic daily activities and conversations. Behavior Research Methods, Instruments, and Computers, 33 , 517–523.
• Mehl, M. R., Robbins, M. L., & Deters, G. F. (2012). Naturalistic observation of health-relevant social processes: The electronically activated recorder (EAR) methodology in psychosomatics. Psychosomatic Medicine, 74 , 410–417.
• Mehl, M. R., Vazire, S., Ramírez-Esparza, N., Slatcher, R. B., & Pennebaker, J. W. (2007). Are women really more talkative than men? Science, 317 , 82.
• Miller, G. (2012). The smartphone psychology manifesto. Perspectives in Psychological Science , 7, 221–237.
• Mortenson, C. R., & Cialdini, R. B. (2010). Full-cycle social psychology for theory and application. Social and Personality Psychology Compass, 4, 53–63.
• Pennebaker, J. W., Mehl, M. R., Niederhoffer, K. (2003). Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology, 54 , 547–577.
• Ramírez-Esparza, N., Mehl, M. R., Álvarez Bermúdez, J., & Pennebaker, J. W. (2009). Are Mexicans more or less sociable than Americans? Insights from a naturalistic observation study. Journal of Research in Personality, 43 , 1–7.
• Rathje, W., & Murphy, C. (2001). Rubbish! The archaeology of garbage . New York, NY: Harper Collins.
• Reis, H. T., & Gosling, S. D. (2010). Social psychological methods outside the laboratory. In S. T. Fiske, D. T. Gilbert, & G. Lindzey, (Eds.), Handbook of social psychology (5th ed., Vol. 1, pp. 82–114). New York, NY: Wiley.
• Sapolsky, R. (2004). Why zebras don’t get ulcers: A guide to stress, stress-related diseases and coping . New York, NY: Henry Holt and Co.
• Schlotz, W. (2012). Ambulatory psychoneuroendocrinology: Assessing salivary cortisol and other hormones in daily life. In M.R. Mehl & T.S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 193–209). New York, NY: Guilford Press.
• Smyth, J., Ockenfels, M. C., Porter, L., Kirschbaum, C., Hellhammer, D. H., & Stone, A. A. (1998). Stressors and mood measured on a momentary basis are associated with salivary cortisol secretion. Psychoneuroendocrinology, 23 , 353–370.
• Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16 , 199–202.
• Stone, A. A., Reed, B. R., Neale, J. M. (1987). Changes in daily event frequency precede episodes of physical symptoms. Journal of Human Stress, 13 , 70–74.
• Webb, E. J., Campbell, D. T., Schwartz, R. D., Sechrest, L., & Grove, J. B. (1981). Nonreactive measures in the social sciences . Boston, MA: Houghton Mifflin Co.
• White, W. B., Schulman, P., McCabe, E. J., & Dey, H. M. (1989). Average daily blood pressure, not office blood pressure, determines cardiac function in patients with hypertension. Journal of the American Medical Association, 261 , 873–877.
• Whyte, W. H. (1980). The social life of small urban spaces . Washington, DC: The Conservation Foundation.
• Wilhelm, F.H., & Grossman, P. (2010). Emotions beyond the laboratory: Theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment. Biological Psychology, 84 , 552–569.
• Wilhelm, P., Perrez, M., & Pawlik, K. (2012). Conducting research in daily life: A historical review. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life . New York, NY: Guilford Press.
• Wilson, R., & Gosling, S. D., & Graham, L. (2012). A review of Facebook research in the social sciences. Perspectives on Psychological Science, 7 , 203–220.
• Yarkoni, T. (2012). Psychoinformatics: New horizons at the interface of the psychological and computing sciences. Current Directions in Psychological Science, 21 , 391–397.

Bridging the Lab and the Real World

• Applied Psychology
• Developmental Psychology
• Infant Development
• Sign Language

Whether and to what degree discoveries made in the lab generalize to the real world has been a long-standing debate among researchers of all stripes. New advances in technology and methodologies are enabling psychological scientists to bridge this divide and bring the controlled assessment of the lab into the world at large. Five researchers working in a variety of areas came together at the 2017 International Convention of Psychological Science in Vienna, to discuss the ways in which they balance, combine, and synergize the confines of the lab with the complex reality of our world.

Gesturing Toward Language

APS Past President Susan Goldin-Meadow of th University of Chicago uses a combination of lab-based and real-world environments to examine another aspect of infant development: gesture and its relation to language acquisition. It may seem intuitive that children use gestures as a stand-in for words they don’t know or can’t yet say, but Goldin-Meadow has found the movements are more than that. Pointing gestures not only function like words in children’s speech, but may actually be part of the word-learning process.

Goldin-Meadow first found indications of this by comparing the spontaneous gesture production of typically developing 14-month-old children with their vocabulary at 54 months. In addition to a correlation between gesture and vocabulary, she also found that the well-established positive association between socioeconomic status and child vocabulary size can be partially mediated by gesture production at 14 months.

To go deeper into this relationship and investigate the possible causal role of gesture, experimenters manipulated gesture production during a series of experimental sessions in the children’s homes. Experimenters did not use gesture in their sessions, used gesture but did not instruct the child to do so, or gestured and encouraged the child to do the same. Children who were told to gesture used more words in a follow-up assessment than did those who had only witnessed the experimenter gesture or who saw no gestures at all. They also produced more gestures with their parents outside of the experimental session. Because the experimenters manipulated gesture, the findings provide convincing evidence that gesturing can play a causal role in word learning.

In a series of lab-based studies (which will be followed up by neuroimaging studies), Goldin-Meadow also found that performing a gesture of an action (e.g., miming the turning of a knob) helps children better generalize that word to other knob-turning situations than does simply turning the knob themselves.

These observations about children’s use of gesturing and language in the real world, supported by laboratory testing, demonstrate how these two settings can work synergistically to provide us with new insights into development.

Just Moving to Move

APS Fellow Karen Adolph , a professor of psychology at New York University, aims to capture the complexity of infant learning beyond what has been observed in the lab. For example, technological advances such as head-mounted eye tracking for mobile infants have revealed that they don’t attend to their caregivers’ faces as much as previously believed — in one study, Adolph found that infants spent about 16% of the time looking at their parents and only 5% of the time focused specifically on parents’ faces.

Lab setups to study infant walking typically involve getting subjects to walk in a continuous, forward, straight path through a designated recording area. But infant ambulation in the real world is often far from a continuous, forward, straight path — babies stop and start, walk in every direction, and move in curves. These components were previously not able to be studied in the lab-based paradigm, but a larger recording area with a pressure-sensitive floor has enabled researchers to track how babies walk freely around a room.

Technological advances have allowed Adolph to investigate not just how babies walk, but why. Young children were typically theorized to use walking to get to a destination that they can see but that is not reachable from their current position. Using the same eye-tracking technology as the previous experiment, Adolph found that these destination-based bouts of walking account for only about 18% of toddlers’ movement. Sometimes they look toward one destination but then walk to another in what Adolph terms “discovery bouts,” accounting for about 10% of their walking trips. Babies, it turns out, are largely wanderers; most bouts do not have a destination at all.

“They are just moving to move,” says Adolph.

While lab tasks have the advantage of being well-controlled, Adolph’s findings reveal how they fail to capture the full picture.

“The cost of over-simplifying behaviors is that we lose sight of the phenomenon that we want to study,” she said. “In developmental psychology, over-reliance on laboratory tasks has led us to develop erroneous and superfluous theories about infant development.”

Adolph hopes to correct this trend moving forward.

Attention in Detail

To draw clear conclusions about the complex construct of attention, lab-based studies have typically separated out two main aspects of attentional shifting. These shifts can be driven by endogenous signals, which are voluntary and strategic (e.g., searching for a specific target), or exogenous signals, where attention is automatically shifted in response to an external stimulus — a more reflexive reaction.

Voluntary attentional shifting has been linked to the dorsal frontoparietal (dFP) region of the brain, while the ventral frontoparietal (vFP) area has been thought to primarily mediate reflexive shifts, though more recent evidence suggests the two systems may also interact and overlap with each other.

The lab studies that have provided these insights, however, use primarily stereotyped paradigms and simple, repeated stimuli, a far cry from the cohesive, complex visuospatial landscape we encounter in our everyday lives. Recent technological advances in eye tracking, saliency maps, and imaging methods have enabled researchers like Emiliano Macaluso, a professor at the Lyon Neuroscience Research Center, to study attention in a more naturalistic setting, where stimuli are more numerous and dynamic and where there is not always an explicit task to be done or goal to be achieved.

Macaluso has used dynamic visual environments, including first-person perspective videos and virtual environment setups, to examine how attentional shift is mediated in the brain under different conditions and to compare the responses of healthy subjects with those of subjects with lesions in the vFP region. Throughout these studies, he also has mapped and compared brain activation in individuals when viewing task-relevant objects and objects that were relevant only to a previous task, as well as when shifting their attention toward salient stimuli. These activation patterns were largely segregated between the ventral and dorsal networks, but also revealed greater nuance in how the brain controls attention, as the vFP was found to play a role in orienting attention toward task-relevant objects, and lesions in the ventral region interfered with this processing. Conversely, the dFP was activated during orientation toward salient events and locations.

Macaluso hopes to continue using these naturalistic settings to gain even greater insight into how the brain mediates the processes of attentional control.

Language Richness and Reward

Language and communication do not exist in a vacuum; they are constrained by both lower-level sensorimotor processes as well as higher-level social factors. Rick Dale, a cognitive scientist at the University of California, Los Angeles, has examined these dynamics in the lab. In one experiment, he and collaborators tracked the eye movements of two people looking at a shared screen while one spoke and the other listened. They found that when speakers’ and listeners’ eye movements were more closely aligned, the listener performed better on a subsequent comprehension test.

“Language and communication are actively entangled in these perceptual motor processes,” Dale said.

Thanks to the internet, large natural datasets are more readily available than ever, allowing Dale to investigate the social side of these constraints in the real world. Using Yelp Dataset, a natural dataset of reviews and tips written by hundreds of thousands of users about more than 60,000 businesses, Dale was able to study the relationship between social connectedness and various facets of the language of these reviews. He found a pattern of “community innovation,” in which more interconnected networks of users tended to use richer language than did less connected ones, indicating there may be a social incentive to use more lexically rich language.

To examine these phenomena in more controlled environments, Dale and his student recruited internet-based and lab participants to produce language by typing it out for an extended time. Participants faced simple, familiar tasks such as summarizing plots of favorite movies or writing reviews (as in the Yelp dataset). He measured their typing speeds and textual richness. He found that typing speeds were higher for low-richness textual inputs than for highly rich ones.

“There’s an incentive needed to communicate a richer message, and social incentive can induce a participant to invest more cognitive effort,” Dale explained. He noted that integrating new natural data with traditional psychological research can unveil causal linkages between the cognitive and the social. “What’s left is to link those typing dynamics to the social structure, so that indeed you can overcome the cognitive cost of typing richer text when there’s a potential social incentive that’s conditioned by the social network that one’s in.”

Navigating New Technology

Smartphones have changed not only the ways in which we communicate with each other, but also how we interact with the world. Yvonne Rogers, a professor of Interaction Design at University College London, focuses on questions about how technology affects people’s lives, how people behave when encountering a new technology, and how technology can be leveraged to engage communities and to inform new understandings of behavior.

One target of Rogers’ investigation has been the effect of using a GPS-enabled mobile device versus a paper map for navigation. Lab studies have shown that people using paper maps take less time to get to their destination, better remember the route, and have a better mental model of the area traversed.

Rogers wondered, however, what people were doing with all the cognitive resources freed up by mobile navigation, which trades cognitively taxing decision-making for lower-effort instruction-following. In a task where subjects had to navigate a route through London with either a paper map or a GPS, she found that participants who used paper maps looked at them more frequently, especially before critical turns, while those using smartphones checked them less and usually after critical turns. Subjects using smartphones recalled more street views and drew more detailed maps of the route they took. Users of paper maps drew routes that were more spatially accurate, but had fewer subjective and emotional descriptions of their trip than did smartphone users.

Rogers said this suggests that smartphone users experience the environment differently and adopt different strategies for navigating.

“Our findings compared to the previous lab findings were quite different, but also more positive. What we’re saying is [smartphone navigation] is not all bad,” she said.

This research is illustrative of the many new avenues of study that technological advances have opened up.

“I think researchers have new opportunities to change what they do, and to ask different questions and to use different techniques and methodologies by which to explore them,” Rogers said.

Technology is a good thing to have. However if it reduces, sometimes significantly, human interaction then it leads to people who are loners and hence having chances of mental problems. We know that people with mental problems are on the rise. I feel it is due to parenting which seems to have strayed from its mission of child rearing. I feel that this going astray may be assigned to technology which has created 24/7 global connectivity.

APS regularly opens certain online articles for discussion on our website. Effective February 2021, you must be a logged-in APS member to post comments. By posting a comment, you agree to our Community Guidelines and the display of your profile information, including your name and affiliation. Any opinions, findings, conclusions, or recommendations present in article comments are those of the writers and do not necessarily reflect the views of APS or the article’s author. For more information, please see our Community Guidelines .

Please login with your APS account to comment.

Teaching: Ethical Research to Help Romania’s Abandoned Children 

An early intervention experiment in Bucharest can introduce students to the importance of responsive caregiving during human development.

Silver Linings in the Demographic Revolution 

Podcast: In her final column as APS President, Alison Gopnik makes the case for more effectively and creatively caring for vulnerable humans at either end of life.

Communicating Psychological Science: The Lifelong Consequences of Early Language Skills

“When families are informed about the importance of conversational interaction and are provided training, they become active communicators and directly contribute to reducing the word gap (Leung et al., 2020).”

Privacy Overview

Chapter 15 Research Methods in the Real World

Applying what you’ve learned.

The examples of sociological research provided throughout this text come from a variety of positions on the basic-public-applied continuum presented in Chapter 1 "Introduction" . Some examples came from scholarly, peer-reviewed journal articles, others from public-interest magazines, and others from applied settings. Nevertheless, students sometimes walk away from a research course wondering how any of what they’ve learned applies to their lives today and to their future plans. In this, the final chapter, we explore that question. We’ll consider the variety of locations where research might crop up in your “real-world” life. For some, research might be a career. For others, perhaps research will provide a means to become engaged in social change efforts. For all of us, I hope that public sociology will present itself from time to time, perhaps in our reading, our web surfing, our television viewing, or our conversations with others. At the end of this chapter, we’ll remind ourselves of some of the answers to the “why should I care” question that we addressed at the beginning of this text. I hope that by now you have your own ideas about how you might answer that question but I’ll nevertheless remind you of the answers that we’ve already covered and provide a few others that perhaps hadn’t occurred to you.

15.1 Doing Research for a Living

Learning objectives.

• Identify the areas outside of academia where sociologists are most commonly employed.
• Define evaluation research and provide an example of it.
• Describe the work of a market researcher.
• Describe what sociologists working in policy and other government research do.

There are a variety of employers who hire social researchers. These include, but are not necessarily limited to, market research firms, corporations, public relations and communications firms, academic institutions, think tanks and other private research firms, public research firms and policy groups, and all levels of government. Some businesses hire social researchers to assist with personnel selection, many universities hire social researchers for their research institutes, For example, see University of Washington’s Social Development Research Group ( http://www.sdrg.org/ ), University of North Carolina at Chapel Hill’s Carolina Population Center ( http://www.cpc.unc.edu/ ), Penn State’s Survey Research Center ( http://www.ssri.psu.edu/survey ), University of Nebraska’s Public Policy Center ( http://ppc.unl.edu/ ), and University of Minnesota’s Immigration History Research Center ( http://www.ihrc.umn.edu/ ), to name just a few. and other firms such as Gallup ( http://www.gallup.com/home.aspx ) and Nielsen ( http://www.nielsen.com/us/en.html ) hire social researchers to examine societal trends. The areas where sociologists holding undergraduate degrees in research are most likely to find employment as researchers are in evaluation research, market research, and government research. Each of these represents a particular use of research rather than a research method per se. Evaluation, market, and government researchers may use any of the data collection or analysis strategies we described in Chapter 8 "Survey Research: A Quantitative Technique" and Chapter 12 "Other Methods of Data Collection and Analysis" , but their purpose and aims may differ. We’ll explore each of these different uses of social scientific research methods in the following.

Evaluation Research

As you may recall from the definition provided in Chapter 1 "Introduction" , evaluation research is research that is conducted to assess the effects of specific programs or policies. Evaluation research is often used when some form of social intervention is planned, such as welfare reform or school curriculum change. It might be used to assess the extent to which intervention is necessary by attempting to define and diagnose social problems, and it might also be used to understand whether applied interventions have had their intended consequences. Let’s consider a couple of specific examples of evaluation research to better understand how and when it is employed.

In Chapter 1 "Introduction" , I mentioned my experience conducting evaluation research with a transitional housing program. Among other services, workers at the transitional housing locations counseled residents on finding and maintaining employment. One purpose of the evaluation research therefore was to determine whether residents felt they were able to transition successfully back into their communities after a period of institutionalization by obtaining employment that could sustain a life outside of the transitional housing site. This outcomes assessment The act of judging whether a desired goal has been achieved. was conducted in order to determine whether the jobs counseling provided by the transitional housing employees produced the desired goal of preparing residents for finding and maintaining employment.

My first experience with evaluation research occurred during my senior year of college. That year, I conducted an internship at a hospital development office. My main task as an intern was to help the office assess how effective it had been in the preceding years in meeting its goal of raising local awareness of and support for the hospital. Using interview research methodology, I collected data from hospital employees and board members as well as members of the local community to learn about what people knew about the hospital, its development office, and the hospital’s services and needs. This project culminated in written report and a final presentation to several members of the hospital board in which I and the development office director outlined several recommendations for future development office activities based on the feedback provided by the people I had interviewed.

Being able to apply what I’d learned in my research methods class to a real-world problem and solutions was an invaluable experience. Not only that, while gaining this experience I was able to contribute to the well-being of my community by helping a needed local resource (the hospital) find ways improve its relationship with the community. Perhaps you could look for similar opportunities in your community. Of course, this specific example isn’t one of “doing research for a living,” as suggested by this section’s title, but it certainly gave me an experience worth noting on my resume and got me in the door of several potential employers for interviews when I began looking for jobs.

There are many other instances of applied evaluation research conducted by social scientists who are employed by firms for their skills as researchers. Just google the phrase evaluation research firm and you’ll find scores of examples. Different firms may specialize in different areas of research. For example, Hoffman Clark & Associates, a California-based firm, specializes in public health and K–12 education assessment ( http://www.hoffmanclark.org/index.php ). Arizona firm LeCroy & Milligan Associates Inc. conducts evaluation research in the areas of criminal justice and health and human services ( http://www.lecroymilligan.com/index.html ). In Colorado, Outcomes Inc. focuses on children and families ( http://www.outcomescolorado.com/home ). Wilder Research, based in Minnesota, conducts evaluation research designed to help strengthen families and their communities ( http://www.wilder.org/research.0.html ). Massachusetts firm Social Science Research & Evaluation Inc. specializes in, among other areas, evaluation research on highway safety and transportation ( http://www.ssre.org/index.html ). Finally, Inventivo Design LLC in Colorado tailors its evaluation research services to corporations wishing to assess whether their investments “meet the goals of management and deliver on objectives” ( http://www.inventivodesign.com ). As you can see from this very limited sampling of evaluation research firms, employment as an evaluation researcher could take you to just about any area of the country and involve work with any number of industries and sectors.

Market Research

Market research is another way that you might engage in social scientific research to make a living. Just as with evaluation research, market research is not a particular research method per se. Instead, it is a particular way of utilizing research methodology for a particular purpose. Market research Research that is conducted for the purpose of guiding businesses and other organizations as they make decisions about how best to sell, improve, or promote a product or service. is research that is conducted for the purpose of guiding businesses and other organizations as they make decisions about how best to sell, improve, or promote a product or service. This sort of research might involve gathering data from and about one’s core market and customers, about competitors, or about an industry more generally. Market research occurs in a variety of settings and institutions. Some firms specialize in market research specifically and are hired by others who wish to learn more about how to best promote or sell a product or service. Market research might also be conducted in-house, perhaps by large businesses that sell products or by nonprofits that wish to better understand how best to meet the needs of their clientele or promote their services.

Market researchers assess how best to sell, improve, or promote a product by gathering data about that product’s consumers. Understanding consumers’ preferences, tastes, attitudes, and behaviors can help point an organization in the right direction in its effort to reach and appeal to consumers. There are many ways to do this. You could observe customers in a store to watch which displays draw them in and which they ignore. You could administer a survey to assess consumers’ satisfaction with a good or service. You could conduct covert observations by being a secret shopper or dining someplace as though you, the researcher, are a real customer. You could conduct focus groups with consumers. As you already know from reading this text, social scientific research is an excellent way to gauge people’s preferences, tastes, attitudes, and behaviors. Each of these market research methods requires knowledge and skills in collecting data from human subjects—the very thing that sociological researchers do.

In the preceding section I identified just a small sampling of the many evaluation research firms that exist throughout the United States. There are also many firms that exist for the sole purpose of carrying out market research, all of which hire individuals who have a background in or knowledge about social scientific research methodology. Market research firms specialize in all kinds of areas. For example, Arbitron Inc. focuses on media, gathering data about radio audiences around the globe ( http://www.arbitron.com/home/content.stm ). From Maine, Market Decisions conducts market research on “a wide variety of topics from public policy to branding to feasibility” ( http://www.marketdecisions.com/index.php ). Nielsen, a company many are familiar with, conducts media research of all kinds ( http://www.nielsen.com/us/en.html ) but is perhaps best known for its ratings of television programming in the United States ( http://www.nielsen.com/us/en/insights/top10s/television.html ). Specializing in the area of information technology, Gartner collects data to help its clients make IT-related decisions ( http://www.gartner.com/technology/home.jsp ). These are just a few of the many potential market research employers that seek individuals with research skills.

Policy and Other Government Research

Finally, many social science researchers do policy and other government-related kinds of work. In fact, the federal government is one of the largest employers of applied social science researchers. Government and policy research could be in any number of areas. For example, nonpartisan private firms such as Child Trends ( http://www.childtrends.org/index.cfm ) conduct research that is specifically intended to be useful for policymakers. In the case of Child Trends, researchers aim to improve the lives of children by “conducting high-quality research and sharing it with the people and institutions whose decisions and actions affect children” ( http://www.childtrends.org/_catdisp_page.cfm?LID=124 ). Other private firms, such as Belden Russonello & Stewart, conduct research aimed at helping create social change, including projects on biodiversity, education, and energy use ( http://www.brspoll.com/index.htm ).

As for government work, Contexts magazine recently published an article featuring four sociological researchers to whom President Obama’s administration has turned, “relying on their unique understanding of American society to apply the most relevant research to policy-making” (2010, p. 14). Working for the G-man. (2010, Fall). Contexts, 9, 14–15. Those researchers include James P. Lynch, Bureau of Justice Statistics Director; John Laub, Director of the National Institute of Justice; Robert M. Groves, US Census Bureau Director; and David Harris, Deputy Assistant Secretary for Human Services Policy in the US Department of Health and Human Services.

Key Takeaway

• Sociologists are employed in many arenas. Some of the most common include evaluation research, market research, and policy and other government research.
• If you’re interested in hearing more from sociologists who do research, or sociology more generally, for a living, check out Contexts ’ article on “embedded sociologists” (Nyseth, Shannon, Heise, & McElrath, 2011) Nyseth, H., Shannon, S., Heise, K., & McElrath, S. M. (2011). Embedded sociologists. Contexts, 10, 44–50. who work in fields as diverse as epidemiology to housing rights to human resources. The article can be found online at http://contexts.org/articles/spring-2011/embedded-sociologists/ .

15.2 Doing Research for a Cause

• Define and provide at least one example of action research.
• Define stakeholders .

Some sociologists engage in research for reasons in addition to or aside from career motivations. These individuals might conduct some form of action research. While action research may be conducted as part of a person’s paid employment, as described in Section 15.1 "Doing Research for a Living" , you might also conduct action research as a volunteer working for a cause that you find worthy. If you’ve discovered that you have an interest in sociological research but would rather not pursue a career in research, perhaps some volunteer involvement in action is for you.

Action research Research that is conducted for the purpose of creating some form of social change. , sometimes referred to as participatory action research, is defined as research that is conducted for the purpose of creating some form of social change. When conducting action research, scholars collaborate with community stakeholders The groups or individuals for whom research is of direct benefit or concern. at all stages of the research process with the aim of producing results that will be usable in the community and by scientists. On the continuum of basic to applied research, action research is very far on the applied end of the spectrum. Sociologists who engage in this form of research never just go it alone; instead, they collaborate with the people who are affected by the research. Kristin Esterberg puts it quite eloquently when she says, “At heart, all action researchers are concerned that research not simply contribute to knowledge but also lead to positive changes in people’s lives” (2002, p. 137). Esterberg, K. G. (2002). Qualitative methods in social research . Boston, MA: McGraw-Hill. Action research was first developed in the 1960s and 1970s (Freire, 1970) Freire, P. (1970). Pedagogy of the oppressed (M. B. Ramos, Trans.). New York, NY: Herder and Herder. for the purpose of empowering individuals in underdeveloped nations (Reason, 1994). Reason, P. (1994). Participation in human inquiry . London, UK: Sage. Since then, action research has become increasingly popular among scholars who wish for their work to have tangible outcomes that benefit the groups that they study.

There are many excellent examples of action research. Some of these focus solely on arriving at useful outcomes for the communities upon which and with whom research is conducted. Other action research projects result in some new knowledge that has a practical application and purpose in addition to the creation of knowledge for basic scientific purposes. A search using the key term action research in Sociological Abstracts will yield a number of examples of the latter type.

One example of action research can be seen in Fred Piercy and colleagues’ (Piercy, Franz, Donaldson, & Richard, 2011) Piercy, F. P., Franz, N., Donaldson, J. L., & Richard, R. F. (2011). Consistency and change in participatory action research: Reflections on a focus group study about how farmers learn. The Qualitative Report, 16, 820–829. work with farmers in Virginia, Tennessee, and Louisiana. Together with farmers in these states, the researchers conducted focus groups to understand how farmers learn new information about farming. Ultimately, the aim of this study was to “develop more meaningful ways to communicate information to farmers about sustainable agriculture.” This improved communication, the researchers and farmers believed, would benefit not just researchers interested in the topic but also farmers and their communities. Farmers and researchers were both involved in all aspects of the research, from designing the project and determining focus group questions to conducting the focus groups and finally to analyzing data and disseminating findings.

Many additional examples of action research can be found at Loyola University Chicago’s Center for Urban Research and Learning (CURL; http://www.luc.edu/curl/index.shtml ). At the center, researchers seek “to promote equality and to improve people’s lives in communities throughout the Chicago metropolitan region.” For example, in 2006 researchers at CURL embarked on a project to assess the impact on small, local retailers of new Walmart stores entering urban areas (Jones, 2008). Jones, S. M. (2008, May 13). Cities may mute effect of Wal-Mart. Chicago Tribune . The study found that, while the effect of Walmart on local retailers seems to have a larger impact in rural areas, Chicago-area local retailers did not experience as dramatic an impact. Nevertheless a “small but statistically significant relationship” was found between Walmart’s arrival in the city and local retailers’ closing their doors. This and other research conducted by CURL aims to raise awareness about and promote positive social change around issues affecting the lives of people in the Chicago area. CURL meets this aim by collaborating with members of the community to shape a research agenda, collect and analyze data, and disseminate results.

Perhaps one of the most unique and rewarding aspects of engaging in action research is that it is often interdisciplinary. Action research projects might bring together researchers from any number of disciplines, from the social sciences, such as sociology, political science, and psychology; to an assortment of physical and natural sciences, such as biology and chemistry; to engineering, philosophy, and history (to name just a few). One recent example of this kind of interdisciplinary action research can be seen in the University of Maine’s Sustainability Solutions Initiative (SSI) ( http://www.umaine.edu/sustainabilitysolutions/index.htm ). This initiative unites researchers from across campus together with local community members to “connect knowledge with action in ways that promote strong economies, vibrant communities, and healthy ecosystems in and beyond Maine.” The knowledge-action connection is essential to SSI’s mission, and the collaboration between community stakeholders and researchers is crucial to maintaining that connection. SSI is a relatively new effort; stay tuned to the SSI website to follow how this collaborative action research initiative develops.

Anyone interested in social change can benefit from having some understanding of social scientific research methods. The knowledge you’ve gained from your methods course can be put to good use even if you don’t have an interest in pursuing a career in research. As a member of a community, perhaps you will find that the opportunity to engage in action research presents itself to you one day. And your background in research methodology will no doubt assist you and your collaborators in your effort to make life better for yourself and those who share your interests, circumstances, or geographic region.

Key Takeaways

• Action research is conducted by researchers who wish to create some form of social change.
• Action research is often conducted by teams of interdisciplinary researchers.
• If you’re interested in learning more about action research, or perhaps reading some specific examples of action research, check out the journal Gateways . It is a free, electronic, peer-reviewed scholarly journal focused on community-engaged research. Here’s the link: http://www.luc.edu/curl/uts/index.html .

15.3 Public Sociology

Learning objective.

• Identify and describe at least two examples of public sociology.

In Chapter 1 "Introduction" , we discussed public sociology and its place on the continuum of applied-basic research. One of the most delightful consequences of the trend toward public sociology is that the discipline has become more visible and more accessible to much broader audiences than perhaps ever before. But even with the increased accessibility of sociological research, you’ll find that having a basic understanding of how sociologists conduct research, which you’ve gained from this text, is beneficial. In this section, we’ll take a look at a few recent examples of public sociology and examine how your background in sociological research methods can help you read, make sense of, discuss, and even share the findings you come across.

In recent months, I’ve been interviewed by a journalist writing for a website run by Dr. Mehmet Oz of The Dr. Oz Show ( http://www.youbeauty.com ) and another writing for a website dedicated to any and every thing having to do with “video games and geek culture” ( http://www.unwinnable.com ). Inspired by the fall 2011 television programming lineup in the United States—in particular two new shows, including one featuring Playboy Bunnies and the other focused on the experiences of early PanAm flight attendants—the youbeauty.com interview focused on how expressions of gender, workplace norms, and harassment have changed in the past few decades. You can read the final article at http://www.youbeauty.com/relationships/the-secret-to-success-good-grooming . Interestingly, while the single quote attributed to me is accurate, the context within which I made the remark is not provided. One important caution for sociologists who choose to participate in press interviews is that your perspective may not always be represented in a way that you’d choose. In the other interview, conducted for an article on how heroism has changed since September 11, 2001 (Bannen, 2011), Bannen, B. (2011, July 19). Superheroes in a post-9/11 society. Unwinnable . Retrieved from http://www.unwinnable.com/2011/07/29/superheroes-in-a-post-911-society/ I was asked questions about patterns of social change. In both cases I was “doing” public sociology, drawing from my own background and knowledge about the sociological perspective on human behavior to help make sense of recent and current trends in society.

Many other sociologists engage in public sociology as well. Professor Pepper Schwartz, whose name you might recall from Chapter 4 "Beginning a Research Project" , is perhaps one of the most recognized public sociologists. In Chapter 4 "Beginning a Research Project" , I mentioned Schwartz’s role as the relationship expert for the dating website PerfectMatch.com. Schwartz is also the sex and relationship expert for the American Association for Retired Persons, for whom she writes a regular column offering advice to those aged 50 and up. Her participation with these venues enables Schwartz to provide relevant sociological understanding, perspective, and knowledge to broad audiences.

Another example of public sociology can be seen in Professor Nikki Jones’s work. Jones, an urban ethnographer who studies adolescent girls’ violence, has found that the “mean girl” phenomenon represented in so much of our popular culture and so many news stories today is far more hype than reality (Chesney-Lind & Jones, 2010; Jones, 2009). Jones, N. (2009). Between good and ghetto: African American girls and inner-city violence . Piscataway, NJ: Rutgers; Chesney-Lind, M., & Jones, N. (Eds.). (2010). Fighting for girls: New perspectives on gender and violence . Albany, NY: SUNY Press. In an effort to promote a better understanding of this and other matters of public interest upon which sociological and other scholarly evidence can and should be brought to bear, Jones collaborates with two other editors to maintain the website The Public Intellectual ( http://thepublicintellectual.org ). The site publishes work by academics and other researchers who write pieces intended to debunk “common knowledge” on matters of public concern, analyze social policies and problems, and examine cultural trends.

Finally, Professors Lisa Wade and Gwen Sharp provide another excellent example of public sociology on their website Sociological Images ( http://thesocietypages.org/socimages ). The site provides sociological observations and commentary on images of all kinds, from advertisements to charts and graphs, and from around the globe. Their aim is to “encourage all kinds of people to exercise and develop their sociological imagination by presenting brief sociological discussions of compelling and timely imagery that spans the breadth of sociological inquiry.” The images Wade and Sharp display on the site are chosen for their ability to illustrate sociological ideas in a way that is both compelling and accessible to sociological and nonsociological audiences alike. Peruse their site and as you’ll see from the comments noted underneath each of the discussion/image posts that the Sociological Images audience runs the gamut in background, ideology, and perspective. In other words, the site accomplishes the exact aim of public sociology: to engage the public.

• One of the positive consequences of public sociology is that the discipline has become more visible and more accessible to much broader audiences than in the past.
• Having a background in sociological research methods can help you read, make sense of, discuss, and share the research findings you encounter.
• Check out at least one of the websites mentioned in the preceding section. What do you think of these examples of public sociology? Ask one of your nonsociologist friends to peruse the site. Discuss what you each found compelling. How does your knowledge of sociological research methods shape your understanding of what you’ve read?
• Discuss public sociology with a few of your sociology peers. In what areas do you think public sociology can and should play a role?

15.4 Revisiting an Earlier Question: Why Should We Care?

• Define transferable skills .
• Identify several of the transferable skills you’ve gained from your understanding of sociological research methods.

I hope that by now I’ve managed to convince you that developing an understanding of how sociologists conduct research has many benefits. On the chance that I haven’t done so, or in case you simply want a refresher, let’s spend this final section of the final chapter reviewing some of the reasons you might care about research methods.

Transferable Skills

In Chapter 1 "Introduction" , I suggested that one reason to care about research methods is that knowing how to conduct social science research could lead to a variety of job opportunities. The skills and knowledge you’ve gained from this text will situate you well for a number of research-oriented positions. Moreover, your background in social science research methodology provides you with a number of transferable skills The conglomeration of tasks that a person develops proficiency in from one realm that can be applied in another realm. that will serve you well in any profession you choose. Transferable skills are the conglomeration of tasks that a person develops proficiency in from one realm that can be applied in another realm. Whether you realize it or not, you have gained a host of transferable skills from taking a course in social scientific research methods. Those skills can assist you in your search for employment in a variety of arenas.

Perhaps the primary transferable skill you’ve developed by learning how to conduct social scientific research is an ability to solve problems. Not only that, you are now also better equipped to identify problems. What do social researchers do if not identify social problems and then seek to gain knowledge aimed at understanding and eradicating those problems? Having the ability to seek out problems and the requisite knowledge and tools to begin to solve those problems is crucial in many areas of employment. The investigative skills you’ve developed as a result of learning how to conduct social scientific research can be put to use in just about any job where taken-for-granted assumptions are called into question. These might include jobs such as journalism, but work in criminal justice requires investigative skills as does just about any position that requires one to solve problems, ask questions, and learn new ways of doing things.

Related to the problem-identification and problem-solving skills that you’ve developed by learning how to conduct social scientific research is another important ability: a talent for asking good questions. Not only is the ability to ask good questions essential in many areas of employment (and in most areas life as well), but also this skill is linked to another key area that comes up in research methods courses and is appreciated in many realms: critical thinking The careful evaluation of assumptions, actions, values, and other factors that influence a particular decision or way of being or doing. . Thinking critically does not mean that someone sits backs and criticizes every idea or person that comes her way. Critical thinking is a skill that takes practice to develop. It involves the careful evaluation of assumptions, actions, values, and other factors that influence a particular way of being or doing. It requires an ability to identify both weaknesses and strengths in taken-for-granted ways of doing things. A person who thinks critically should be able to demonstrate some level of understanding of the varying positions one might take on any given issue, even if he or she does not agree with those positions.

Understanding sociological research methods also means having some understanding of how to analyze, synthesize, and interpret information. And having a well-developed ability to carefully take in, think about, and understand the meaning of new information that you are confronted with will serve you well in all varieties of life circumstance and employment. In addition, the ability to communicate and clearly express oneself, both in writing and orally, is crucial in all professions. As you practice the tasks described throughout this text, you will attain and improve the oral and written communication skills that so many employers value. Finally, related to the ability to communicate effectively is the ability to effectively frame an argument or presentation. Successfully framing an argument requires not only good communication skills but also strength in the area of listening to others.

In sum, the transferable skills you’ve gained as a result of learning how to conduct social scientific research include the following:

• Identifying problems
• Identifying solutions to problems
• Investigative skills and techniques
• Asking good questions
• Framing an argument
• Critical thinking
• Analyzing, synthesizing, and interpreting information
• Oral and written communication skills

Table 15.1 "Transferable Skills Featured in This Text" links each of the identified transferable skills to specific chapters in the text.

Table 15.1 Transferable Skills Featured in This Text

Understanding Yourself, Your Circumstances, and Your World

Perhaps the most rewarding consequence of understanding social scientific research methods is the ability to gain a better understanding of yourself, your circumstances, and your world. Through the application of social scientific research methods, sociologists have asked—and answered—many of the world’s most pressing questions. Certainly those answers are not always complete, nor are they infallible, but the quest for knowledge and understanding is an ongoing process. As social scientists continue the process of asking questions and seeking answers, perhaps you will choose to participate in that quest now that you have gained some knowledge and skill in how to conduct research.

Having thought about what you know and how you know it, as well as what others claim to know and how they know it, I hope will provide you with some clarity in an often-murky world. Whether you choose to adopt the particular ways of knowing described in this text as your preferred ways of knowing is totally up to you. I hope that you will find that the knowledge you’ve gained here is of use, perhaps in terms of your personal life and interests, in your relationships with others, or in your longer-range school or career goals.

• Having a background in social science research methodology provides you with a number of transferable skills.
• Having a background in social science research methodology gives you the opportunity to gain greater insight into yourself, your circumstances, and your world.
• If you’re interested in gaining some more research experience, check out the National Science Foundation’s Research for Undergraduates (REU) program. The program provides opportunities for students to conduct research at a host institution along with a small group of undergraduate peers. To learn more about the program and search for current locations hosting REU programs, see the following: http://www.nsf.gov/crssprgm/reu/ .
• Review Table 15.1 "Transferable Skills Featured in This Text" . Are there transferable skills listed there that you’re not yet convinced you’ve attained? If so, take another look at the cited chapter(s). Are there transferable skills you feel you’ve gained that are not listed in the table? If so, what are they and in which chapter(s) are they featured?

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

6 Conducting Psychology Research in the Real World

Original chapter by matthias r. mehl adapted by the queen’s university psychology department.

This Open Access chapter was originally written for the NOBA project. Information on the NOBA project can be found below.

We encourage students to use the “Three-Step Method” for support in their learning. Please find our version of the Three-Step Method, created in collaboration with Queen’s Student Academic Success Services, at the following link: https://sass.queensu.ca/psyc100/

Because of its ability to determine cause-and-effect relationships, the laboratory experiment is traditionally considered the method of choice for psychological science. One downside, however, is that as it carefully controls conditions and their effects, it can yield findings that are out of touch with reality and have limited use when trying to understand real-world behavior. This module highlights the importance of also conducting research outside the psychology laboratory, within participants’ natural, everyday environments, and reviews existing methodologies for studying daily life

Learning Objectives

• Identify limitations of the traditional laboratory experiment.
• Explain ways in which daily life research can further psychological science.
• Know what methods exist for conducting psychological research in the real world.

Introduction

Psychology can be applied in many ways, leading to many different career paths. Before we begin to introduce you to some of the many ways that psychology, and psychology research can be applied, it may be helpful to think about careers that can open to you as a result of studying psychology. The following video explores pathways for both learning about, and pursuing career options, related to psychology.

Applying Psychological Science

The laboratory experiment is traditionally considered the “gold standard” in psychology research. This is because only laboratory experiments can clearly separate cause from effect and therefore establish causality. Despite this unique strength, it is also clear that a scientific field that is mainly based on controlled laboratory studies ends up lopsided. Specifically, it accumulates a lot of knowledge on what can happen—under carefully isolated and controlled circumstances—but it has little to say about what actually does happen under the circumstances that people actually encounter in their daily lives.

For example, imagine you are a participant in an experiment that looks at the effect of being in a good mood on generosity, a topic that may have a good deal of practical application. Researchers create an internally-valid, carefully-controlled experiment where they randomly assign you to watch either a happy movie or a neutral movie, and then you are given the opportunity to help the researcher out by staying longer and participating in another study. If people in a good mood are more willing to stay and help out, the researchers can feel confident that – since everything else was held constant – your positive mood led you to be more helpful. However, what does this tell us about helping behaviors in the real world? Does it generalize to other kinds of helping, such as donating money to a charitable cause? Would all kinds of happy movies produce this behavior, or only this one? What about other positive experiences that might boost mood, like receiving a compliment or a good grade? And what if you were watching the movie with friends, in a crowded theatre, rather than in a sterile research lab? Taking research out into the real world can help answer some of these sorts of important questions.

As one of the founding fathers of social psychology remarked, “Experimentation in the laboratory occurs, socially speaking, on an island quite isolated from the life of society” ( Lewin, 1944 , p. 286). This module highlights the importance of going beyond experimentation and also conducting research outside the laboratory ( Reis & Gosling, 2010 ), directly within participants’ natural environments, and reviews existing methodologies for studying daily life.

Rationale for Conducting Psychology Research in the Real World

One important challenge researchers face when designing a study is to find the right balance between ensuring Internal Validity , or the degree to which a study allows unambiguous causal inferences, and External Validity , or the degree to which a study ensures that potential findings apply to settings and samples other than the ones being studied ( Brewer, 2000 ). Unfortunately, these two kinds of validity tend to be difficult to achieve at the same time, in one study. This is because creating a controlled setting, in which all potentially influential factors (other than the experimentally-manipulated variable) are controlled, is bound to create an environment that is quite different from what people naturally encounter (e.g., using a happy movie clip to promote helpful behavior). However, it is the degree to which an experimental situation is comparable to the corresponding real-world situation of interest that determines how generalizable potential findings will be. In other words, if an experiment is very far-off from what a person might normally experience in everyday life, you might reasonably question just how useful its findings are.

Because of the incompatibility of the two types of validity, one is often—by design—prioritized over the other. Due to the importance of identifying true causal relationships, psychology has traditionally emphasized internal over external validity. However, in order to make claims about human behavior that apply across populations and environments, researchers complement traditional laboratory research, where participants are brought into the lab, with field research where, in essence, the psychological laboratory is brought to participants. Field studies allow for the important test of how psychological variables and processes of interest “behave” under real-world circumstances (i.e., what actually does happen rather than what can happen ). They can also facilitate “downstream” operationalizations of constructs that measure life outcomes of interest directly rather than indirectly.

Take, for example, the fascinating field of psychoneuroimmunology, where the goal is to understand the interplay of psychological factors – such as personality traits or one’s stress level – and the immune system. Highly sophisticated and carefully controlled experiments offer ways to isolate the variety of neural, hormonal, and cellular mechanisms that link psychological variables such as chronic stress to biological outcomes such as immunosuppression (a state of impaired immune functioning; Sapolsky, 2004 ). Although these studies demonstrate impressively how psychological factors can affect health-relevant biological processes, they—because of their research design—remain mute about the degree to which these factors actually do undermine people’s everyday health in real life. It is certainly important to show that laboratory stress can alter the number of natural killer cells in the blood. But it is equally important to test to what extent the levels of stress that people experience on a day-to-day basis result in them catching a cold more often or taking longer to recover from one. The goal for researchers, therefore, must be to complement traditional laboratory experiments with less controlled studies under real-world circumstances. The term ecological validity is used to refer the degree to which an effect has been obtained under conditions that are typical for what happens in everyday life ( Brewer, 2000 ). In this example, then, people might keep a careful daily log of how much stress they are under as well as noting physical symptoms such as headaches or nausea. Although many factors beyond stress level may be responsible for these symptoms, this more correlational approach can shed light on how the relationship between stress and health plays out outside of the laboratory.

An Overview of Research Methods for Studying Daily Life

Capturing “life as it is lived” has been a strong goal for some researchers for a long time. Wilhelm and his colleagues recently published a comprehensive review of early attempts to systematically document daily life ( Wilhelm, Perrez, & Pawlik, 2012 ). Building onto these original methods, researchers have, over the past decades, developed a broad toolbox for measuring experiences, behavior, and physiology directly in participants’ daily lives ( Mehl & Conner, 2012 ). Figure 1 provides a schematic overview of the methodologies described below.

Studying Daily Experiences

Starting in the mid-1970s, motivated by a growing skepticism toward highly-controlled laboratory studies, a few groups of researchers developed a set of new methods that are now commonly known as the experience-sampling method ( Hektner, Schmidt, & Csikszentmihalyi, 2007 ), ecological momentary assessment ( Stone & Shiffman, 1994 ), or the diary method ( Bolger & Rafaeli, 2003 ). Although variations within this set of methods exist, the basic idea behind all of them is to collect in-the-moment (or, close-to-the-moment) self-report data directly from people as they go about their daily lives. This is typically accomplished by asking participants’ repeatedly (e.g., five times per day) over a period of time (e.g., a week) to report on their current thoughts and feelings. The momentary questionnaires often ask about their location (e.g., “Where are you now?”), social environment (e.g., “With whom are you now?”), activity (e.g., “What are you currently doing?”), and experiences (e.g., “How are you feeling?”). That way, researchers get a snapshot of what was going on in participants’ lives at the time at which they were asked to report.

Technology has made this sort of research possible, and recent technological advances have altered the different tools researchers are able to easily use. Initially, participants wore electronic wristwatches that beeped at preprogrammed but seemingly random times, at which they completed one of a stack of provided paper questionnaires. With the mobile computing revolution, both the prompting and the questionnaire completion were gradually replaced by handheld devices such as smartphones. Being able to collect the momentary questionnaires digitally and time-stamped (i.e., having a record of exactly when participants responded) had major methodological and practical advantages and contributed to experience sampling going mainstream ( Conner, Tennen, Fleeson, & Barrett, 2009 ).

Over time, experience sampling and related momentary self-report methods have become very popular, and, by now, they are effectively the gold standard for studying daily life. They have helped make progress in almost all areas of psychology ( Mehl & Conner, 2012 ). These methods ensure receiving many measurements from many participants, and has further inspired the development of novel statistical methods ( Bolger & Laurenceau, 2013 ). Finally, and maybe most importantly, they accomplished what they sought out to accomplish: to bring attention to what psychology ultimately wants and needs to know about, namely “what people actually do, think, and feel in the various contexts of their lives” ( Funder, 2001 , p. 213). In short, these approaches have allowed researchers to do research that is more externally valid, or more generalizable to real life, than the traditional laboratory experiment.

To illustrate these techniques, consider a classic study, Stone, Reed, and Neale ( 1987 ), who tracked positive and negative experiences surrounding a respiratory infection using daily experience sampling. They found that undesirable experiences peaked and desirable ones dipped about four to five days prior to participants coming down with the cold. More recently, Killingsworth and Gilbert ( 2010 ) collected momentary self-reports from more than 2,000 participants via a smartphone app. They found that participants were less happy when their mind was in an idling, mind-wandering state, such as surfing the Internet or multitasking at work, than when it was in an engaged, task-focused one, such as working diligently on a paper. These are just two examples that illustrate how experience-sampling studies have yielded findings that could not be obtained with traditional laboratory methods.

Recently, the day reconstruction method (DRM) ( Kahneman, Krueger, Schkade, Schwarz, & Stone, 2004 ) has been developed to obtain information about a person’s daily experiences without going through the burden of collecting momentary experience-sampling data. In the DRM, participants report their experiences of a given day retrospectively after engaging in a systematic, experiential reconstruction of the day on the following day. As a participant in this type of study, you might look back on yesterday, divide it up into a series of episodes such as “made breakfast,” “drove to work,” “had a meeting,” etc. You might then report who you were with in each episode and how you felt in each. This approach has shed light on what situations lead to moments of positive and negative mood throughout the course of a normal day.

Studying Daily Behavior

Experience sampling is often used to study everyday behavior (i.e., daily social interactions and activities). In the laboratory, behavior is best studied using direct behavioral observation (e.g., video recordings). In the real world, this is, of course, much more difficult. As Funder put it, it seems it would require a “detective’s report [that] would specify in exact detail everything the participant said and did, and with whom, in all of the contexts of the participant’s life” ( Funder, 2007 , p. 41).

As difficult as this may seem, Mehl and colleagues have developed a naturalistic observation methodology that is similar in spirit. Rather than following participants—like a detective—with a video camera (see Craik, 2000 ), they equip participants with a portable audio recorder that is programmed to periodically record brief snippets of ambient sounds (e.g., 30 seconds every 12 minutes). Participants carry the recorder (originally a microcassette recorder, now a smartphone app) on them as they go about their days and return it at the end of the study. The recorder provides researchers with a series of sound bites that, together, amount to an acoustic diary of participants’ days as they naturally unfold—and that constitute a representative sample of their daily activities and social encounters. Because it is somewhat similar to having the researcher’s ear at the participant’s lapel, they called their method the electronically activated recorder, or EAR ( Mehl, Pennebaker, Crow, Dabbs, & Price, 2001 ). The ambient sound recordings can be coded for many things, including participants’ locations (e.g., at school, in a coffee shop), activities (e.g., watching TV, eating), interactions (e.g., in a group, on the phone), and emotional expressions (e.g., laughing, sighing). As unnatural or intrusive as it might seem, participants report that they quickly grow accustomed to the EAR and say they soon find themselves behaving as they normally would.

In a cross-cultural study, Ramírez-Esparza and her colleagues used the EAR method to study sociability in the United States and Mexico. Interestingly, they found that although American participants rated themselves significantly higher than Mexicans on the question, “I see myself as a person who is talkative,” they actually spent almost 10 percent less time talking than Mexicans did ( Ramírez-Esparza, Mehl, Álvarez Bermúdez, & Pennebaker, 2009 ). In a similar way, Mehl and his colleagues used the EAR method to debunk the long-standing myth that women are considerably more talkative than men. Using data from six different studies, they showed that both sexes use on average about 16,000 words per day. The estimated sex difference of 546 words was trivial compared to the immense range of more than 46,000 words between the least and most talkative individual (695 versus 47,016 words; Mehl, Vazire, Ramírez-Esparza, Slatcher, & Pennebaker, 2007 ). Together, these studies demonstrate how naturalistic observation can be used to study objective aspects of daily behavior and how it can yield findings quite different from what other methods yield ( Mehl, Robbins, & Deters, 2012 ).

A series of other methods and creative ways for assessing behavior directly and unobtrusively in the real world are described in a seminal book on real-world, subtle measures ( Webb, Campbell, Schwartz, Sechrest, & Grove, 1981 ). For example, researchers have used time-lapse photography to study the flow of people and the use of space in urban public places ( Whyte, 1980 ). More recently, they have observed people’s personal (e.g., dorm rooms) and professional (e.g., offices) spaces to understand how personality is expressed and detected in everyday environments ( Gosling, Ko, Mannarelli, & Morris, 2002 ). They have even systematically collected and analyzed people’s garbage to measure what people actually consume (e.g., empty alcohol bottles or cigarette boxes) rather than what they say they consume ( Rathje & Murphy, 2001 ). Because people often cannot and sometimes may not want to accurately report what they do, the direct—and ideally nonreactive—assessment of real-world behavior is of high importance for psychological research ( Baumeister, Vohs, & Funder, 2007 ).

Studying Daily Physiology

In addition to studying how people think, feel, and behave in the real world, researchers are also interested in how our bodies respond to the fluctuating demands of our lives. What are the daily experiences that make our “blood boil”? How do our neurotransmitters and hormones respond to the stressors we encounter in our lives? What physiological reactions do we show to being loved—or getting ostracized? You can see how studying these powerful experiences in real life, as they actually happen, may provide more rich and informative data than one might obtain in an artificial laboratory setting that merely mimics these experiences.

Also, in pursuing these questions, it is important to keep in mind that what is stressful, engaging, or boring for one person might not be so for another. It is, in part, for this reason that researchers have found only limited correspondence between how people respond physiologically to a standardized laboratory stressor (e.g., giving a speech) and how they respond to stressful experiences in their lives. To give an example, Wilhelm and Grossman ( 2010 ) describe a participant who showed rather minimal heart rate increases in response to a laboratory stressor (about five to 10 beats per minute) but quite dramatic increases (almost 50 beats per minute) later in the afternoon while watching a soccer game. Of course, the reverse pattern can happen as well, such as when patients have high blood pressure in the doctor’s office but not in their home environment—the so-called white coat hypertension ( White, Schulman, McCabe, & Dey, 1989 ).

Ambulatory physiological monitoring – that is, monitoring physiological reactions as people go about their daily lives – has a long history in biomedical research and an array of monitoring devices exist ( Fahrenberg & Myrtek, 1996 ). Among the biological signals that can now be measured in daily life with portable signal recording devices are the electrocardiogram (ECG), blood pressure, electrodermal activity (or “sweat response”), body temperature, and even the electroencephalogram (EEG) ( Wilhelm & Grossman, 2010 ). Most recently, researchers have added ambulatory assessment of hormones (e.g., cortisol) and other biomarkers (e.g., immune markers) to the list ( Schlotz, 2012 ). The development of ever more sophisticated ways to track what goes on underneath our skins as we go about our lives is a fascinating and rapidly advancing field.

In a recent study, Lane, Zareba, Reis, Peterson, and Moss ( 2011 ) used experience sampling combined with ambulatory electrocardiography (a so-called Holter monitor) to study how emotional experiences can alter cardiac function in patients with a congenital heart abnormality (e.g., long QT syndrome). Consistent with the idea that emotions may, in some cases, be able to trigger a cardiac event, they found that typical—in most cases even relatively low intensity— daily emotions had a measurable effect on ventricular repolarization, an important cardiac indicator that, in these patients, is linked to risk of a cardiac event. In another study, Smyth and colleagues ( 1998 ) combined experience sampling with momentary assessment of cortisol, a stress hormone. They found that momentary reports of current or even anticipated stress predicted increased cortisol secretion 20 minutes later. Further, and independent of that, the experience of other kinds of negative affect (e.g., anger, frustration) also predicted higher levels of cortisol and the experience of positive affect (e.g., happy, joyful) predicted lower levels of this important stress hormone. Taken together, these studies illustrate how researchers can use ambulatory physiological monitoring to study how the little—and seemingly trivial or inconsequential—experiences in our lives leave objective, measurable traces in our bodily systems.

Studying Online Behavior

Another domain of daily life that has only recently emerged is virtual daily behavior or how people act and interact with others on the Internet. Irrespective of whether social media will turn out to be humanity’s blessing or curse (both scientists and laypeople are currently divided over this question), the fact is that people are spending an ever increasing amount of time online. In light of that, researchers are beginning to think of virtual behavior as being as serious as “actual” behavior and seek to make it a legitimate target of their investigations ( Gosling & Johnson, 2010 ).

One way to study virtual behavior is to make use of the fact that most of what people do on the Web—emailing, chatting, tweeting, blogging, posting— leaves direct (and permanent) verbal traces. For example, differences in the ways in which people use words (e.g., subtle preferences in word choice) have been found to carry a lot of psychological information ( Pennebaker, Mehl, & Niederhoffer, 2003 ). Therefore, a good way to study virtual social behavior is to study virtual language behavior. Researchers can download people’s—often public—verbal expressions and communications and analyze them using modern text analysis programs (e.g., Pennebaker, Francis, & Booth, 2001 ).

For example, Cohn, Mehl, and Pennebaker ( 2004 ) downloaded blogs of more than a thousand users of lifejournal.com, one of the first Internet blogging sites, to study how people responded socially and emotionally to the attacks of September 11, 2001. In going “the online route,” they could bypass a critical limitation of coping research, the inability to obtain baseline information; that is, how people were doing before the traumatic event occurred. Through access to the database of public blogs, they downloaded entries from two months prior to two months after the attacks. Their linguistic analyses revealed that in the first days after the attacks, participants expectedly expressed more negative emotions and were more cognitively and socially engaged, asking questions and sending messages of support. Already after two weeks, though, their moods and social engagement returned to baseline, and, interestingly, their use of cognitive-analytic words (e.g., “think,” “question”) even dropped below their normal level. Over the next six weeks, their mood hovered around their pre-9/11 baseline, but both their social engagement and cognitive-analytic processing stayed remarkably low. This suggests a social and cognitive weariness in the aftermath of the attacks. In using virtual verbal behavior as a marker of psychological functioning, this study was able to draw a fine timeline of how humans cope with disasters.

Reflecting their rapidly growing real-world importance, researchers are now beginning to investigate behavior on social networking sites such as Facebook ( Wilson, Gosling, & Graham, 2012 ). Most research looks at psychological correlates of online behavior such as personality traits and the quality of one’s social life but, importantly, there are also first attempts to export traditional experimental research designs into an online setting. In a pioneering study of online social influence, Bond and colleagues ( 2012 ) experimentally tested the effects that peer feedback has on voting behavior. Remarkably, their sample consisted of 16 million (!) Facebook users. They found that online political-mobilization messages (e.g., “I voted” accompanied by selected pictures of their Facebook friends) influenced real-world voting behavior. This was true not just for users who saw the messages but also for their friends and friends of their friends. Although the intervention effect on a single user was very small, through the enormous number of users and indirect social contagion effects, it resulted cumulatively in an estimated 340,000 additional votes—enough to tilt a close election. In short, although still in its infancy, research on virtual daily behavior is bound to change social science, and it has already helped us better understand both virtual and “actual” behavior.

“Smartphone Psychology”?

A review of research methods for studying daily life would not be complete without a vision of “what’s next.” Given how common they have become, it is safe to predict that smartphones will not just remain devices for everyday online communication but will also become devices for scientific data collection and intervention ( Kaplan & Stone, 2013 ; Yarkoni, 2012 ). These devices automatically store vast amounts of real-world user interaction data, and, in addition, they are equipped with sensors to track the physical (e. g., location, position) and social (e.g., wireless connections around the phone) context of these interactions. Miller ( 2012 , p. 234) states, “The question is not whether smartphones will revolutionize psychology but how, when, and where the revolution will happen.” Obviously, their immense potential for data collection also brings with it big new challenges for researchers (e.g., privacy protection, data analysis, and synthesis). Yet it is clear that many of the methods described in this module—and many still to be developed ways of collecting real-world data—will, in the future, become integrated into the devices that people naturally and happily carry with them from the moment they get up in the morning to the moment they go to bed.

Psychology and Climate Change

Climate change is an important topic, and one where psychology is making a big impact. Learn more in the following interactive resource:

This module sought to make a case for psychology research conducted outside the lab. If the ultimate goal of the social and behavioral sciences is to explain human behavior, then researchers must also—in addition to conducting carefully controlled lab studies—deal with the “messy” real world and find ways to capture life as it naturally happens.

Mortensen and Cialdini ( 2010 ) refer to the dynamic give-and-take between laboratory and field research as “ full-cycle psychology ”. Going full cycle, they suggest, means that “researchers use naturalistic observation to determine an effect’s presence in the real world, theory to determine what processes underlie the effect, experimentation to verify the effect and its underlying processes, and a return to the natural environment to corroborate the experimental findings” ( Mortensen & Cialdini, 2010, p. 53 ). To accomplish this, researchers have access to a toolbox of research methods for studying daily life that is now more diverse and more versatile than it has ever been before. So, all it takes is to go ahead and—literally—bring science to life.

Check Your Knowledge

To help you with your studying, we’ve included some practice questions for this module. These questions do not necessarily address all content in this module. They are intended as practice, and you are responsible for all of the content in this module even if there is no associated practice question. To promote deeper engagement with the material, we encourage you to create some questions of your own for your practice. You can then also return to these self-generated questions later in the course to test yourself.

Ambulatory assessment An overarching term to describe methodologies that assess the behavior, physiology, experience, and environments of humans in naturalistic settings.

Daily Diary method A methodology where participants complete a questionnaire about their thoughts, feelings, and behavior of the day at the end of the day.

Day reconstruction method (DRM) A methodology where participants describe their experiences and behavior of a given day retrospectively upon a systematic reconstruction on the following day.

Ecological momentary assessment An overarching term to describe methodologies that repeatedly sample participants’ real-world experiences, behavior, and physiology in real time.

Ecological validity The degree to which a study finding has been obtained under conditions that are typical for what happens in everyday life.

Electronically activated recorder, or EAR A methodology where participants wear a small, portable audio recorder that intermittently records snippets of ambient sounds around them.

Experience-sampling method A methodology where participants report on their momentary thoughts, feelings, and behaviors at different points in time over the course of a day.

External validity The degree to which a finding generalizes from the specific sample and context of a study to some larger population and broader settings.

Full-cycle psychology A scientific approach whereby researchers start with an observational field study to identify an effect in the real world, follow up with laboratory experimentation to verify the effect and isolate the causal mechanisms, and return to field research to corroborate their experimental findings.

Generalize Generalizing, in science, refers to the ability to arrive at broad conclusions based on a smaller sample of observations. For these conclusions to be true the sample should accurately represent the larger population from which it is drawn.

Internal validity The degree to which a cause-effect relationship between two variables has been unambiguously established.

Linguistic inquiry and word count A quantitative text analysis methodology that automatically extracts grammatical and psychological information from a text by counting word frequencies.

Lived day analysis A methodology where a research team follows an individual around with a video camera to objectively document a person’s daily life as it is lived.

White coat hypertension A phenomenon in which patients exhibit elevated blood pressure in the hospital or doctor’s office but not in their everyday lives.

• Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2 , 396–403.
• Bolger, N., & Laurenceau, J-P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research . New York, NY: Guilford Press.

Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616.

• Bond, R. M., Jones, J. J., Kramer, A. D., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61 million-person experiment in social influence and political mobilization. Nature , 489, 295–298.

Brewer, M. B. (2000). Research design and issues of validity. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology (pp. 3–16). New York, NY: Cambridge University Press.

Cohn, M. A., Mehl, M. R., & Pennebaker, J. W. (2004). Linguistic indicators of psychological change after September 11, 2001. Psychological Science, 15, 687–693.

• Conner, T. S., Tennen, H., Fleeson, W., & Barrett, L. F. (2009). Experience sampling methods: A modern idiographic approach to personality research. Social and Personality Psychology Compass, 3 , 292–313.

Craik, K. H. (2000). The lived day of an individual: A person-environment perspective. In W. B. Walsh, K. H. Craik, & R. H. Price (Eds.), Person-environment psychology: New directions and perspectives (pp. 233–266). Mahwah, NJ: Lawrence Erlbaum Associates.

• Fahrenberg, J., &. Myrtek, M. (Eds.) (1996). Ambulatory assessment: Computer-assisted psychological and psychophysiological methods in monitoring and field studies . Seattle, WA: Hogrefe & Huber.
• Funder, D. C. (2007). The personality puzzle . New York, NY: W. W. Norton & Co.
• Funder, D. C. (2001). Personality. Review of Psychology, 52, 197–221.
• Gosling, S. D., & Johnson, J. A. (2010). Advanced methods for conducting online behavioral research . Washington, DC: American Psychological Association.
• Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: Personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82 , 379–398.
• Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life . Thousand Oaks, CA: Sage.
• Kahneman, D., Krueger, A., Schkade, D., Schwarz, N., and Stone, A. (2004). A survey method for characterizing daily life experience: The Day Reconstruction Method. Science , 306, 1776–780.
• Kaplan, R. M., & Stone A. A. (2013). Bringing the laboratory and clinic to the community: Mobile technologies for health promotion and disease prevention. Annual Review of Psychology, 64 , 471-498.
• Killingsworth, M. A., & Gilbert, D. T. (2010). A wandering mind is an unhappy mind. Science , 330, 932.
• Lane, R. D., Zareba, W., Reis, H., Peterson, D., &, Moss, A. (2011). Changes in ventricular repolarization duration during typical daily emotion in patients with Long QT Syndrome. Psychosomatic Medicine, 73 , 98–105.
• Lewin, K. (1944) Constructs in psychology and psychological ecology . University of Iowa Studies in Child Welfare, 20, 23–27.
• Mehl, M. R., & Conner, T. S. (Eds.) (2012). Handbook of research methods for studying daily life . New York, NY: Guilford Press.
• Mehl, M. R., Pennebaker, J. W., Crow, M., Dabbs, J., & Price, J. (2001). The electronically activated recorder (EAR): A device for sampling naturalistic daily activities and conversations. Behavior Research Methods, Instruments, and Computers, 33 , 517–523.
• Mehl, M. R., Robbins, M. L., & Deters, G. F. (2012). Naturalistic observation of health-relevant social processes: The electronically activated recorder (EAR) methodology in psychosomatics. Psychosomatic Medicine, 74 , 410–417.
• Mehl, M. R., Vazire, S., Ramírez-Esparza, N., Slatcher, R. B., & Pennebaker, J. W. (2007). Are women really more talkative than men? Science, 317 , 82.
• Miller, G. (2012). The smartphone psychology manifesto. Perspectives in Psychological Science , 7, 221–237.

Mortenson, C. R., & Cialdini, R. B. (2010). Full-cycle social psychology for theory and application. Social and Personality Psychology Compass, 4, 53–63.

• Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word count: LIWC 2001.  Mahway: Lawrence Erlbaum Associates ,  71, 1-22 .
• Pennebaker, J. W., Mehl, M. R., Niederhoffer, K. (2003). Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology, 54 , 547–577.
• Ramírez-Esparza, N., Mehl, M. R., Álvarez Bermúdez, J., & Pennebaker, J. W. (2009). Are Mexicans more or less sociable than Americans? Insights from a naturalistic observation study. Journal of Research in Personality, 43 , 1–7.
• Rathje, W., & Murphy, C. (2001). Rubbish! The archaeology of garbage . New York, NY: Harper Collins.

Reis, H. T., & Gosling, S. D. (2010). Social psychological methods outside the laboratory. In S. T. Fiske, D. T. Gilbert, & G. Lindzey, (Eds.), Handbook of social psychology (5th ed., Vol. 1, pp. 82–114). New York, NY: Wiley.

• Sapolsky, R. (2004). Why zebras don’t get ulcers: A guide to stress, stress-related diseases and coping . New York, NY: Henry Holt and Co.

Schlotz, W. (2012). Ambulatory psychoneuroendocrinology: Assessing salivary cortisol and other hormones in daily life. In M.R. Mehl & T.S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 193–209). New York, NY: Guilford Press.

• Smyth, J., Ockenfels, M. C., Porter, L., Kirschbaum, C., Hellhammer, D. H., & Stone, A. A. (1998). Stressors and mood measured on a momentary basis are associated with salivary cortisol secretion. Psychoneuroendocrinology, 23 , 353–370.
• Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16 , 199–202.
• Stone, A. A., Reed, B. R., Neale, J. M. (1987). Changes in daily event frequency precede episodes of physical symptoms. Journal of Human Stress, 13 , 70–74.
• Webb, E. J., Campbell, D. T., Schwartz, R. D., Sechrest, L., & Grove, J. B. (1981). Nonreactive measures in the social sciences . Boston, MA: Houghton Mifflin Co.
• White, W. B., Schulman, P., McCabe, E. J., & Dey, H. M. (1989). Average daily blood pressure, not office blood pressure, determines cardiac function in patients with hypertension. Journal of the American Medical Association, 261 , 873–877.
• Whyte, W. H. (1980). The social life of small urban spaces . Washington, DC: The Conservation Foundation.
• Wilhelm, F.H., & Grossman, P. (2010). Emotions beyond the laboratory: Theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment. Biological Psychology, 84 , 552–569.
• Wilhelm, P., Perrez, M., & Pawlik, K. (2012). Conducting research in daily life: A historical review. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life . New York, NY: Guilford Press.
• Wilson, R., & Gosling, S. D., & Graham, L. (2012). A review of Facebook research in the social sciences. Perspectives on Psychological Science, 7 , 203–220.
• Yarkoni, T. (2012). Psychoinformatics: New horizons at the interface of the psychological and computing sciences. Current Directions in Psychological Science, 21 , 391–397.

How to cite this Chapter using APA Style:

Mehl, M. R. (2019). Conducting psychology research in the real world. Adapted for use by Queen’s University. Original chapter in R. Biswas-Diener & E. Diener (Eds), Noba textbook series: Psychology.  Champaign, IL: DEF publishers. Retrieved from  http://noba.to/hsfe5k3d

Copyright and Acknowledgment:

This material is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit:   http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_US .

This material is attributed to the Diener Education Fund (copyright © 2018) and can be accessed via this link: http://noba.to/hsfe5k3d.

Additional information about the Diener Education Fund (DEF) can be accessed here .

In research, the degree to which one can extend conclusions drawn from the findings of a study to other groups or situations not included in the study.

Lewin, K. (1944) Constructs in psychology and psychological ecology. University of Iowa Studies in Child Welfare, 20, 23–27.

The degree to which a cause-effect relationship between two variables has been unambiguously established.

The degree to which a finding generalizes from the specific sample and context of a study to some larger population and broader settings.

Sapolsky, R. (2004). Why zebras don’t get ulcers: A guide to stress, stress-related diseases and coping. New York, NY: Henry Holt and Co.

The degree to which a study finding has been obtained under conditions that are typical for what happens in everyday life.

Wilhelm, P., Perrez, M., & Pawlik, K. (2012). Conducting research in daily life: A historical review. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life. New York, NY: Guilford Press.

Mehl, M. R., & Conner, T. S. (Eds.) (2012). Handbook of research methods for studying daily life. New York, NY: Guilford Press.

A methodology where participants report on their momentary thoughts, feelings, and behaviors at different points in time over the course of a day.

Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life. Thousand Oaks, CA: Sage.

An overarching term to describe methodologies that repeatedly sample participants’ real-world experiences, behavior, and physiology in real time.

Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16, 199–202.

A methodology where participants complete a questionnaire about their thoughts, feelings, and behavior of the day at the end of the day.

Conner, T. S., Tennen, H., Fleeson, W., & Barrett, L. F. (2009). Experience sampling methods: A modern idiographic approach to personality research. Social and Personality Psychology Compass, 3, 292–313.

Bolger, N., & Laurenceau, J-P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research. New York, NY: Guilford Press.

Funder, D. C. (2001). Personality. Review of Psychology, 52, 197–221.Gosling, S. D., & Johnson, J. A. (2010). Advanced methods for conducting online behavioral research. Washington, DC: American Psychological Association.

Stone, A. A., Reed, B. R., Neale, J. M. (1987). Changes in daily event frequency precede episodes of physical symptoms. Journal of Human Stress, 13, 70–74.

Killingsworth, M. A., & Gilbert, D. T. (2010). A wandering mind is an unhappy mind. Science, 330, 932.

A methodology where participants describe their experiences and behavior of a given day retrospectively upon a systematic reconstruction on the following day.

Kahneman, D., Krueger, A., Schkade, D., Schwarz, N., and Stone, A. (2004). A survey method for characterizing daily life experience: The Day Reconstruction Method. Science, 306, 1776–780.

Funder, D. C. (2007). The personality puzzle. New York, NY: W. W. Norton & Co.

A methodology where participants wear a small, portable audio recorder that intermittently records snippets of ambient sounds around them.

Mehl, M. R., Pennebaker, J. W., Crow, M., Dabbs, J., & Price, J. (2001). The electronically activated recorder (EAR): A device for sampling naturalistic daily activities and conversations. Behavior Research Methods, Instruments, and Computers, 33, 517–523.

Ramírez-Esparza, N., Mehl, M. R., Álvarez Bermúdez, J., & Pennebaker, J. W. (2009). Are Mexicans more or less sociable than Americans? Insights from a naturalistic observation study. Journal of Research in Personality, 43, 1–7.

Mehl, M. R., Vazire, S., Ramírez-Esparza, N., Slatcher, R. B., & Pennebaker, J. W. (2007). Are women really more talkative than men? Science, 317, 82.

Mehl, M. R., Robbins, M. L., & Deters, G. F. (2012). Naturalistic observation of health-relevant social processes: The electronically activated recorder (EAR) methodology in psychosomatics. Psychosomatic Medicine, 74, 410–417.

Webb, E. J., Campbell, D. T., Schwartz, R. D., Sechrest, L., & Grove, J. B. (1981). Nonreactive measures in the social sciences. Boston, MA: Houghton Mifflin Co.

Whyte, W. H. (1980). The social life of small urban spaces. Washington, DC: The Conservation Foundation.

Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: Personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82, 379–398.

Rathje, W., & Murphy, C. (2001). Rubbish! The archaeology of garbage. New York, NY: Harper Collins.

Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2, 396–403.

Wilhelm, F.H., & Grossman, P. (2010). Emotions beyond the laboratory: Theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment. Biological Psychology, 84, 552–569.

A phenomenon in which patients exhibit elevated blood pressure in the hospital or doctor’s office but not in their everyday lives.

White, W. B., Schulman, P., McCabe, E. J., & Dey, H. M. (1989). Average daily blood pressure, not office blood pressure, determines cardiac function in patients with hypertension. Journal of the American Medical Association, 261, 873–877.

Fahrenberg, J., &. Myrtek, M. (Eds.) (1996). Ambulatory assessment: Computer-assisted psychological and psychophysiological methods in monitoring and field studies. Seattle, WA: Hogrefe & Huber.

An overarching term to describe methodologies that assess the behavior, physiology, experience, and environments of humans in naturalistic settings.

Lane, R. D., Zareba, W., Reis, H., Peterson, D., &, Moss, A. (2011). Changes in ventricular repolarization duration during typical daily emotion in patients with Long QT Syndrome. Psychosomatic Medicine, 73, 98–105.

Smyth, J., Ockenfels, M. C., Porter, L., Kirschbaum, C., Hellhammer, D. H., & Stone, A. A. (1998). Stressors and mood measured on a momentary basis are associated with salivary cortisol secretion. Psychoneuroendocrinology, 23, 353–370.

Gosling, S. D., & Johnson, J. A. (2010). Advanced methods for conducting online behavioral research. Washington, DC: American Psychological Association.

Pennebaker, J. W., Mehl, M. R., Niederhoffer, K. (2003). Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology, 54, 547–577.

Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates, 71, 1-22.

A quantitative text analysis methodology that automatically extracts grammatical and psychological information from a text by counting word frequencies.

Wilson, R., & Gosling, S. D., & Graham, L. (2012). A review of Facebook research in the social sciences. Perspectives on Psychological Science, 7, 203–220.

Bond, R. M., Jones, J. J., Kramer, A. D., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61 million-person experiment in social influence and political mobilization. Nature, 489, 295–298.

Kaplan, R. M., & Stone A. A. (2013). Bringing the laboratory and clinic to the community: Mobile technologies for health promotion and disease prevention. Annual Review of Psychology, 64, 471-498.

Yarkoni, T. (2012). Psychoinformatics: New horizons at the interface of the psychological and computing sciences. Current Directions in Psychological Science, 21, 391–397.

Miller, G. (2012). The smartphone psychology manifesto. Perspectives in Psychological Science, 7, 221–237.

A scientific approach whereby researchers start with an observational field study to identify an effect in the real world, follow up with laboratory experimentation to verify the effect and isolate the causal mechanisms, and return to field research to corroborate their experimental findings.

PSYC 100: Principles of Psychology F23 Copyright © by PSYC100 is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• Together: The Science of Social Psychology

Conducting Psychology Research in the Real World

University of Arizona

Because of its ability to determine cause-and-effect relationships, the laboratory experiment is traditionally considered the method of choice for psychological science. One downside, however, is that as it carefully controls conditions and their effects, it can yield findings that are out of touch with reality and have limited use when trying to understand real-world behavior. This module highlights the importance of also conducting research outside the psychology laboratory, within participants’ natural, everyday environments, and reviews existing methodologies for studying daily life

• Ambulatory assessment
• Diary methods
• Ecological momentary assessment
• Experience sampling method
• Field Research
• Internet methodologies
• Learning Objectives
• Identify limitations of the traditional laboratory experiment.
• Explain ways in which daily life research can further psychological science.
• Know what methods exist for conducting psychological research in the real world.

Introduction

The laboratory experiment is traditionally considered the “gold standard” in psychology research. This is because only laboratory experiments can clearly separate cause from effect and therefore establish causality. Despite this unique strength, it is also clear that a scientific field that is mainly based on controlled laboratory studies ends up lopsided. Specifically, it accumulates a lot of knowledge on what can happen—under carefully isolated and controlled circumstances—but it has little to say about what actually does happen under the circumstances that people actually encounter in their daily lives.

For example, imagine you are a participant in an experiment that looks at the effect of being in a good mood on generosity, a topic that may have a good deal of practical application. Researchers create an internally-valid, carefully-controlled experiment where they randomly assign you to watch either a happy movie or a neutral movie, and then you are given the opportunity to help the researcher out by staying longer and participating in another study. If people in a good mood are more willing to stay and help out, the researchers can feel confident that – since everything else was held constant – your positive mood led you to be more helpful. However, what does this tell us about helping behaviors in the real world? Does it generalize to other kinds of helping, such as donating money to a charitable cause? Would all kinds of happy movies produce this behavior, or only this one? What about other positive experiences that might boost mood, like receiving a compliment or a good grade? And what if you were watching the movie with friends, in a crowded theatre, rather than in a sterile research lab? Taking research out into the real world can help answer some of these sorts of important questions.

As one of the founding fathers of social psychology remarked, “Experimentation in the laboratory occurs, socially speaking, on an island quite isolated from the life of society” ( Lewin, 1944 , p. 286). This module highlights the importance of going beyond experimentation and also conducting research outside the laboratory ( Reis & Gosling, 2010 ), directly within participants’ natural environments, and reviews existing methodologies for studying daily life.

Rationale for Conducting Psychology Research in the Real World

One important challenge researchers face when designing a study is to find the right balance between ensuring internal validity , or the degree to which a study allows unambiguous causal inferences, and external validity , or the degree to which a study ensures that potential findings apply to settings and samples other than the ones being studied ( Brewer, 2000 ). Unfortunately, these two kinds of validity tend to be difficult to achieve at the same time, in one study. This is because creating a controlled setting, in which all potentially influential factors (other than the experimentally-manipulated variable) are controlled, is bound to create an environment that is quite different from what people naturally encounter (e.g., using a happy movie clip to promote helpful behavior). However, it is the degree to which an experimental situation is comparable to the corresponding real-world situation of interest that determines how generalizable potential findings will be. In other words, if an experiment is very far-off from what a person might normally experience in everyday life, you might reasonably question just how useful its findings are.

Because of the incompatibility of the two types of validity, one is often—by design—prioritized over the other. Due to the importance of identifying true causal relationships, psychology has traditionally emphasized internal over external validity. However, in order to make claims about human behavior that apply across populations and environments, researchers complement traditional laboratory research, where participants are brought into the lab, with field research where, in essence, the psychological laboratory is brought to participants. Field studies allow for the important test of how psychological variables and processes of interest “behave” under real-world circumstances (i.e., what actually does happen rather than what can happen ). They can also facilitate “downstream” operationalizations of constructs that measure life outcomes of interest directly rather than indirectly.

Take, for example, the fascinating field of psychoneuroimmunology, where the goal is to understand the interplay of psychological factors - such as personality traits or one’s stress level - and the immune system. Highly sophisticated and carefully controlled experiments offer ways to isolate the variety of neural, hormonal, and cellular mechanisms that link psychological variables such as chronic stress to biological outcomes such as immunosuppression (a state of impaired immune functioning; Sapolsky, 2004 ). Although these studies demonstrate impressively how psychological factors can affect health-relevant biological processes, they—because of their research design—remain mute about the degree to which these factors actually do undermine people’s everyday health in real life. It is certainly important to show that laboratory stress can alter the number of natural killer cells in the blood. But it is equally important to test to what extent the levels of stress that people experience on a day-to-day basis result in them catching a cold more often or taking longer to recover from one. The goal for researchers, therefore, must be to complement traditional laboratory experiments with less controlled studies under real-world circumstances. The term ecological validity is used to refer the degree to which an effect has been obtained under conditions that are typical for what happens in everyday life (Brewer, 2000). In this example, then, people might keep a careful daily log of how much stress they are under as well as noting physical symptoms such as headaches or nausea. Although many factors beyond stress level may be responsible for these symptoms, this more correlational approach can shed light on how the relationship between stress and health plays out outside of the laboratory.

An Overview of Research Methods for Studying Daily Life

Capturing “life as it is lived” has been a strong goal for some researchers for a long time. Wilhelm and his colleagues recently published a comprehensive review of early attempts to systematically document daily life ( Wilhelm, Perrez, & Pawlik, 2012 ). Building onto these original methods, researchers have, over the past decades, developed a broad toolbox for measuring experiences, behavior, and physiology directly in participants’ daily lives ( Mehl & Conner, 2012 ). Figure 1 provides a schematic overview of the methodologies described below.

Studying Daily Experiences

Starting in the mid-1970s, motivated by a growing skepticism toward highly-controlled laboratory studies, a few groups of researchers developed a set of new methods that are now commonly known as the experience-sampling method ( Hektner, Schmidt, & Csikszentmihalyi, 2007 ), ecological momentary assessment ( Stone & Shiffman, 1994 ), or the diary method ( Bolger & Rafaeli, 2003 ). Although variations within this set of methods exist, the basic idea behind all of them is to collect in-the-moment (or, close-to-the-moment) self-report data directly from people as they go about their daily lives. This is typically accomplished by asking participants’ repeatedly (e.g., five times per day) over a period of time (e.g., a week) to report on their current thoughts and feelings. The momentary questionnaires often ask about their location (e.g., “Where are you now?”), social environment (e.g., “With whom are you now?”), activity (e.g., “What are you currently doing?”), and experiences (e.g., “How are you feeling?”). That way, researchers get a snapshot of what was going on in participants’ lives at the time at which they were asked to report.

Technology has made this sort of research possible, and recent technological advances have altered the different tools researchers are able to easily use. Initially, participants wore electronic wristwatches that beeped at preprogrammed but seemingly random times, at which they completed one of a stack of provided paper questionnaires. With the mobile computing revolution, both the prompting and the questionnaire completion were gradually replaced by handheld devices such as smartphones. Being able to collect the momentary questionnaires digitally and time-stamped (i.e., having a record of exactly when participants responded) had major methodological and practical advantages and contributed to experience sampling going mainstream ( Conner, Tennen, Fleeson, & Barrett, 2009 ).

Over time, experience sampling and related momentary self-report methods have become very popular, and, by now, they are effectively the gold standard for studying daily life. They have helped make progress in almost all areas of psychology (Mehl & Conner, 2012). These methods ensure receiving many measurements from many participants, and has further inspired the development of novel statistical methods ( Bolger & Laurenceau, 2013 ). Finally, and maybe most importantly, they accomplished what they sought out to accomplish: to bring attention to what psychology ultimately wants and needs to know about, namely “what people actually do, think, and feel in the various contexts of their lives” ( Funder, 2001 , p. 213). In short, these approaches have allowed researchers to do research that is more externally valid, or more generalizable to real life, than the traditional laboratory experiment.

To illustrate these techniques, consider a classic study, Stone, Reed, and Neale ( 1987 ), who tracked positive and negative experiences surrounding a respiratory infection using daily experience sampling. They found that undesirable experiences peaked and desirable ones dipped about four to five days prior to participants coming down with the cold. More recently, Killingsworth and Gilbert ( 2010 ) collected momentary self-reports from more than 2,000 participants via a smartphone app. They found that participants were less happy when their mind was in an idling, mind-wandering state, such as surfing the Internet or multitasking at work, than when it was in an engaged, task-focused one, such as working diligently on a paper. These are just two examples that illustrate how experience-sampling studies have yielded findings that could not be obtained with traditional laboratory methods.

Recently, the day reconstruction method (DRM) ( Kahneman, Krueger, Schkade, Schwarz, & Stone, 2004 ) has been developed to obtain information about a person’s daily experiences without going through the burden of collecting momentary experience-sampling data. In the DRM, participants report their experiences of a given day retrospectively after engaging in a systematic, experiential reconstruction of the day on the following day. As a participant in this type of study, you might look back on yesterday, divide it up into a series of episodes such as “made breakfast,” “drove to work,” “had a meeting,” etc. You might then report who you were with in each episode and how you felt in each. This approach has shed light on what situations lead to moments of positive and negative mood throughout the course of a normal day.

Studying Daily Behavior

Experience sampling is often used to study everyday behavior (i.e., daily social interactions and activities). In the laboratory, behavior is best studied using direct behavioral observation (e.g., video recordings). In the real world, this is, of course, much more difficult. As Funder put it, it seems it would require a “detective’s report [that] would specify in exact detail everything the participant said and did, and with whom, in all of the contexts of the participant’s life” ( Funder, 2007 , p. 41).

As difficult as this may seem, Mehl and colleagues have developed a naturalistic observation methodology that is similar in spirit. Rather than following participants—like a detective—with a video camera (see Craik, 2000 ), they equip participants with a portable audio recorder that is programmed to periodically record brief snippets of ambient sounds (e.g., 30 seconds every 12 minutes). Participants carry the recorder (originally a microcassette recorder, now a smartphone app) on them as they go about their days and return it at the end of the study. The recorder provides researchers with a series of sound bites that, together, amount to an acoustic diary of participants’ days as they naturally unfold—and that constitute a representative sample of their daily activities and social encounters. Because it is somewhat similar to having the researcher’s ear at the participant’s lapel, they called their method the electronically activated recorder, or EAR ( Mehl, Pennebaker, Crow, Dabbs, & Price, 2001 ). The ambient sound recordings can be coded for many things, including participants’ locations (e.g., at school, in a coffee shop), activities (e.g., watching TV, eating), interactions (e.g., in a group, on the phone), and emotional expressions (e.g., laughing, sighing). As unnatural or intrusive as it might seem, participants report that they quickly grow accustomed to the EAR and say they soon find themselves behaving as they normally would.

In a cross-cultural study, Ramírez-Esparza and her colleagues used the EAR method to study sociability in the United States and Mexico. Interestingly, they found that although American participants rated themselves significantly higher than Mexicans on the question, “I see myself as a person who is talkative,” they actually spent almost 10 percent less time talking than Mexicans did ( Ramírez-Esparza, Mehl, Álvarez Bermúdez, & Pennebaker, 2009 ). In a similar way, Mehl and his colleagues used the EAR method to debunk the long-standing myth that women are considerably more talkative than men. Using data from six different studies, they showed that both sexes use on average about 16,000 words per day. The estimated sex difference of 546 words was trivial compared to the immense range of more than 46,000 words between the least and most talkative individual (695 versus 47,016 words; Mehl, Vazire, Ramírez-Esparza, Slatcher, & Pennebaker, 2007 ). Together, these studies demonstrate how naturalistic observation can be used to study objective aspects of daily behavior and how it can yield findings quite different from what other methods yield ( Mehl, Robbins, & Deters, 2012 ).

A series of other methods and creative ways for assessing behavior directly and unobtrusively in the real world are described in a seminal book on real-world, subtle measures ( Webb, Campbell, Schwartz, Sechrest, & Grove, 1981 ). For example, researchers have used time-lapse photography to study the flow of people and the use of space in urban public places ( Whyte, 1980 ). More recently, they have observed people’s personal (e.g., dorm rooms) and professional (e.g., offices) spaces to understand how personality is expressed and detected in everyday environments ( Gosling, Ko, Mannarelli, & Morris, 2002 ). They have even systematically collected and analyzed people’s garbage to measure what people actually consume (e.g., empty alcohol bottles or cigarette boxes) rather than what they say they consume ( Rathje & Murphy, 2001 ). Because people often cannot and sometimes may not want to accurately report what they do, the direct—and ideally nonreactive—assessment of real-world behavior is of high importance for psychological research ( Baumeister, Vohs, & Funder, 2007 ).

Studying Daily Physiology

In addition to studying how people think, feel, and behave in the real world, researchers are also interested in how our bodies respond to the fluctuating demands of our lives. What are the daily experiences that make our “blood boil”? How do our neurotransmitters and hormones respond to the stressors we encounter in our lives? What physiological reactions do we show to being loved—or getting ostracized? You can see how studying these powerful experiences in real life, as they actually happen, may provide more rich and informative data than one might obtain in an artificial laboratory setting that merely mimics these experiences.

Also, in pursuing these questions, it is important to keep in mind that what is stressful, engaging, or boring for one person might not be so for another. It is, in part, for this reason that researchers have found only limited correspondence between how people respond physiologically to a standardized laboratory stressor (e.g., giving a speech) and how they respond to stressful experiences in their lives. To give an example, Wilhelm and Grossman ( 2010 ) describe a participant who showed rather minimal heart rate increases in response to a laboratory stressor (about five to 10 beats per minute) but quite dramatic increases (almost 50 beats per minute) later in the afternoon while watching a soccer game. Of course, the reverse pattern can happen as well, such as when patients have high blood pressure in the doctor’s office but not in their home environment—the so-called white coat hypertension ( White, Schulman, McCabe, & Dey, 1989 ).

Ambulatory physiological monitoring – that is, monitoring physiological reactions as people go about their daily lives - has a long history in biomedical research and an array of monitoring devices exist ( Fahrenberg & Myrtek, 1996 ). Among the biological signals that can now be measured in daily life with portable signal recording devices are the electrocardiogram (ECG), blood pressure, electrodermal activity (or “sweat response”), body temperature, and even the electroencephalogram (EEG) (Wilhelm & Grossman, 2010). Most recently, researchers have added ambulatory assessment of hormones (e.g., cortisol) and other biomarkers (e.g., immune markers) to the list ( Schlotz, 2012 ). The development of ever more sophisticated ways to track what goes on underneath our skins as we go about our lives is a fascinating and rapidly advancing field.

In a recent study, Lane, Zareba, Reis, Peterson, and Moss ( 2011 ) used experience sampling combined with ambulatory electrocardiography (a so-called Holter monitor) to study how emotional experiences can alter cardiac function in patients with a congenital heart abnormality (e.g., long QT syndrome). Consistent with the idea that emotions may, in some cases, be able to trigger a cardiac event, they found that typical—in most cases even relatively low intensity— daily emotions had a measurable effect on ventricular repolarization, an important cardiac indicator that, in these patients, is linked to risk of a cardiac event. In another study, Smyth and colleagues ( 1998 ) combined experience sampling with momentary assessment of cortisol, a stress hormone. They found that momentary reports of current or even anticipated stress predicted increased cortisol secretion 20 minutes later. Further, and independent of that, the experience of other kinds of negative affect (e.g., anger, frustration) also predicted higher levels of cortisol and the experience of positive affect (e.g., happy, joyful) predicted lower levels of this important stress hormone. Taken together, these studies illustrate how researchers can use ambulatory physiological monitoring to study how the little—and seemingly trivial or inconsequential—experiences in our lives leave objective, measurable traces in our bodily systems.

Studying Online Behavior

Another domain of daily life that has only recently emerged is virtual daily behavior or how people act and interact with others on the Internet. Irrespective of whether social media will turn out to be humanity’s blessing or curse (both scientists and laypeople are currently divided over this question), the fact is that people are spending an ever increasing amount of time online. In light of that, researchers are beginning to think of virtual behavior as being as serious as “actual” behavior and seek to make it a legitimate target of their investigations ( Gosling & Johnson, 2010 ).

One way to study virtual behavior is to make use of the fact that most of what people do on the Web—emailing, chatting, tweeting, blogging, posting— leaves direct (and permanent) verbal traces. For example, differences in the ways in which people use words (e.g., subtle preferences in word choice) have been found to carry a lot of psychological information ( Pennebaker, Mehl, & Niederhoffer, 2003 ). Therefore, a good way to study virtual social behavior is to study virtual language behavior. Researchers can download people’s—often public—verbal expressions and communications and analyze them using modern text analysis programs (e.g., Pennebaker, Booth, & Francis, 2007).

For example, Cohn, Mehl, and Pennebaker ( 2004 ) downloaded blogs of more than a thousand users of lifejournal.com, one of the first Internet blogging sites, to study how people responded socially and emotionally to the attacks of September 11, 2001. In going “the online route,” they could bypass a critical limitation of coping research, the inability to obtain baseline information; that is, how people were doing before the traumatic event occurred. Through access to the database of public blogs, they downloaded entries from two months prior to two months after the attacks. Their linguistic analyses revealed that in the first days after the attacks, participants expectedly expressed more negative emotions and were more cognitively and socially engaged, asking questions and sending messages of support. Already after two weeks, though, their moods and social engagement returned to baseline, and, interestingly, their use of cognitive-analytic words (e.g., “think,” “question”) even dropped below their normal level. Over the next six weeks, their mood hovered around their pre-9/11 baseline, but both their social engagement and cognitive-analytic processing stayed remarkably low. This suggests a social and cognitive weariness in the aftermath of the attacks. In using virtual verbal behavior as a marker of psychological functioning, this study was able to draw a fine timeline of how humans cope with disasters.

Reflecting their rapidly growing real-world importance, researchers are now beginning to investigate behavior on social networking sites such as Facebook ( Wilson, Gosling, & Graham, 2012 ). Most research looks at psychological correlates of online behavior such as personality traits and the quality of one’s social life but, importantly, there are also first attempts to export traditional experimental research designs into an online setting. In a pioneering study of online social influence, Bond and colleagues ( 2012 ) experimentally tested the effects that peer feedback has on voting behavior. Remarkably, their sample consisted of 16 million (!) Facebook users. They found that online political-mobilization messages (e.g., “I voted” accompanied by selected pictures of their Facebook friends) influenced real-world voting behavior. This was true not just for users who saw the messages but also for their friends and friends of their friends. Although the intervention effect on a single user was very small, through the enormous number of users and indirect social contagion effects, it resulted cumulatively in an estimated 340,000 additional votes—enough to tilt a close election. In short, although still in its infancy, research on virtual daily behavior is bound to change social science, and it has already helped us better understand both virtual and “actual” behavior.

“Smartphone Psychology”?

A review of research methods for studying daily life would not be complete without a vision of “what’s next.” Given how common they have become, it is safe to predict that smartphones will not just remain devices for everyday online communication but will also become devices for scientific data collection and intervention ( Kaplan & Stone, 2013 ; Yarkoni, 2012 ). These devices automatically store vast amounts of real-world user interaction data, and, in addition, they are equipped with sensors to track the physical (e. g., location, position) and social (e.g., wireless connections around the phone) context of these interactions. Miller ( 2012 , p. 234) states, “The question is not whether smartphones will revolutionize psychology but how, when, and where the revolution will happen.” Obviously, their immense potential for data collection also brings with it big new challenges for researchers (e.g., privacy protection, data analysis, and synthesis). Yet it is clear that many of the methods described in this module—and many still to be developed ways of collecting real-world data—will, in the future, become integrated into the devices that people naturally and happily carry with them from the moment they get up in the morning to the moment they go to bed.

This module sought to make a case for psychology research conducted outside the lab. If the ultimate goal of the social and behavioral sciences is to explain human behavior, then researchers must also—in addition to conducting carefully controlled lab studies—deal with the “messy” real world and find ways to capture life as it naturally happens.

Mortensen and Cialdini (2010) refer to the dynamic give-and-take between laboratory and field research as “ full-cycle psychology ”. Going full cycle, they suggest, means that “researchers use naturalistic observation to determine an effect’s presence in the real world, theory to determine what processes underlie the effect, experimentation to verify the effect and its underlying processes, and a return to the natural environment to corroborate the experimental findings” (Mortensen & Cialdini, 2010, p. 53). To accomplish this, researchers have access to a toolbox of research methods for studying daily life that is now more diverse and more versatile than it has ever been before. So, all it takes is to go ahead and—literally—bring science to life.

• Outside Resources
• Discussion Questions
• What do you think about the tradeoff between unambiguously establishing cause and effect (internal validity) and ensuring that research findings apply to people’s everyday lives (external validity)? Which one of these would you prioritize as a researcher? Why?
• What challenges do you see that daily-life researchers may face in their studies? How can they be overcome?
• What ethical issues can come up in daily-life studies? How can (or should) they be addressed?
• How do you think smartphones and other mobile electronic devices will change psychological research? What are their promises for the field? And what are their pitfalls?
• Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2 , 396–403.
• Bolger, N., & Laurenceau, J-P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research . New York, NY: Guilford Press.
• Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616.
• Bond, R. M., Jones, J. J., Kramer, A. D., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61 million-person experiment in social influence and political mobilization. Nature , 489, 295–298.
• Brewer, M. B. (2000). Research design and issues of validity. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology (pp. 3–16). New York, NY: Cambridge University Press.
• Cohn, M. A., Mehl, M. R., & Pennebaker, J. W. (2004). Linguistic indicators of psychological change after September 11, 2001. Psychological Science, 15, 687–693.
• Conner, T. S., Tennen, H., Fleeson, W., & Barrett, L. F. (2009). Experience sampling methods: A modern idiographic approach to personality research. Social and Personality Psychology Compass, 3 , 292–313.
• Craik, K. H. (2000). The lived day of an individual: A person-environment perspective. In W. B. Walsh, K. H. Craik, & R. H. Price (Eds.), Person-environment psychology: New directions and perspectives (pp. 233–266). Mahwah, NJ: Lawrence Erlbaum Associates.
• Fahrenberg, J., &. Myrtek, M. (Eds.) (1996). Ambulatory assessment: Computer-assisted psychological and psychophysiological methods in monitoring and field studies . Seattle, WA: Hogrefe & Huber.
• Funder, D. C. (2007). The personality puzzle . New York, NY: W. W. Norton & Co.
• Funder, D. C. (2001). Personality. Review of Psychology, 52, 197–221.
• Gosling, S. D., & Johnson, J. A. (2010). Advanced methods for conducting online behavioral research . Washington, DC: American Psychological Association.
• Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002). A room with a cue: Personality judgments based on offices and bedrooms. Journal of Personality and Social Psychology, 82 , 379–398.
• Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life . Thousand Oaks, CA: Sage.
• Kahneman, D., Krueger, A., Schkade, D., Schwarz, N., and Stone, A. (2004). A survey method for characterizing daily life experience: The Day Reconstruction Method. Science , 306, 1776–780.
• Kaplan, R. M., & Stone A. A. (2013). Bringing the laboratory and clinic to the community: Mobile technologies for health promotion and disease prevention. Annual Review of Psychology, 64 , 471-498.
• Killingsworth, M. A., & Gilbert, D. T. (2010). A wandering mind is an unhappy mind. Science , 330, 932.
• Lane, R. D., Zareba, W., Reis, H., Peterson, D., &, Moss, A. (2011). Changes in ventricular repolarization duration during typical daily emotion in patients with Long QT Syndrome. Psychosomatic Medicine, 73 , 98–105.
• Lewin, K. (1944) Constructs in psychology and psychological ecology . University of Iowa Studies in Child Welfare, 20, 23–27.
• Mehl, M. R., & Conner, T. S. (Eds.) (2012). Handbook of research methods for studying daily life . New York, NY: Guilford Press.
• Mehl, M. R., Pennebaker, J. W., Crow, M., Dabbs, J., & Price, J. (2001). The electronically activated recorder (EAR): A device for sampling naturalistic daily activities and conversations. Behavior Research Methods, Instruments, and Computers, 33 , 517–523.
• Mehl, M. R., Robbins, M. L., & Deters, G. F. (2012). Naturalistic observation of health-relevant social processes: The electronically activated recorder (EAR) methodology in psychosomatics. Psychosomatic Medicine, 74 , 410–417.
• Mehl, M. R., Vazire, S., Ramírez-Esparza, N., Slatcher, R. B., & Pennebaker, J. W. (2007). Are women really more talkative than men? Science, 317 , 82.
• Miller, G. (2012). The smartphone psychology manifesto. Perspectives in Psychological Science , 7, 221–237.
• Mortenson, C. R., & Cialdini, R. B. (2010). Full-cycle social psychology for theory and application. Social and Personality Psychology Compass, 4, 53–63.
• Pennebaker, J. W., Mehl, M. R., Niederhoffer, K. (2003). Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology, 54 , 547–577.
• Ramírez-Esparza, N., Mehl, M. R., Álvarez Bermúdez, J., & Pennebaker, J. W. (2009). Are Mexicans more or less sociable than Americans? Insights from a naturalistic observation study. Journal of Research in Personality, 43 , 1–7.
• Rathje, W., & Murphy, C. (2001). Rubbish! The archaeology of garbage . New York, NY: Harper Collins.
• Reis, H. T., & Gosling, S. D. (2010). Social psychological methods outside the laboratory. In S. T. Fiske, D. T. Gilbert, & G. Lindzey, (Eds.), Handbook of social psychology (5th ed., Vol. 1, pp. 82–114). New York, NY: Wiley.
• Sapolsky, R. (2004). Why zebras don’t get ulcers: A guide to stress, stress-related diseases and coping . New York, NY: Henry Holt and Co.
• Schlotz, W. (2012). Ambulatory psychoneuroendocrinology: Assessing salivary cortisol and other hormones in daily life. In M.R. Mehl & T.S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 193–209). New York, NY: Guilford Press.
• Smyth, J., Ockenfels, M. C., Porter, L., Kirschbaum, C., Hellhammer, D. H., & Stone, A. A. (1998). Stressors and mood measured on a momentary basis are associated with salivary cortisol secretion. Psychoneuroendocrinology, 23 , 353–370.
• Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16 , 199–202.
• Stone, A. A., Reed, B. R., Neale, J. M. (1987). Changes in daily event frequency precede episodes of physical symptoms. Journal of Human Stress, 13 , 70–74.
• Webb, E. J., Campbell, D. T., Schwartz, R. D., Sechrest, L., & Grove, J. B. (1981). Nonreactive measures in the social sciences . Boston, MA: Houghton Mifflin Co.
• White, W. B., Schulman, P., McCabe, E. J., & Dey, H. M. (1989). Average daily blood pressure, not office blood pressure, determines cardiac function in patients with hypertension. Journal of the American Medical Association, 261 , 873–877.
• Whyte, W. H. (1980). The social life of small urban spaces . Washington, DC: The Conservation Foundation.
• Wilhelm, F.H., & Grossman, P. (2010). Emotions beyond the laboratory: Theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment. Biological Psychology, 84 , 552–569.
• Wilhelm, P., Perrez, M., & Pawlik, K. (2012). Conducting research in daily life: A historical review. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life . New York, NY: Guilford Press.
• Wilson, R., & Gosling, S. D., & Graham, L. (2012). A review of Facebook research in the social sciences. Perspectives on Psychological Science, 7 , 203–220.
• Yarkoni, T. (2012). Psychoinformatics: New horizons at the interface of the psychological and computing sciences. Current Directions in Psychological Science, 21 , 391–397.

• Creative Commons License

How to cite this Noba module using APA Style

Forgot your password?

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

The ‘Real-World Approach’ and Its Problems: A Critique of the Term Ecological Validity

Gijs a. holleman.

1 Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, Netherlands

2 Department of Developmental Psychology, Utrecht University, Utrecht, Netherlands

Ignace T. C. Hooge

Chantal kemner.

3 Brain Center, University Medical Center Utrecht, Utrecht, Netherlands

Roy S. Hessels

A popular goal in psychological science is to understand human cognition and behavior in the ‘real-world.’ In contrast, researchers have typically conducted their research in experimental research settings, a.k.a. the ‘psychologist’s laboratory.’ Critics have often questioned whether psychology’s laboratory experiments permit generalizable results. This is known as the ‘real-world or the lab’-dilemma. To bridge the gap between lab and life, many researchers have called for experiments with more ‘ecological validity’ to ensure that experiments more closely resemble and generalize to the ‘real-world.’ However, researchers seldom explain what they mean with this term, nor how more ecological validity should be achieved. In our opinion, the popular concept of ecological validity is ill-formed, lacks specificity, and falls short of addressing the problem of generalizability. To move beyond the ‘real-world or the lab’-dilemma, we believe that researchers in psychological science should always specify the particular context of cognitive and behavioral functioning in which they are interested, instead of advocating that experiments should be more ‘ecologically valid’ in order to generalize to the ‘real-world.’ We believe this will be a more constructive way to uncover the context-specific and context-generic principles of cognition and behavior.

Introduction

A popular goal in psychological science is to understand human cognition and behavior in the ‘real-world.’ In contrast, researchers have traditionally conducted experiments in specialized research settings, a.k.a. the ‘psychologist’s laboratory’ ( Danziger, 1994 ; Hatfield, 2002 ). Over the course of psychology’s history, critics have often questioned whether psychology’s lab-based experiments permit the generalization of results beyond the laboratory settings within which these results are typically obtained. In response, many researchers have advocated for more ‘ecologically valid’ experiments, as opposed to the so-called ‘conventional’ laboratory methods ( Neisser, 1976 ; Aanstoos, 1991 ; Kingstone et al., 2008 ; Shamay-Tsoory and Mendelsohn, 2019 ; Osborne-Crowley, 2020 ). In recent years, several technological advances (e.g., virtual reality, wearable eye trackers, mobile EEG devices, fNIRS, biosensors, etc.) have further galvanized researchers to emphasize the importance of studying human cognition and behavior in the ‘real-world,’ as new technologies will aid researchers in overcoming some of the inherent limitations of laboratory experiments ( Schilbach, 2015 ; Shamay-Tsoory and Mendelsohn, 2019 ; Sonkusare et al., 2019 ).

In this article, we will argue that the general aspiration of researchers to understand human cognition and behavior in the ‘real-world’ by conducting experiments that are more ‘ecologically valid’ (henceforth referred to as the ‘real-world approach’) is not without its problems. Most notably, we will argue that the popular term ‘ecological validity,’ which is widely used nowadays by researchers to discuss whether experimental research resembles and generalizes to the ‘real-world,’ is shrouded in both conceptual and methodological confusion. As we ourselves are interested in cognitive and behavioral functioning in the context of people’s everyday experience, and conduct experiments across various ‘laboratory’ and ‘real-world’ environments, we have seen how the uncritical use of the term ‘ecological validity’ can lead to rather misleading and counterproductive discussions. This not only holds for how this concept is used in many scholarly articles and textbooks, but also in presentations and discussions of experimental research at conferences, during the review process, and when talking with students about experimental design and the analysis of evidence.

Although the usage of the term ecological validity has previously been criticized by several scholars ( Hammond, 1998 ; Schmuckler, 2001 ; cf. Araujo et al., 2007 ; Dunlosky et al., 2009 ), we think that these critiques have largely been overlooked. Therefore, it will be necessary to cover some of the same ground. The contribution of this article is threefold. First, we extend the critique of the term ecological validity and apply it to the field of social attention. Second, we scrutinize some of the assumptions that guide the contemporary framework of ecological validity, specifically those regarding artificiality–naturality and simplicity–complexity. Finally, our article is meant to educate a new generation of students and researchers on the historical roots and conceptual issues of the term ecological validity. This article consists of four parts. First, we will provide a brief history of the so-called ‘real-world or the lab’-dilemma and discuss several definitions and interpretations of the term ecological validity. Second, we will go into the historical roots of the concept of ecological validity and describe how the original meaning of this concept has transformed significantly. Third, we will scrutinize the prevailing assumptions that seems to guide how researchers are currently using the term ecological validity. Finally, we will apply our conceptual analysis to a specific field of study, namely the field of social attention. In recent years, this field has been particularly concerned with issues of ecological validity and generalizability. Therefore, the field of social attention offers an exemplary case to explain how the uncritical use of the terms ‘ecological validity’ and the ‘real-world’ may lead to misleading and counterproductive conclusions.

A Brief History of the ‘Real-World or the Lab’-Dilemma

The popular story of psychology (or the broader ‘cognitive sciences’) has it that “psychology became a science by rising from the ‘armchair’ of speculation and uncontrolled observation, and entering the laboratory to undertake controlled observation and measurement” ( Hatfield, 2002 , p. 208). The ‘psychologist’s laboratory’, a special room furnished with all kinds of lab paraphernalia and sophisticated equipment, has been regarded as the celebrated vehicle of psychology’s journey into sciencehood ( Danziger, 1994 ; Goodwin, 2015 ). However, despite psychologists’ long tradition of laboratory experimentation (for a history and discussion, see Gillis and Schneider, 1966 ), there also have been many critical voices saying that psychology’s laboratory experiments are too limited in scope to study how people function in daily life. For example, Brunswik (1943 , p. 262) once wrote that experimental psychology was limited to “narrow-spanning problems of artificially isolated proximal or peripheral technicalities of mediation which are not representative of the larger patterns of life”. Barker (1968 , p. 3) wrote that “it is impossible to create in the laboratory the frequency, duration, scope and magnitude of some important human conditions.” Neisser (1976 , p. 34) wrote that “contemporary studies of cognitive processes usually use stimulus material that is abstract, discontinuous, and only marginally real.” Bronfenbrenner (1977 , p. 513) wrote that “many of these experiments involve situations that are unfamiliar, artificial, and short-lived and that call for unusual behaviors that are difficult to generalize to other settings.” Kingstone et al. (2008 , p. 355) declared that “the research performed in labs, and the findings they generate, are in principle and in practice unlikely to be of relevance to the more complex situations that people experience in everyday lif e, ” and Shamay-Tsoory and Mendelsohn (2019 , p. 1) stated that “ conventional experimental psychological approaches have mainly focused on investigating behavior of individuals as isolated agents situated in artificial, sensory, and socially deprived environments, limiting our understanding of naturalistic cognitive, emotional, and social phenomena.”

According to these scholars, psychological science is faced with a gloomy predicament: findings and results based on highly controlled and systematically designed laboratory experiments may not be a great discovery but only a “mere laboratory curiosity” ( Gibson, 1970 , pp. 426–427). As Anderson et al. (1999 , p. 3) put it: “A common truism has been that … laboratory studies are good at telling whether or not some manipulation of an independent variable causes changes in the dependent variable, but many scholars assume that these results do not generalize to the “real-world.” The general concern is that, due to the ‘artificiality’ and ‘simplicity’ of the laboratory, some (if not many) lab-based experiments do not adequately represent the ‘naturality’ and ‘complexity’ of psychological phenomena in everyday life (see Figure 1 ). This problem has become familiar to psychologists as the ‘real-world or the lab’-dilemma ( Hammond and Stewart, 2001 ). At the heart of psychology’s ‘real-world or the lab’-dilemma lies a pernicious methodological choice: “Should it [psychological science] pursue the goal of generality by demanding that research be generalizable to “real life” (aka the “real-world”), or should it pursue generalizability by holding onto its traditional laboratory research paradigm?” ( Hammond and Stewart, 2001 , p. 7).

Examples of historical and contemporary laboratory rooms and field experiments. (A) A laboratory room from the early 20th century. A participant is seated in front a ‘disc tachistoscope,’ an apparatus to display visual images (adapted from Hilton, 1920 ). (B) A picture of a field experiment by J. J. Gibson. Observers had to judge the size of an object in the distance (adapted from Gibson, 1950 ). (C) A 21st century eye tracking laboratory. A participant is seated in front of a SMI Hi-Speed tower-mounted eye tracker (based on Valtakari et al., 2019 ). (D) A wearable eye-tracker (barely visible) is used to measure gaze behavior while participants walked through corridors with human crowds ( Hessels et al., 2020 ). Copyright statement – Panels (A,B) . All photographs are used under the provision of the “fair use” U.S. Copyright Act 107 and Dutch Copyright Law Article 15a for non-profit purposes of research, education and scholarly comment. The photograph from W. Hilton’s book: Applied Psychology: Driving Power of Thought (Original date of publication, 1920). Retrieved April 1, 2020, from http://www.gutenberg.org/files/33076/33076-h/33076-h.htm . The photograph from J. J. Gibson’s book: The Perception of the Visual World (Original date of publication, 1950, Figure 74, p. 184) was retrieved from a copy of the Utrecht University library. (C,D) Photographs are owned by the authors and the people depicted in the images gave consent for publication.

Although psychological science is comprised of many specialized research areas, the goal to understand human cognition and behavior in the ‘real-world’ has become a critically acclaimed goal for psychologists and cognitive scientists of all stripes. Indeed, examples of the ‘real-world or the lab’-dilemma can be found not only in various ‘applied’ fields of psychology, such as ergonomics ( Hoc, 2001 ), clinical (neuro)psychology ( Wilson, 1993 ; Parsons, 2015 ), educational psychology ( Dunlosky et al., 2009 ), sport psychology ( Davids, 1988 ), marketing and consumer psychology ( Smith et al., 1998 ), and the psychology of driving ( Rogers et al., 2005 ), but also in the so-called ‘basic’ fields of psychological science, such as the study of perception ( Brunswik, 1956 ; Gibson, 1979/2014 ), attention ( Simons and Levin, 1998 ; Peelen and Kastner, 2014 ), memory ( Banaji and Crowder, 1989 ; Neisser, 1991 ; Cohen and Conway, 2007 ), social cognition ( Schilbach et al., 2013 ; Schilbach, 2015 ; Shamay-Tsoory and Mendelsohn, 2019 ; Osborne-Crowley, 2020 ), judgment-and-decision making ( Koehler, 1996 ), and child development ( Lewkowicz, 2001 ; Schmuckler, 2001 ; Adolph, 2019 ).

The ‘Real-World Approach’: A Call for Ecological Validity

In the past decades, researchers have often discussed how they may overcome some of the limitations of laboratory-based experiments. Perhaps the largest common denominator of what we call the ‘real-world approach’ is a strong emphasis on ‘ecological validity.’ Over the past decades, the term ecological validity has made its appearance whenever researchers became concerned with the potential limitations of laboratory experiments (see e.g., Jenkins, 1974 ; Neisser, 1976 ; Banaji and Crowder, 1989 ; Aanstoos, 1991 ; Koehler, 1996 ; Smilek et al., 2006 ; Risko et al., 2012 ; Schilbach, 2015 ; Caruana et al., 2017 ; Shamay-Tsoory and Mendelsohn, 2019 ; Osborne-Crowley, 2020 ). As Neisser (1976 , p. 33) famously put it:

“The concept of ecological validity has become familiar to psychologists. It reminds them that the artificial situation created for an experiment may differ from the everyday world in crucial ways. When this is so, the results may be irrelevant to the phenomena that one would really like to explain.”

The main problem, according to Neisser and many others, is that experiments in psychological science are generally “lacking in ecological validity” ( Neisser, 1976 , p. 7; Smilek et al., 2006 ; Shamay-Tsoory and Mendelsohn, 2019 ; Sonkusare et al., 2019 ). Aanstoos (1991 , p. 77) even referred to this problem as the “ecological validity crisis.” To counter this problem, many researchers have called for studies with ‘more’ or ‘greater’ ecological validity. For example, Koehler (1996 , p. 1) advocated for a “more ecologically valid research program,” Schilbach (2015 , p. 130) argued for “the inclusion of more ecologically valid conditions,” and Smilek et al. (2006 , p. 104) suggested that “in order for results to generalize to real-world scenarios we need to use tasks with greater ecological validity.” Clearly, ecological validity is regarded as an important feature of experimental research by researchers who pursue the ‘real-world approach.’ However, in our opinion, and we are not alone in this regard (see also Hammond, 1998 ; Araujo et al., 2007 ; Dunlosky et al., 2009 ), this notion of ecological validity has caused considerable confusion. To foreshadow some of our criticism of ecological validity, we will show that this concept has largely been detached from its original parentage (cf. Brunswik, 1949 ), and is now host to different interpretations guided by questionable assumptions (for a history, see Hammond, 1998 ). Worst of all, the concept is often wielded as a blunt weapon to criticize and dismiss experiments, even though researchers seldom make explicit what definition of ecological validity they use or by which set of criteria they have evaluated a study’s ecological validity (as previously pointed out by Hammond, 1998 ; Schmuckler, 2001 ; Dunlosky et al., 2009 ).

The Big Umbrella of Ecological Validity

In past decades, the concept of ecological validity has been related to various facets of psychological research, for example, the ecological validity of stimuli ( Neisser, 1976 ; Risko et al., 2012 ; Jack and Schyns, 2017 ), the ecological validity of tasks ( Smilek et al., 2006 ; Krakauer et al., 2017 ), the ecological validity of conditions ( Schilbach, 2015 ; Blanco-Elorrieta and Pylkkänen, 2018 ), the ecological validity of research settings ( Bronfenbrenner, 1977 ; Schmuckler, 2001 ), the ecological validity of results ( Eaton and Clore, 1975 ; Greenwald, 1976 ; Silverstein and Stang, 1976 ), the ecological validity of theories ( Neisser, 1976 ), the ecological validity of research designs ( Rogers et al., 2005 ), the ecological validity of methods ( Banaji and Crowder, 1989 ), the ecological validity of phenomena ( Johnston et al., 2014 ), the ecological validity of data ( Aspland and Gardner, 2003 ), and the ecological validity of paradigms ( Macdonald and Tatler, 2013 ; Schilbach et al., 2013 ). However, despite the popular usage of this term, specific definitions and requirements of ecological validity are not always clear.

A closer look at the literature suggests that different definitions and interpretations are used by researchers. Let’s consider some examples of the literature where researchers have been more explicit in their definitions of ecological validity. For example, Ashcraft and Radvansky (2009 , p. 511) defined ecological validity as: “The hotly debated principle that research must resemble the situations and task demands that are characteristic of the real-world rather than rely on artificial laboratory settings and tasks so that results will generalize to the real-world, that is, will have ecological validity.” Another influential definition of ecological validity was given by Bronfenbrenner (1977) , who defined ecological validity as “the extent to which the environment experienced by the subjects in a scientific investigation has the properties it is supposed or assumed to have by the investigator” (p. 516). In Bronfenbrenner’s view, a study’s ecological validity should not be predicated on the extent to which the research context resembles or is carried out in a ‘real-life’ environment. Instead, theoretical considerations should guide one’s methodological decisions on what type of research context is most appropriate given one’s focus of inquiry. For example, if one is interested in the behavioral responses of children when they are placed in a ‘strange situation’ then a laboratory room may be adequately suited for that particular research goal. However, if one is interested in how children behave within their home environment, then a laboratory room may not be the most suitable research context. As Bronfenbrenner (1977 , p. 516) remarked: “Specifically, so far as young children are concerned, the results indicate that the strangeness of the laboratory situation tends to increase anxiety and other negative feeling states and to decrease manifestations of social competence.”

Ecological validity has also been used interchangeably with (or regarded as a necessary component of) ‘external validity’ ( Berkowitz and Donnerstein, 1982 ; Mook, 1983 ; Hoc, 2001 ). The concept of external validity typically refers to whether a given study result or conclusion, usually obtained under one set of conditions and with one group of participants, can also be generalized to other people, tasks, and situations ( Campbell, 1957 ). For example, in the literature on neuropsychological assessment and rehabilitation, ecological validity has primarily been conceptualized as “ … the degree to which clinical tests of cognitive functioning predict functional impairment” ( Higginson et al., 2000 , p. 185). In this field, there has been much discussion about whether the neuropsychological tests used by clinicians accurately predict cognitive and behavioral impairments in everyday life ( Heinrichs, 1990 ; Wilson, 1993 ). One major concern is that the test materials are either too abstract or too general to adequately represent the kind of problems that people with cognitive and neurological impairments encounter in their daily routines, for example, while cooking or buying food at the supermarket. In response, various efforts have been made to increase the ecological validity of neuropsychological tests, for example, by developing performance measures with relevance for everyday tasks and activities ( Shallice and Burgess, 1991 ; Alderman et al., 2003 ), by combining and correlating tests results with behavioral observations and self-reports ( Wilson, 1993 ; Higginson et al., 2000 ), and by using Virtual Reality (VR) applications to create test situations in which a patient’s cognitive and functional impairments are likely to be expressed ( Parsons, 2015 ; Parsons et al., 2017 ).

The Historical Roots of Ecological Validity

As we have seen, definitions and interpretations of ecological validity may not only differ among researchers, but also across various subfields within psychology. As such, it is not always clear how the concept should be interpreted. Interestingly, the term ecological validity used to have a very precise meaning when it was first introduced to psychological science by Brunswik (1949 , 1952 , 1955 , 1956) . Brunswik coined the term ‘ecological validity’ to describe the correlation between a proximal sensory cue (e.g., retinal stimulation) and a distal object-variable (e.g., object in the environment). In Brunswik’s terminology, ecological validity refers to a measure (a correlation coefficient) that describes a probabilistic relationship between the distal and proximal layers of an organism-environment system. According to Brunswik (1955) : “A correlation between ecological variables, one which is capable of standing in this manner as a probability cue for the other, may thus be labeled “ecological validity”” (p. 199). Brunswik (1952) believed psychology to primarily be a science of organism-environment relations in which the “organism has to cope with an environment full of uncertainties” (p. 22). In Brunswik’s ‘lens model’ ( Brunswik, 1952 ), the ecological validities of perceptual cues indicate the potential utility of these cues for the organism to achieve its behavioral goals. Note that Brunswik’s concept of ecological validity is very different from how the term is generally used nowadays, namely to discuss and evaluate whether some laboratory-based experiments resemble and generalize to the ‘real-world’ (cf. Neisser, 1976 ; Smilek et al., 2006 ; Ashcraft and Radvansky, 2009 ; Shamay-Tsoory and Mendelsohn, 2019 ).

The erosion and distortion of Brunswik’s definition of ecological validity has been documented by several scholars (e.g., Hammond, 1998 ; Araujo et al., 2007 ; Holleman et al., in press ). As explained by Hammond (1998) , the original definition of ecological validity, as Brunswik (1949 , 1952) introduced it, has been conflated with Brunswik’s ‘representative design’ of experiments ( Brunswik, 1955 , 1956 ). Representative design was Brunswik’s methodological program for psychological science to achieve generalizability of results. To achieve this, researchers should not only conduct proper sampling on the side of the subjects, by sampling subjects who are representative of a specific ‘target population’ (e.g., children, patients), but researchers should also sample stimuli, tasks, and situations which are representative of a specific ‘target ecology.’ As such, an experiment may be treated as a sample of this ‘target ecology.’ By virtue of sampling theory, researchers may then determine whether results can be generalized to the intended conditions. In short, representative design requires researchers to first specify the conditions toward which they intend to generalize their findings, and then specify how those conditions are represented in the experimental arrangement ( Brunswik, 1956 ). For more in-depth discussions on representative design, see Hammond and Stewart (2001) ; Dhami et al. (2004) , and Hogarth (2005) .

A Systematic Approach to Ecological Validity?

The current lack of terminological precision surrounding ecological validity is, to say the least, problematic. There seems to be no agreed upon definition in the literature, nor any means of classification to determine or evaluate a study’s ecological validity. This seems to be at odds with the relative ease by which researchers routinely invoke this concept to discuss the limitations and shortcomings of laboratory experiments. All the while, researchers seldom make clear how they have determined a study’s ecological (in)validity. As Schmuckler (2001 , p. 419) pointed out: “One consequence of this problem is that concerns with ecological validity can be raised in most experimental situations.” To overcome these problems, several scholars have emphasized the need for a more systematic approach to ecological validity ( Lewkowicz, 2001 ; Schmuckler, 2001 ; Kingstone et al., 2008 ; Risko et al., 2012 ). For example, Lewkowicz (2001 , p. 443) wrote that: “What is missing is an independent, objective, and operational definition of the concept of ecological validity that makes it possible to quantify a stimulus or event as more or less ecologically valid.” According to Schmuckler (2001) , ecological validity can be evaluated on at least three dimensions: (1) the nature of the stimuli ; (2) the nature of task, behavior, or response ; (3) the nature of the research context . Researchers have primarily discussed these dimensions in terms of their artificiality–naturality (e.g., Hoc, 2001 ; Schmuckler, 2001 ; Risko et al., 2012 ; Shamay-Tsoory and Mendelsohn, 2019 ; Sonkusare et al., 2019 ), and their simplicity–complexity (e.g., Kingstone et al., 2008 ; Peelen and Kastner, 2014 ; Lappi, 2015 ). As such, a general framework can be construed where stimuli, tasks, behaviors, and research contexts can be evaluated on a continuum of artificiality–naturality and simplicity–complexity (see also Risko et al., 2012 ; Lappi, 2015 ; Shamay-Tsoory and Mendelsohn, 2019 ; Osborne-Crowley, 2020 ). At one extreme is the laboratory, characterized by its artificiality and simplicity. At the other extreme is the ‘real-world,’ characterized by its naturality and complexity. According to this multidimensional framework, researchers may determine a study’s overall ecological validity by combining (e.g., averaging or summing) the main components of ecological validity (i.e., stimuli, tasks/behaviors, research context) in terms of their relative artificiality–naturality and simplicity–complexity. However, while many researchers have conceptualized ecological validity alongside these dimensions, we think there are several problems to consider. Since the dimensions of this framework are supposedly important to determine the ecological validity of experimental research, this then raises the question of how researchers can judge the artificiality–naturality and simplicity–complexity of particular experiments. This question will be explored in the following sections.

Artificiality – Naturality

The contrast between ‘artificiality’ and ‘naturality’ is a particularly prominent point of discussion in the ‘real-world or the lab’-dilemma and when researchers talk about the ecological validity of experimental research practices ( Hoc, 2001 ; Kingstone et al., 2008 ; Shamay-Tsoory and Mendelsohn, 2019 ). According to Hoc (2001 , pp. 282–283), ‘artificial’ situations are “those that are specifically designed for research” and ‘natural’ situations are “the target situations to be understood by research” . Importantly, Hoc (2001) notes that this distinction is made from the perspective of the researcher. However, this artificiality–naturality distinction should also be considered from the subject’s point of view. For example, according to Sonkusare et al. (2019) : “naturalistic paradigms can be heuristically defined as those that employ the rich, multimodal dynamic stimuli that represent our daily lived experience, such as film clips, TV advertisements, news items, and spoken narratives, or that embody relatively unconstrained interactions with other agents, gaming environments, or virtual realities” (p. 700). Furthermore, researchers have long recognized that artificiality arises when the experimental methods employed by researchers interfere with the naturality of the psychological phenomena one aims to study. Consequently, there is always an inherent trade-off between the degree of artificiality imposed by the experimental conditions and the naturality of the phenomena under scientific investigation ( Brunswik, 1956 ; Barker, 1968 ; Banaji and Crowder, 1989 ; Kingstone et al., 2008 ; Risko et al., 2012 ; Caruana et al., 2017 ). However, as Winograd (1988) has previously remarked, it remains difficult to “draw a line where artificiality ends and ecological validity … for real events begins” (p. 18).

Interestingly, discussions on the naturality–artificiality of experimental methods have a long pedigree in psychological science. By the end of the 19th century, Thorndike (1899) and Mills (1899) already argued fiercely about what methodology should be favored to study the behavior of cats. Mills dismissed Thorndike’s work because of the artificiality of the experimental methods employed by Thorndike (see Figure 2 ), whereas Thorndike regarded the ethological approach favored by Mills as a collection of uncritical observations and anecdotes. Mills (1899 , p. 264) wrote that: “Dr. Thorndike … has given the impression that I have not made experiments, or ‘crucial experiments’ … I may remark that a laboratory as ordinarily understood is not well suited for making psychological experiments on animals” . Mills’ point was that: “cats placed in small enclosures … cannot be expected to act naturally. Thus, nothing from about their normal behavior can be determined from their behavior in highly artificial, abnormal surroundings” ( Goodwin, 2015 , p. 200). In response to Mills, Thorndike (1899 , p. 414) replied: “Professor Mills does not argue in concrete terms, does not criticize concrete unfitness in the situations I devised for the animals. He simply names them unnatural.” Thorndike clearly did not accept Mills’ charge on the artificiality of his experimental arrangements to study the behavior of cats because Mills did not define what should be considered natural behavior in the first place.

A ‘puzzle box’ devised by Thorndike (1899 , 2017) to study learning behavior of cats. A hungry cat is placed in a box which can be opened if the cat pushes a latch. A food reward (‘positive reinforcer’) will be obtained by the cat if it figures out how to escape from the box. Thorndike discovered that after several trials, the time it takes the cat to escape from the box decreases. Experiments with puzzle boxes remain popular today to study the cognitive capacities of animals, for example, see Richter et al. (2016) for a study with octopuses. Copyright statement – Image created and owned by author IH and is based on E. L. Thorndike’s book: Animal Intelligence (Original date of publication, 1911, Figure 1, p. 30).

We think that this historical discussion between Thorndike and Mills is illuminating, because it characterizes the heart of the discussion on ecological validity nowadays. Namely, what exactly did Mills consider to be ‘natural’ or ‘normal’ behavior? And how did Mills determine that Thorndike’s experiments failed to capture the ‘natural’ behavior of cats? Following Thorndike’s point on the matter, we think that researchers cannot readily determine the naturality–artificiality of any given experimental arrangement, at least not without specifying what is entailed by these ascriptions. As Dunlosky et al. (2009 , p. 431) previously remarked: “A naturalistic setting guarantees nothing, especially given that “naturalistic” is never unpacked – what does it mean?”. Indeed, our survey of the literature also shows that the historical discussion between Thorndike and Mills is by no means a discussion of the past. In fact, we regularly encounter discussions on the ‘artificiality’ and ‘naturality’ of experimental setups, the presentation of stimuli, the behavior of participants, or the specific tasks and procedures used in experiments – not only in the literature, but also among our colleagues and reviewers. We must often ask for the specifics, because such remarks typically remain undefined by those who toss them around.

Simplicity – Complexity

The contemporary framework of ecological validity also posits that the laboratory and the ‘real-world’ are inversely proportional in terms of their simplicity–complexity. Many researchers have lamented that laboratory experiments have a ‘reductionistic’ tendency to simplify the complexity of the psychological phenomena under study (e.g., Neisser, 1976 ; Kingstone et al., 2008 ; Shamay-Tsoory and Mendelsohn, 2019 ; Sonkusare et al., 2019 ). For example, Sonkusare et al. (2019 , p. 699) stated that “the ecological validity of these abstract, laboratory-style experiments is debatable, as in many ways they do not resemble the complexity and dynamics of stimuli and behaviors in real-life.” But what exactly is meant by complexity? Let’s consider some examples from the literature. In the field of social attention, researchers have often used schematic images, photographs and videos of people and social scenes as stimuli to study the cognitive, behavioral, and physiological processes of face perception, gaze following and joint attention ( Langton et al., 2000 ; Frischen et al., 2007 ; Puce and Bertenthal, 2015 ). However, in recent years, there has been considerable debate that such stimuli are not ‘ecologically valid’ because they do not “capture the complexity of real social situations” ( Birmingham et al., 2012 , p. 30). While we agree that looking at a photographic image of a person’s face is different from looking at a living and breathing person, in what ways do these situations differ in complexity? Do these scholars mean that looking at a ‘live’ person is more complex than looking at a picture of that person? Or do they mean that the former is more complex than the latter from the perspective of the researcher who wants to understand the cognitive, behavioral, and physiological processes of face perception and social attention?

To take another example, Gabor patches are often used as stimuli by experimental psychologists to study ‘low-level visual processing’ (see Figure 3 ). Experimental psychologists use Gabor patches as visual stimuli because they offer a high degree of experimental control over various stimulus parameters (e.g., spatial frequency bandwidths, orientation, contrast, size, location). Gabor patches can described with mathematical precision (i.e., ”Gaussian-windowed sinusoidal gratings,” Fredericksen et al., 1997 , p. 1), and their spatial properties are considered to be a good representation of the receptive field profiles in the primary visual cortex. While Gabor patches may be considered ‘simple’ to researchers who study the relation between low-level visual processing and neural activity in terms of orientation-tuning and hemodynamic response functions, they also point to the yet to be explained ‘complexity’ of the many possible relations between other cognitive processes and patterns of neural activity in the brain. On the other hand, a naïve participant (who likely has no clue about what researchers have discovered about low-level visual processing) may describe these Gabor patches as blurry, kind of stripy, zebra-like circles, and think that they are incredibly boring to look at for many trials while lying quietly in a MRI scanner.

Are Gabor patches simple or complex compared to a picture of zebras? (A) A Gabor patch. (B) A photograph of zebras. The uniquely striped patterns of the zebra makes them most familiar to humans, whereas the question why zebras have such beautiful stripes remains the topic of much discussion among biologists, see e.g., Caro and Stankowich (2015) and Larison et al. (2015) . Copyright statement – Images are used under the provision of the “fair use” U.S. Copyright Act 107 and Dutch Copyright Law Article 15a for non-profit purposes of research, education and scholarly comment. Image of Gabor patch was adapted from Todorović (2016 , May 30). Retrieved April 1, 2020, from http://neuroanatody.com/2016/05/whats-in-a-gabor-patch/ ). Photograph of zebras was made by Ajay Lalu and has been made publicly available by the owner for non-profit purposes via Pixabay . Retrieved on April 1, 2020, from https://pixabay.com/nl/users/ajaylalu-1897335/ .

Our point here is that simplicity–complexity is in the eye of the beholder. Who is to say what is more simple or complex? Physicists, computer scientists, information theorists, and evolutionary biologists have developed various definitions and measures of complexity (e.g., physical complexity, computational complexity, effective complexity, algorithmic complexity, statistical complexity, structural complexity, functional complexity, etc.), typically expressed in strictly mathematical terms ( Edmonds, 1995 ; Gell-Mann, 1995 ; Adami, 2002 ). But what definitions and measures of complexity are used by psychologists and cognitive scientists? Researchers in psychological science seem to have more loosely used the term complexity, for example, to describe a wide range of biological, behavioral, cognitive, social, and cultural phenomena, which typically contain lots of many’s (i.e., many parts, many variables, many degrees of freedom). Researchers may refer to various phenomena as ‘complex’ because they are simply not (yet) understood, as in “the brain is too complex for us to understand” ( Edmonds, 1995 , p. 4). Yet, such intuitive notions of complexity, whether they are caused by ignorance or whether they are used to describe something’s size, number, or variety ( Edmonds, 1995 ), are not very helpful to evaluate the simplicity–complexity of stimuli, tasks, and situations, nor do such notions provide any formula by which these components can be summed to determine the total ecological validity of a given study. According to Gell-Mann (1995 , p. 16):

“As measures of something like complexity for an entity in the real-world, all such quantities are to some extent context-dependent or even subjective. They depend on the coarse graining (level of detail) of the description of the entity, on the previous knowledge and understanding of the world that is assumed, on the language employed, on the coding method used for conversion from that language into a string of bits, and on the particular idealized computer chosen as a standard.”

The ‘Real World’ or the ‘Laboratory’: Psychology’s False Dilemma?

We have discussed several problems with how researchers have used the term ‘ecological validity’. In short, the concept of ecological validity has transformed significantly over the past several decades since it was introduced by Brunswik (1949) . It has lost most of its former theoretical and methodological cohesion (for a history, see Hammond, 1998 ), and the definitions and requirements of ecological validity used by researchers nowadays are seldom made explicit. As such, some experiments may be regarded as ‘ecologically valid’ by one researcher while they can be casually dismissed as ‘ecologically invalid’ by others. A closer look at the literature suggests that many researchers seem to assume that everyone understands what is meant by this term, while in fact the concept of ecological validity is seldom defined. As such, the concept of ecological validity is primarily used nowadays to make hand-waving statements about whether some (lab-based) experiments resemble ‘real life,’ or whether some results obtained in the laboratory may or may not generalize to the ‘real-world.’

In our opinion, the contemporary framework of ecological validity eventually falls short of providing researchers with a tractable research program. Researchers seem to primarily base their judgments of ecological validity upon their own particular theoretical assumptions and considerations about the so-called artificiality–naturality and simplicity–complexity of experimental situations, typically in the absence of a more formal set of criteria. As such, while we certainly sympathize with the ‘call for ecological validity’, insofar it has motivated researchers to be critical about the limitations of experimental methods, we also think that the uncritical use of the term ecological validity has caused a lot of confusion, and in some cases has even been counterproductive. Perhaps the most problematic consequence of using the term ecological validity as an easy substitute for the ‘real-world’ was previously pointed out by Hammond (1998) . He commented that:

“There is, of course, no such thing as a “real-world.” It has been assigned no properties, and no definition; it is used simply because of the absence of a theory of tasks or other environments, and thus does not responsibly offer a frame of reference for the generalization” .

In Hammond’s view, the aim to understand cognitive and behavioral functioning in the ‘real-world’ is basically pointless if one does not first define this notion of the ‘real-world.’ As such, researchers have locked themselves “onto the horns of a false dilemma” ( Hammond and Stewart, 2001 , p. 7). Thus, in order to talk sensibly about whether some results can also be generalized to particular situations beyond the experimental conditions in which those results were obtained, researchers first need to specify the range and distributions of the variables and conditions to which their results are supposed to be applicable. Since the notion of the ‘real-world’ patently lacks specificity, this phrase inevitably hampers researchers to specify the range and boundary conditions of cognitive and behavioral functioning in any given research context, and thus precludes one from getting at the context-specific and context-generic principles of cognition and behavior (see also Kruglanski, 1975 ; Simons et al., 2017 ).

The Nature of the Environment?

Instead of trying to understand cognitive and behavioral functioning in the ‘real-world’, we completely agree with Hammond (1998) that the charge of researchers is to always specify and describe the particular context of behavior in which one is interested. Ultimately, the real challenge for researchers is to develop a theory of how specific environmental contexts are related to various forms of cognitive and behavioral functioning. But what constitutes a psychologist’s theory of the environment? Researchers in psychological science are typically concerned with the nature of the organism, yet, the nature of the environment and its relation to cognitive and behavioral functioning has received considerably less attention from a theoretical point of view ( Barker, 1966 ; Heft, 2013 ). Interestingly, there have been several scholars who have dedicated themselves to precisely this question, and whose theories of cognition and behavior included a clear perspective on the nature of the environment.

According to Tolman and Brunswik (1935) , the nature of the environment, as it appears to the organism, is full of uncertainties. The organism perceives the environment as an array of proximal ‘cues’ and ‘signs’ (i.e., information sources), which are the ‘local representatives’ of various distal objects and events in the organism’s environment. To function more or less efficiently, the organism needs to accumulate, combine, and substitute the information it derives from the available ‘cues’ and ‘signs,’ so that it can adequately adjust its means to achieve its behavioral goals (e.g., finding food or shelter). However, since the environment is inherently probabilistic and only partly predictable, the organism continually needs to adjust its assumptions about the state of the environment based on the available information sources. Another example is given by Barker (1968) , whose concept of ‘behavior settings’ (see also Heft, 2001 ) is key in describing how the environment shapes the frequency and occurrence of human cognition and behavior. Important to behavior settings is that they are the product of the collective actions of a group of individuals. Their geographical location can be specified (e.g., the supermarket, the cinema, etc.), and they have clear temporal and physical boundaries (e.g., opening hours, a door to enter and exit the building). Behavior settings are ‘independent’ of an individual’s subjective experience, yet what goes on inside any behavior setting is characterized by a high degree of interdependency and equivalence of actions between individuals (e.g., most people who are inside a supermarket are shopping for groceries and people in cinemas are watching movies). Another ‘classic’ example of a theory of the environment can be found in J. J. Gibson’s book The Ecological Approach to Visual Perception (1979/2014). According to Gibson, there exists a strong mutuality and reciprocity between the organism and its environment. He introduced the concept of ‘affordances’ to explain how the inherent ‘meaning’ of things (i.e., functional significance to the individual) can be directly perceived by an individual perceiver and how this ‘information’ shapes the possibilities for potential actions and experiences. For example, a sufficiently firm and smooth surface may be walk-on-able, run-on-able, or dance-on-able, whereas a rough surface cluttered with obstacles does not afford such actions ( Heft, 2001 ). In short, affordances are properties of an organism-environment system. They are perceiver-relative functional qualities of an object, event or place in the environment and they are dependent on the particular features of the environment and their relationships with the functional capabilities of a particular individual (for more in-depth discussions, see e.g., Heft, 2001 ; Stoffregen, 2003 ).

In order to describe and specify the environment and its relation to cognitive and behavioral functioning, we may draw on these scholars to guide us in a more specific direction. While we do not specifically recommend any of these perspectives, we think they are illuminating because these scholars motivate us to ask questions such as: What is the specific functional context of the cognitive and behavioral processes one is interested in? What are the relevant variables and conditions in this context given one’s focus of inquiry and level of analysis? What do we know or assume to know about the range and distribution of these variables and conditions? And how can these variables and conditions be represented in experimental designs to study specific patterns of cognitive and behavioral functioning? In order to answer some these questions, several researchers have emphasized the importance of first observing how people behave in everyday situations prior to experimentation. For example, Kingstone et al. (2008) advocated for an approach called Cognitive Ethology , which proposes that researchers should first observe how people behave in everyday situations before moving into the laboratory. In a similar vein, Adolph (2019) proposes that researchers should start with a rich description of the behaviors they are interested in order to first identify the “essential invariants” of these behaviors (p. 187).

The Field of Social Attention: Away From the Real-World and Toward Specificity About Context

To exemplify how some of the ideas outlined above may be useful to researchers, we will apply these ideas to a research topic of our interest: social attention. The field of social attention, as briefly discussed previously, is primarily focused on how attention is influenced by socially relevant objects, events, and situations, most notably, interactions with other social agents. In recent decades, it has been argued extensively that the experimental arrangements used by researchers in this field need more ‘ecological validity’ in order to adequately study the relevant characteristics of social attention in the ‘real-world’ ( Risko et al., 2012 , 2016 ; Schilbach et al., 2013 ; Caruana et al., 2017 ; Macdonald and Tatler, 2018 ; Shamay-Tsoory and Mendelsohn, 2019 ). In the light of these concerns, several researchers have advocated to study “real-world social attention” ( Risko et al., 2016 , p. 1) and “real-world social interaction” ( Macdonald and Tatler, 2018 , p. 1; see also Shamay-Tsoory and Mendelsohn, 2019 ). One example of this is given by Macdonald and Tatler (2018) . In this study, Macdonald and Tatler (2018) investigated how social roles given to participants influenced their social gaze behavior during a collaborative task: baking a cake together. Participants were either not given explicit social roles, or they were given a ‘Chef’ or ‘Gatherer’ role. Macdonald and Tatler (2018) showed that, regardless of whether social roles were assigned or not, participants did not gaze at their cake-baking partners very often while carrying out the task. After comparing their results with other so-called ‘real-world interaction studies’ (e.g., Laidlaw et al., 2011 ; Wu et al., 2013 ), the authors stated that: “we are not able to generalize about the specific amount of partner gaze during any given real-world interaction” ( Macdonald and Tatler, 2018 , p. 2171). We think that this statement clearly illustrates how the use of ‘real-world’ and ‘real life’ labels may lead to misleading and potentially counterproductive conclusions, as it seems to imply that ‘real-world interactions’ encompass a clearly defined category of behaviors. However, as argued previously, these so-called ‘real-world interactions’ are not a clearly defined category of behaviors. Instead, statements about generalizability need to be considered within a more constrained and carefully defined context (cf. Brunswik, 1956 ; Simons et al., 2017 ). This would make it more clear what researchers are talking about instead of subsuming studies under the big umbrella of the ‘real-world.’ For example, if the goal is to study how the cognitive and behavioral processes of social attention are influenced by different contexts and situations, researchers need to specify social gaze behavior as a function of these different contexts and situations.

Thus, instead of studying ‘real-world’ social attention in the context of ‘real-world’ social interactions, researchers should first try to describe and understand cake-baking attention ( Macdonald and Tatler, 2018 ), sharing-a-meal attention ( Wu et al., 2013 ), waiting-room attention ( Laidlaw et al., 2011 ), walking-on-campus attention ( Foulsham et al., 2011 ), Lego-block-building attention ( Macdonald and Tatler, 2013 ), playing-word-games attention ( Ho et al., 2015 ), interviewee-attention ( Freeth et al., 2013 ), and garage-sale attention ( Rubo and Gamer, 2018 ). By doing so, we may begin to understand the context-generic and context-specific aspects of attentional processes, allowing for a more sophisticated theory of social attention. These examples not only show the wide variety of behavioral tasks and contexts that are possible to study in relation to social attention, they also show that uncritical references to ‘ecological validity’ a.k.a. ‘real-worldliness’ are not very helpful to specify the relevant characteristics of particular behavioral contexts.

There are also good examples where researchers have been more explicit about the specific characteristics of social situations that they are interested in. Researchers in the field of social attention have, for example, tried to unravel the different functions of gaze behavior. One important function of gaze behavior is to acquire visual information from the world, however, within a social context, gaze may also signal important information to others which may be used to initiate and facilitate social interaction (see e.g., Gobel et al., 2015 ; Risko et al., 2016 ). In a series of experiments, researchers have systematically varied whether, and the degree to which social interaction between two people was possible, and measured how gaze was modulated as a function of the social context ( Laidlaw et al., 2011 ; Gobel et al., 2015 ; Gregory and Antolin, 2019 ; Holleman et al., 2020 ). In other studies, researchers have been explicit about the task-demands and social contexts that elicit specific patterns of gaze behavior, for example, in the context of face-to-face interactions and conversational exchanges ( Ho et al., 2015 ; Hessels et al., 2019 ). We think that, if researchers would try to be more explicit in their descriptions of task-demands and social contexts in relation to gaze, this may prove to be a solid basis for a more sophisticated theory of social attention, yet such work remains challenging (for a recent review, see Hessels, in press ).

We have argued that the ‘real-world approach’ and its call for ecological validity has several problems. The concept of ecological validity itself is seldom defined and interpretations differ among researchers. We believe that references to ecological validity and the ‘real-world’ can become superfluous if researchers would clearly specify and describe the particular contexts of behavior in which they are interested. This will be a more constructive way to uncover the context-specific and context-generic principles of cognition and behavior. As a final note, we hope that editors and reviewers will safeguard journals from publishing papers where terms such as ‘ecological validity’ and the ‘real-world’ are used without specification.

Author Contributions

GH and RH drafted the manuscript. RH, IH, and CK edited and revised the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding. GH and RH were supported by the Consortium on Individual Development (CID). CID is funded through the Gravitation programme of the Dutch Ministry of Education, Culture, and Science and the NWO (grant no. 024.001.003 awarded to author CK).

• Aanstoos C. M. (1991). Experimental psychology and the challenge of real life. Am. Psychol. 1 : 77 10.1037/0003-066x.46.1.77 [ CrossRef ] [ Google Scholar ]
• Adami C. (2002). What is complexity? Bioessays 24 1085–1094. [ PubMed ] [ Google Scholar ]
• Adolph K. E. (2019). “ Ecological validity: mistaking the lab for real life ,” in My Biggest Research Mistake: Adventures and Misadventures in Psychological Research , Ed. Sternberg R. (New York, NY: Sage; ), 187–190. 10.4135/9781071802601.n58 [ CrossRef ] [ Google Scholar ]
• Alderman N., Burgess P. W., Knight C., Henman C. (2003). Ecological validity of a simplified version of the multiple errands shopping test. J. Int. Neuropsychol. Soc. 9 31–44. 10.1017/s1355617703910046 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Anderson C. A., Lindsay J. J., Bushman B. J. (1999). Research in the psychological laboratory: truth or triviality? Curr. Direct. Psychol. Sci. 8 3–9. 10.1111/1467-8721.00002 [ CrossRef ] [ Google Scholar ]
• Araujo D., Davids K., Passos P. (2007). Ecological validity, representative design, and correspondence between experimental task constraints and behavioral setting: comment on Rogers. Kadar, and Costall (2005). Ecol. Psychol. 19 69–78. 10.1080/10407410709336951 [ CrossRef ] [ Google Scholar ]
• Ashcraft M., Radvansky G. (2009). Cognition , 5th Edn Upper Saddle River, NJ: Pearson Education, Inc. [ Google Scholar ]
• Aspland H., Gardner F. (2003). Observational measures of parent-child interaction: an introductory review. Child Adolesc. Mental Health 8 136–143. 10.1111/1475-3588.00061 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Banaji M. R., Crowder R. G. (1989). The bankruptcy of everyday memory. Am. Psychol. 44 : 1185 10.1037/0003-066x.44.9.1185 [ CrossRef ] [ Google Scholar ]
• Barker R. G. (1966). “ On the nature of the environment ,” in The Psychology of Egon Brunswik , ed. Hammond K. R. (New York: Holt, Rinehart and Winston; ). [ Google Scholar ]
• Barker R. G. (1968). Ecological Psychology: Concepts and Methods for Studying the Environment of Human Behavior. Stanford, CA: Stanford University Press. [ Google Scholar ]
• Berkowitz L., Donnerstein E. (1982). External validity is more than skin deep: Some answers to criticisms of laboratory experiments. Am. Psychol. 37 : 245 10.1037/0003-066x.37.3.245 [ CrossRef ] [ Google Scholar ]
• Birmingham E., Ristic J., Kingstone A. (2012). “ Investigating social attention: a case for increasing stimulus complexity in the laboratory ,” in Cognitive Neuroscience, Development, and Psychopathology: Typical and Atypical Developmental Trajectories of Attention , eds Burack J. A., Enns J. T., Fox N. A. (Oxford University Press: ), 251–276. 10.1093/acprof:oso/9780195315455.003.0010 [ CrossRef ] [ Google Scholar ]
• Blanco-Elorrieta E., Pylkkänen L. (2018). Ecological validity in bilingualism research and the bilingual advantage. Trends Cogn. Sci. 22 1117–1126. 10.1016/j.tics.2018.10.001 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bronfenbrenner U. (1977). Toward an experimental ecology of human development. Am. Psychol. 32 : 513 10.1037/0003-066x.32.7.513 [ CrossRef ] [ Google Scholar ]
• Brunswik E. (1943). Organismic achievement and environmental probability. Psychol. Rev. 50 : 255 10.1037/h0060889 [ CrossRef ] [ Google Scholar ]
• Brunswik E. (1949). Remarks on functionalism in perception. J. Pers. 18 56–65. 10.1111/j.1467-6494.1949.tb01233.x [ CrossRef ] [ Google Scholar ]
• Brunswik E. (1952). The Conceptual Framework of Psychology. Chicago: University of Chicago Press. [ Google Scholar ]
• Brunswik E. (1955). Representative design and probabilistic theory in a functional psychology. Psychol. Rev. 62 : 193 . 10.1037/h0047470 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brunswik E. (1956). Perception and the Representative Design of Psychological Experiments. Berkeley: University of California Press. [ Google Scholar ]
• Campbell D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychol. Bull. 54 : 297 . 10.1037/h0040950 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Caro T., Stankowich T. (2015). Concordance on zebra stripes: a comment on Larison et al.(2015). R. Soc. Open Sci. 2 : 150323 . 10.1098/rsos.150323 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Caruana N., McArthur G., Woolgar A., Brock J. (2017). Simulating social interactions for the experimental investigation of joint attention. Neurosci. Biobehav. Rev. 74 115–125. 10.1016/j.neubiorev.2016.12.022 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cohen G., Conway M. A. (2007). Memory in the Real World. Abingdon: Psychology Press. [ Google Scholar ]
• Danziger K. (1994). Constructing the Subject: Historical Origins of Psychological Research. Cambridge: Cambridge University Press. [ Google Scholar ]
• Davids K. (1988). Ecological validity in understanding sport performance: some problems of definition. Quest 40 126–136. 10.1080/00336297.1988.10483894 [ CrossRef ] [ Google Scholar ]
• Dhami M. K., Hertwig R., Hoffrage U. (2004). The role of representative design in an ecological approach to cognition. Psychol. Bull. 130 : 959 . 10.1037/0033-2909.130.6.959 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Dunlosky J., Bottiroli S., Hartwig M. (2009). “ Sins committed in the name of ecological validity: A call for representative design in education science ,” in Handbook of Metacognition in Education , eds Hacker D. J., Dunlosky J., Graesser A. C. (Abingdon: Routledge; ), 442–452. [ Google Scholar ]
• Eaton W. O., Clore G. L. (1975). Interracial imitation at a summer camp. J. Pers. Soc. Psychol. 32 : 1099 10.1037/0022-3514.32.6.1099 [ CrossRef ] [ Google Scholar ]
• Edmonds B. (1995). “ What is complexity?-the philosophy of complexity per se with application to some examples in evolution ,” in The Evolution of Complexity , Ed. Bonner J. T. (Dordrecht: Kluwer; ). [ Google Scholar ]
• Foulsham T., Walker E., Kingstone A. (2011). The where, what and when of gaze allocation in the lab and the natural environment. Vis. Res. 51 1920–1931. 10.1016/j.visres.2011.07.002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fredericksen R., Bex P. J., Verstraten F. A. (1997). How big is a Gabor patch, and why should we care? JOSA A 14 1–12. [ PubMed ] [ Google Scholar ]
• Freeth M., Foulsham T., Kingstone A. (2013). What affects social attention? Social presence, eye contact and autistic traits. PLoS One 8 : e53286 . 10.1371/journal.pone.0053286 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Frischen A., Bayliss A. P., Tipper S. P. (2007). Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychol. Bull. 133 : 694 . 10.1037/0033-2909.133.4.694 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gell-Mann M. (1995). What is complexity? Remarks on simplicity and complexity by the Nobel Prize-winning author of The Quark and the Jaguar. Complexity 1 16–19. 10.1002/cplx.6130010105 [ CrossRef ] [ Google Scholar ]
• Gibson J. J. (1950). The Perception of the Visual World. Cambridge: Houghton Mifflin Company. [ Google Scholar ]
• Gibson J. J. (1970). On the relation between hallucination and perception. Leonardo 3 425–427. [ Google Scholar ]
• Gibson J. J. (2014). The Ecological Approach to Visual Perception: Classic Edition. New York, NY: Psychology Press. (Original date of publication 1979). [ Google Scholar ]
• Gillis J., Schneider C. (1966). “ The historical preconditions of representative design ,” in The Psychology of Egon Brunswik , ed. Hammond K. R. (New York, NY: Holt, Rinehart & Winston, Inc; ), 204–236. [ Google Scholar ]
• Gobel M. S., Kim H. S., Richardson D. C. (2015). The dual function of social gaze. Cognition 136 359–364. 10.1016/j.cognition.2014.11.040 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Goodwin C. J. (2015). A History of Modern Psychology , 5 Edn Hoboken, NJ: John Wiley & Sons. [ Google Scholar ]
• Greenwald A. G. (1976). Within-subjects designs: to use or not to use? Psychol. Bull. 83 : 314 10.1037/0033-2909.83.2.314 [ CrossRef ] [ Google Scholar ]
• Gregory N. J., Antolin J. V. (2019). Does social presence or the potential for interaction reduce social gaze in online social scenarios? Introducing the “live lab” paradigm. Q. J. Exp. Psychol. 72 779–791. 10.1177/1747021818772812 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hammond K. R. (1998). Ecological Validity: Then and Now. Available online at: http://www.brunswik.org/notes/essay2.html (accessed April 1, 2020). [ Google Scholar ]
• Hammond K. R., Stewart T. R. (2001). The Essential Brunswik: Beginnings, Explications, Applications. New York, NY: Oxford University Press. [ Google Scholar ]
• Hatfield G. (2002). Psychology, philosophy, and cognitive science: reflections on the history and philosophy of experimental psychology. Mind Lang. 17 207–232. 10.1111/1468-0017.00196 [ CrossRef ] [ Google Scholar ]
• Heft H. (2001). Ecological Psychology in Context: James Gibson, Roger Barker, and the Legacy of William James’s Radical Empiricism. Hove: Psychology Press. [ Google Scholar ]
• Heft H. (2013). An ecological approach to psychology. Rev. Gen. Psychol. 17 162–167. 10.1037/a0032928 [ CrossRef ] [ Google Scholar ]
• Heinrichs R. W. (1990). Current and emergent applications of neuropsychological assessment: problems of validity and utility. Prof. Psychol. 21 : 171 10.1037/0735-7028.21.3.171 [ CrossRef ] [ Google Scholar ]
• Hessels R. S. (in press). How does gaze to faces support face-to-face interaction? A review and perspective. Psychonom. Bull. Rev. 10.31219/osf.io/8zta5 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hessels R. S., Holleman G. A., Kingstone A., Hooge I. T. C., Kemner C. (2019). Gaze allocation in face-to-face communication is affected primarily by task structure and social context, not stimulus-driven factors. Cognition 184 28–43. 10.1016/j.cognition.2018.12.005 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hessels R. S., van Doorn A. J., Benjamins J. S., Holleman G. A., Hooge I. T. C. (2020). Task-related gaze control in human crowd navigation. Attent. Percept. Psychophys. 10.3758/s13414-019-01952-9 [Online ahead of print] [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Higginson C. I., Arnett P. A., Voss W. D. (2000). The ecological validity of clinical tests of memory and attention in multiple sclerosis. Arch. Clin. Neuropsychol. 15 185–204. 10.1016/s0887-6177(99)00004-9 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hilton W. (1920). Applied Psychology: Driving Power of Thought. The Society of Applied Psychology Available online at: http://www.gutenberg.org/files/33076/33076-h/33076-h.htm (accessed April 1, 2020). [ Google Scholar ]
• Ho S., Foulsham T., Kingstone A. (2015). Speaking and listening with the eyes: gaze signaling during dyadic interactions. PLoS One 10 : e0136905 . 10.1371/journal.pone.0136905 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hoc J.-M. (2001). Towards ecological validity of research in cognitive ergonomics. Theor. Issues Ergon. Sci. 2 278–288. 10.1371/journal.pone.0184488 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hogarth R. M. (2005). The challenge of representative design in psychology and economics. J. Econ. Methodol. 12 253–263. 10.1177/0269216311399663 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Holleman G. A., Hessels R. S., Kemner C., Hooge I. T. C. (2020). Implying social interaction and its influence on gaze behavior to the eyes. PLoS One 15 : e0229203 . 10.1371/journal.pone.0229203 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Holleman G. A., Hooge I. T. C., Kemner C., Hessels R. S. (in press). The reality of ‘real-life’ neuroscience: a commentary on Shamay-Tsoory & Mendelsohn. Perspect. Psychol. Sci. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Jack R. E., Schyns P. G. (2017). Toward a social psychophysics of face communication. Annu. Rev. Psychol. 68 269–297. 10.1146/annurev-psych-010416-044242 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jenkins J. J. (1974). Remember that old theory of memory? Well, forget it. Am. Psychol. 29 : 785 10.1037/h0037399 [ CrossRef ] [ Google Scholar ]
• Johnston P., Molyneux R., Young A. W. (2014). The N170 observed ‘in the wild’: robust event-related potentials to faces in cluttered dynamic visual scenes. Soc. Cogn. Affect. Neurosci. 10 938–944. 10.1093/scan/nsu136 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kingstone A., Smilek D., Eastwood J. D. (2008). Cognitive ethology: a new approach for studying human cognition. Br. J. Psychol. 99 317–340. 10.1348/000712607x251243 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Koehler J. J. (1996). The base rate fallacy reconsidered: descriptive, normative, and methodological challenges. Behav. Brain Sci. 19 1–17. 10.1017/s0140525x00041157 [ CrossRef ] [ Google Scholar ]
• Krakauer J. W., Ghazanfar A. A., Gomez-Marin A., MacIver M. A., Poeppel D. (2017). Neuroscience needs behavior: correcting a reductionist bias. Neuron 93 480–490. 10.1016/j.neuron.2016.12.041 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kruglanski A. W. (1975). The two meanings of external invalidity. Hum. Relat. 28 653–659. 10.1177/001872677502800704 [ CrossRef ] [ Google Scholar ]
• Laidlaw K. E., Foulsham T., Kuhn G., Kingstone A. (2011). Potential social interactions are important to social attention. Proc. Natl. Acad. Sci. U.S.A. 108 5548–5553. 10.1073/pnas.1017022108 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Langton S. R., Watt R. J., Bruce V. (2000). Do the eyes have it? Cues to the direction of social attention. Trends Cogn. Sci. 4 50–59. 10.1016/s1364-6613(99)01436-9 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lappi O. (2015). Eye tracking in the wild: the good, the bad and the ugly. J. Eye Mov. Res. 8 : 1 . 10.1016/j.dcn.2019.100710 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Larison B., Harrigan R. J., Thomassen H. A., Rubenstein D. I., Chan-Golston A. M., Li E., et al. (2015). How the zebra got its stripes: a problem with too many solutions. R. Soc. Open Science 2 : 140452 . 10.1098/rsos.140452 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lewkowicz D. J. (2001). The concept of ecological validity: what are its limitations and is it bad to be invalid? Infancy 2 437–450. 10.1207/s15327078in0204_03 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Macdonald R. G., Tatler B. W. (2013). Do as eye say: gaze cueing and language in a real-world social interaction. J. Vis. 13 1–12. 10.1167/13.4.6 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Macdonald R. G., Tatler B. W. (2018). Gaze in a real-world social interaction: a dual eye-tracking study. Q. J. Exp. Psychol. 71 2162–2173. 10.1177/1747021817739221 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mills W. (1899). The nature of animal intelligence and the methods of investigating it. Psychol. Rev. 6 : 262 10.1037/h0074808 [ CrossRef ] [ Google Scholar ]
• Mook D. G. (1983). In defense of external invalidity. Am. Psychol. 38 : 379 10.1037/0003-066x.38.4.379 [ CrossRef ] [ Google Scholar ]
• Neisser U. (1976). Cognition and Reality: Principles and Implications Of Cognitive Psychology. San Fransisco, CA: W. H. Freeman and Company. [ Google Scholar ]
• Neisser U. (1991). A case of misplaced nostalgia. Am. Psychol. 46 :34–36. 10.1037/0003-066x.46.1.34 [ CrossRef ] [ Google Scholar ]
• Osborne-Crowley K. (2020). Social cognition in the real world: reconnecting the study of social cognition with social reality. Rev. Gen. Psychol. 1–15. 10.4324/9781315648156-1 [ CrossRef ] [ Google Scholar ]
• Parsons T. D. (2015). Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci. 9 : 660 . 10.3389/fnhum.2015.00660 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Parsons T. D., Carlew A. R., Magtoto J., Stonecipher K. (2017). The potential of function-led virtual environments for ecologically valid measures of executive function in experimental and clinical neuropsychology. Neuropsychol. Rehabil. 27 777–807. 10.1080/09602011.2015.1109524 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Peelen M. V., Kastner S. (2014). Attention in the real world: toward understanding its neural basis. Trends Cogn. Sci. 18 242–250. 10.1016/j.tics.2014.02.004 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Puce A., Bertenthal B. I. (2015). The Many Faces of Social Attention: Behavioral and Neural Measures , eds Puce A., Bertenthal B. I. (Switzerland: Springer; ). [ Google Scholar ]
• Richter J. N., Hochner B., Kuba M. J. (2016). Pull or push? Octopuses solve a puzzle problem. PLoS One 11 : e0152048 . 10.1371/journal.pone.0152048 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Risko E. F., Laidlaw K., Freeth M., Foulsham T., Kingstone A. (2012). Social attention with real versus reel stimuli: toward an empirical approach to concerns about ecological validity. Front. Hum. Neurosci. 6 : 143 . 10.3389/fnhum.2012.00143 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Risko E. F., Richardson D. C., Kingstone A. (2016). Breaking the fourth wall of cognitive science: real-world social attention and the dual function of gaze. Curr. Direct. Psychol. Sci. 25 70–74. 10.1177/0963721415617806 [ CrossRef ] [ Google Scholar ]
• Rogers S. D., Kadar E. E., Costall A. (2005). Gaze patterns in the visual control of straight-road driving and braking as a function of speed and expertise. Ecol. Psychol. 17 19–38. 10.1207/s15326969eco1701_2 [ CrossRef ] [ Google Scholar ]
• Rubo M., Gamer M. (2018). “ Virtual reality as a proxy for real-life social attention? ,” Paper presented at the Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. New York, NY. [ Google Scholar ]
• Schilbach L. (2015). Eye to eye, face to face and brain to brain: novel approaches to study the behavioral dynamics and neural mechanisms of social interactions. Curr. Opin. Behav. Sci. 3 130–135. 10.1016/j.cobeha.2015.03.006 [ CrossRef ] [ Google Scholar ]
• Schilbach L., Timmermans B., Reddy V., Costall A., Bente G., Schlicht T., et al. (2013). Toward a second-person neuroscience. Behav. Brain Sci. 36 393–414. 10.1017/s0140525x12000660 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schmuckler M. A. (2001). What is ecological validity? A dimensional analysis. Infancy 2 419–436. 10.1207/s15327078in0204_02 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shallice T., Burgess P. W. (1991). Deficits in strategy application following frontal lobe damage in man. Brain 114 727–741. 10.1093/brain/114.2.727 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shamay-Tsoory S. G., Mendelsohn A. (2019). Real-life neuroscience: an ecological approach to brain and behavior research. Perspect. Psychol. Sci. 14 841–859. 10.1177/1745691619856350 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Silverstein C. H., Stang D. J. (1976). Seating position and interaction in triads: a field study. Sociometry 39 166–170. [ Google Scholar ]
• Simons D. J., Levin D. T. (1998). Failure to detect changes to people during a real-world interaction. Psychonom. Bull. Rev. 5 644–649. 10.3758/bf03208840 [ CrossRef ] [ Google Scholar ]
• Simons D. J., Shoda Y., Lindsay D. S. (2017). Constraints on generality (COG): a proposed addition to all empirical papers. Perspect. Psychol. Sci. 12 1123–1128. 10.1177/1745691617708630 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Smilek D., Birmingham E., Cameron D., Bischof W., Kingstone A. (2006). Cognitive ethology and exploring attention in real-world scenes. Brain Res. 1080 101–119. 10.1016/j.brainres.2005.12.090 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Smith P. W., Feinberg R. A., Burns D. J. (1998). An examination of classical conditioning principles in an ecologically valid advertising context. J. Market. Theor. Pract. 6 63–72. 10.1080/10696679.1998.11501789 [ CrossRef ] [ Google Scholar ]
• Sonkusare S., Breakspear M., Guo C. (2019). Naturalistic stimuli in neuroscience: critically acclaimed. Trends Cogn. Sci. 23 699–714. 10.1016/j.tics.2019.05.004 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Stoffregen T. A. (2003). Affordances as properties of the animal-environment system. Ecol. Psychol. 15 115–134. 10.1016/j.humov.2019.01.002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thorndike E. (1899). A reply to “The nature of animal intelligence and the methods of investigating it”. Psychol. Rev. 6 412–420. 10.1037/h0073289 [ CrossRef ] [ Google Scholar ]
• Thorndike E. (2017). Animal Intelligence: Experimental Studies. Abingdon: Routledge. [ Google Scholar ]
• Todorović A. (2016) What’s in a Gabor Patch? Vol. 2016 Available online at: http://neuroanatody.com/2016/05/whats-in-a-gabor-patch/ (accessed April 1, 2020). [ Google Scholar ]
• Tolman E. C., Brunswik E. (1935). The organism and the causal texture of the environment. Psychol. Rev. 42 : 43 10.1037/h0062156 [ CrossRef ] [ Google Scholar ]
• Valtakari N. V., Hooge I. T. C., Benjamins J. S., Keizer A. (2019). An eye-tracking approach to Autonomous sensory meridian response (ASMR): the physiology and nature of tingles in relation to the pupil. PLoS One 14 : e226692 . 10.1371/journal.pone.0226692 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wilson B. A. (1993). Ecological validity of neuropsychological assessment: do neuropsychological indexes predict performance in everyday activities? Appl. Prevent. Psychol. 2 209–215. 10.1016/s0962-1849(05)80091-5 [ CrossRef ] [ Google Scholar ]
• Winograd E. (1988). “ Continuities between ecological and laboratory approaches to memory ,” in Emory Symposia in Cognition, 2. Remembering Reconsidered: Ecological and Traditional Approaches to the Study of Memory eds Neisser U., Winograd E. (Cambridge: Cambridge University Press; ), 11–20. 10.1017/cbo9780511664014.003 [ CrossRef ] [ Google Scholar ]
• Wu D. W.-L., Bischof W. F., Kingstone A. (2013). Looking while eating: the importance of social context to social attention. Sci. Rep. 3 : 2356 . 10.1038/srep02356 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

We couldn’t find any results matching your search.

Please try using other words for your search or explore other sections of the website for relevant information.

We’re sorry, we are currently experiencing some issues, please try again later.

Our team is working diligently to resolve the issue. Thank you for your patience and understanding.

News & Insights

Is Now a Good Time to Buy Bitcoin? (Updated 2024)

April 15, 2024 — 04:30 pm EDT

Written by Melissa Pistilli for Investing News Network  ->

Bitcoin is prone to price volatility, with wide swings to the upside and downside. The most recent upswing comes alongside growing institutional demand for the cryptocurrency as an attractive asset class.

Bitcoin's value has rallied over the last few quarters, increasing from about US$26,000 in mid-September 2023 to an all-time high of around US$73,000 in mid-March of this year. Of course, there have been bumps in the road — in January, prices sank to US$39,000 despite the launch of the the first US spot Bitcoin exchange-traded funds (ETFs). But as the sector gets closer to the next halving event , Bitcoin attention has once again gone hyperbolic.

Will this upward trajectory continue, or is Bitcoin's value likely to start plummeting again in the immediate future?

That's not an easy question to answer, and buying Bitcoin isn't a simple decision. Before you enter the market, you need to understand both Bitcoin and the wider crypto market. Read on to learn the basics.

​What gives Bitcoin its value?

Bitcoin was the world's first cryptocurrency, created in January 2009 by the mysterious Satoshi Nakamoto.

Conceived as a virtual alternative to fiat currency, Bitcoin is built atop blockchain technology, which it uses for both validation and security. Blockchain itself is a distributed digital ledger of transactions, operating through a combination of private keys, public keys and network consensus.

The best analogy to explain how this works in practice involves Google Docs . Imagine a document that's shared with a group of collaborators. Everyone has access to the same document, and each collaborator can see the edits other collaborators have made. If anyone makes an edit that the other collaborators don't approve of, they can roll it back.

Going back to Bitcoin, the virtual currency primarily validates transactions through proof of work . Also known as Bitcoin mining, this competitive and incredibly resource-intensive process is the means by which new Bitcoins are generated.

How it works is deceptively simple. Each Bitcoin transaction adds a new "block" to the ledger, identified by a 64 digit encrypted hexadecimal number known as a hash. Each block uses the block immediately preceding it to generate its hash, creating a ledger that theoretically cannot be tampered with. Bitcoin miners collectively attempt to guess the encrypted hex code for each new block — whoever correctly identifies the hash then validates the transaction and receives a small amount of Bitcoins as a reward.

From an investment perspective, Bitcoin toes the line between being a medium of exchange and a speculative digital asset. It also lacks any central governing body to regulate its distribution. As one might expect, these factors together make Bitcoin quite volatile, and therefore somewhat risky as an investment target.

As for the source of this volatility, Bitcoin's value is primarily influenced by five factors.

1. Supply and demand

It's widely known that no more than 21 million Bitcoins can be produced, and that's unlikely to happen before 2140 .

Only a certain number of Bitcoins are released each year, and this rate is reduced every four years by halving the reward for Bitcoin mining. The last of these "halvings" occurred in May 2020, and the next one is due this April. When it happens, there may be a significant increase in Bitcoin demand, largely driven by media coverage and investor interest.

Bitcoin demand is also strengthening in countries experiencing currency devaluation and high inflation.

It would be remiss not to mention that Bitcoin represents an ideal mechanism for supporting illicit activities — meaning that increasing cybercrime could itself be a demand driver.

2. Production costs

It's said that Bitcoin benefits from minimal production costs. This isn't exactly true, however. Solving even a single hash requires immense processing power, and it's believed that crypto mining collectively uses more electricity than some small countries . It's also believed that miners were largely responsible for the chip shortage experienced throughout the pandemic due to buying and burning out vast quantities of graphics cards.

These costs together have only a minimal influence on Bitcoin's overall value . The complexity of Bitcoin's hashing algorithms and the fact that they can vary wildly in complexity are far more impactful.

3. Competition

Bitcoin's cryptocurrency market share has sharply declined over the years. In 2017, it maintained a market share of over 80 percent . Bitcoin's current market share is just over 52 percent .

Despite that fall, Bitcoin remains the dominant force in the cryptocurrency market and is the marker by which many other cryptocurrencies determine their value. However, there is no guarantee that this will always remain the case. There are now scores of Bitcoin alternatives, known collectively as altcoins.

The most significant of these is Ethereum. Currently accounting for roughly 16 percent of the crypto market, Ethereum's market cap has increased by about 200 percent from last year. Some experts have suggested that Ethereum may even overtake Bitcoin , but others don't see that as a possibility in the near future.

4. Regulations

Bitcoin may itself be unregulated, but it is not immune to the effects of government legislation. For instance, China's 2021 ban of the cryptocurrency caused a sharp price drop, though it quickly rallied in the following months. The European Union has also attempted to ban Bitcoin in the past , and the US was recently accused of trying to do the same .

A ban in either region could be devastating for Bitcoin's overall value.

5. Public interest and media coverage

As with any speculative commodity, Bitcoin is greatly influenced by the court of public opinion.

Perhaps the best example of this occurred in 2021. At that time, a tweet from Tesla's (NASDAQ: TSLA ) Elon Musk caused Bitcoin's price to drop by 30 percent in a single day. This also wiped about US$365 billion off the cryptocurrency market.

Will Bitcoin keep going up?

Bitcoin is notoriously volatile, making it difficult to judge where the cryptocurrency is going next. Currently, Bitcoin sits at a price of roughly US$62,800 . This is a drastic improvement from December 2022, when it fell to US$17,000.

What caused such a steep decline? Per NASDAQ , Bitcoin's dramatic fall can be traced back to a few factors.

First and foremost were the harsh economic conditions of 2022. In order to combat supply shortages and inflation, the US Federal Reserve hiked interest rates aggressively. The war in Ukraine introduced even further uncertainty.

In response, most investors reined in discretionary spending and became less willing to speculate on risky or volatile assets. The catastrophic failure of crypto exchange FTX also left a sour taste in the mouths of investors to the tune of a nearly US$1 billion loss. Already wary of Bitcoin, many took FTX's failure as a sign that their initial instincts were correct.

That said, Bitcoin didn't actually take that long to recover from its post-FTX crash — by mid-January 2023, it had already rallied . After that, the cryptocurrency experienced one of its most promising years in recent memory. Its volatility has even dropped considerably, with Bloomberg noting that it's the lowest it's ever been since 2020.

Remember, however, that there's no such thing as a guaranteed investment, especially when it comes to cryptocurrencies. On the one hand, there's virtually no chance that Bitcoin will experience a crash to zero like Terra Luna . On the other hand, we also cannot take for granted that its value will continue to climb. It all depends on who you ask .

For example, veteran analyst Peter Brandt said in February that if Bitcoin could break past its previous high, the cryptocurrency could easily reach a new record of US$200,000 by September 2025. ARK Invest CEO Cathie Wood even believes Bitcoin could be worth over US$75 trillion by 2030 . Twitter co-founder Jack Dorsey is also a major proponent of Bitcoin, believing that it has the potential to one day replace fiat currency entirely.

Crypto industry specialists surveyed in early 2024 by UK fintech firm Finder shared somewhat more conservative estimates, although they are still quite promising and point to prices above US$100,000 in the near future. They believe that Bitcoin could rise to a value of roughly US$122,688 by 2025, and US$366,935 by 2030.

Not everyone is so optimistic about Bitcoin's prospects. Pav Hundal, lead market analyst at Swyftxt, has expressed concerns about Bitcoin's future in the context of continued geopolitical upheaval and economic uncertainty. Billionaire investor Warren Buffet, meanwhile, has not minced words regarding his opinion on Bitcoin and its future.

According to Buffet, Bitcoin is an unproductive asset with no unique value. He also feels that it doesn't count as a true currency — in fact, he called it “rat poison”. Moreover, he believes that the crypto market as a whole will end badly.

Regardless of whether you believe Bitcoin's proponents or naysayers, it's clear that it has some incredibly prominent backers in both the investment world and the wider business landscape. Business analytics platform MicroStrategy (NASDAQ: MSTR ) is by far the largest public company in the Bitcoin space, with 174,530 Bitcoins to its name. Marathon Digital Holdings (NASDAQ: MARA ) has 13,726 Bitcoins and Tesla holds 10,500 Bitcoins.

There are also plenty of individuals with large holdings , the most significant of which is believed to be Bitcoin's creator, Satoshi Nakamoto. Other prominent names include Michael Saylor, Cameron and Tyler Winklevoss and Tim Draper.

​How to smartly invest in Bitcoin?

Bitcoin is more stable than it's been in years, and the next halving is fast approaching. Taking current market conditions into account, now might well be the perfect time to invest, so long as you remain cognizant of the risks.

But if you opt to jump into the market … what comes next?

What is the process for buying Bitcoin?

The good news is that investing in Bitcoin is actually quite simple. If you're purchasing through a stockbroker, it's a similar process to buying shares of a company. Otherwise, you may need to gather your personal information and bank account details. It's recommended to secure your network with a VPN prior to performing any Bitcoin transactions.

The first step in purchasing Bitcoin is to join an exchange. Coinbase Global (NASDAQ: COIN ) and Binance are two of the most popular, but there's also Kraken and Bybit. If you're an advanced trader outside the US, you might consider Bitfinex.

Once you've chosen an exchange, you'll need a crypto wallet. Many first-time investors choose a software-based or "hot" wallet either maintained by their chosen crypto exchange or operated by a service provider. While simpler to set up and more convenient overall, hot wallets tend to be less secure as they can be compromised by data breaches.

Another option is a "cold" wallet — a specialized piece of hardware specifically designed to store cryptocurrency. It's basically a purpose-built flash drive. If you plan to invest large amounts in crypto, a cold wallet is the better option.

Once you've acquired and configured your wallet, you may choose to connect either the wallet or your crypto exchange account to your bank account. This is not strictly necessary, and some seasoned investors don't bother to do this.

Finally, with your wallet fully configured and your exchange account set up, it's time to place your order.

Best practises for investing in Bitcoin

The most important thing to remember about Bitcoin is that it is a high-risk asset. Never invest money that you aren't willing to lose. Treat Bitcoin as a means of slowly growing your existing wealth rather than an all-or-nothing gamble.

As with other investments, it's important to hedge your portfolio. Alongside Bitcoin, you may want to consider investing in other cryptocurrencies like Ethereum, or perhaps an altcoin. You may also want to explore other blockchain-based investments , given that even the stablest cryptocurrencies tend to be fairly volatile.

It's also key to ignore the hype surrounding cryptocurrencies. Recall how many people whipped themselves into a frenzy over non-fungible tokens in 2022. More than 95 percent of the NFTs created during that time are now worthless.

Make decisions based on your own market research and advice from trusted — and more importantly, certified — professionals. If you're putting up investment capital based on an influencer's tweets, you are playing with fire.

You should also start small. A good rule of thumb is not to dedicate more than 10 percent of your overall capital to cryptocurrency. Even that number could be high — again, it's all about moderation.

Make sure to prioritize cybersecurity as well. Cryptocurrencies are an immensely popular target for cybercriminals. In addition to maintaining a cold wallet, make sure you practise proper security hygiene. That means using a VPN and a password manager while also exercising mindfulness in how you browse the web and what you download.

Finally, make an effort to understand what cryptocurrencies are and how they work. One of the reasons Sam Bankman-Fried was able to run FTX as long as he did was because many of his investors didn't fully understand what they were putting their money into. Don't let yourself be fooled by buzzwords or lofty promises about Web3 and the metaverse.

Do your research into the technology behind it all. That way, you'll be far better equipped to recognize when something is a sound investment versus a bottomless money pit.

When should you buy and sell Bitcoin?

Generally speaking, Bitcoin is subject to the same rules as any investment. That is to say, you should always try to buy low and sell high. Unfortunately, given how dramatically Bitcoin's value tends to fluctuate, that's often easier said than done. Again, it's prudent to pay attention to the market and listen to the experts.

What is indirect crypto investing?

Given Bitcoin's volatility, it's understandable that you might be leery of making a direct investment. The good news is that you don't have to. You can indirectly invest into the crypto space through mutual funds, stocks and ETFs.

For more details, check out 11 Canadian Cryptocurrency ETFs and 5 Biggest Blockchain ETFs .

Do a bit of research and touch base with your stockbroker or financial advisor before you go in this direction.

​Investor takeaway

Bitcoin is a fascinating asset. Simultaneously a transactional tool and a speculative commodity, it's attracted the attention of investors almost since it first hit the market. Unfortunately, it's also incredibly volatile.

For that reason, while current market conditions are favorable for anyone considering buying Bitcoin, it is an asset you should purchase only at your own risk. Because while Bitcoin may have the potential for significant returns, you may also lose most of your investment. If that knowledge doesn't bother you, then by all means, purchase away.

Otherwise, there are better — less volatile — options for your capital.

FAQs for buying Bitcoin

What is a realistic bitcoin price prediction for 2025.

Reality and price predictions rarely match up as forecasters have no way of predicting major events like Russia's war with Ukraine or the COVID-19 pandemic. On top of that, the further away the time period, the less realistic the prediction will be.

As such, there is a massive range for 2025 Bitcoin price forecasts. As of April 2024, forecasts for where the Bitcoin price might land in 2025 range from US$74,456.13 to US$270,929.12 . We'll have to wait a a couple of years to see which are correct.

What does Cathie Wood say about Bitcoin?

ARK Invest CEO Cathie Wood is extremely bullish on Bitcoin, telling Bloomberg in February 2023 that her firm believes the cryptocurrency could reach a value of US$1 million by 2030 . A year later, Wood hiked her 2030 bitcoin price prediction astronomically to US$75 trillion .

This is an updated version of an article first published by the Investing News Network in 2023.

Don't forget to follow us @INN_Technology for real-time news updates!

Securities Disclosure: I, Melissa Pistilli, hold no direct investment interest in any company mentioned in this article.

The views and opinions expressed herein are the views and opinions of the author and do not necessarily reflect those of Nasdaq, Inc.

More Related Articles

This data feed is not available at this time.

Sign up for the TradeTalks newsletter to receive your weekly dose of trading news, trends and education. Delivered Wednesdays.

To add symbols:

• Type a symbol or company name. When the symbol you want to add appears, add it to My Quotes by selecting it and pressing Enter/Return.
• Copy and paste multiple symbols separated by spaces.

These symbols will be available throughout the site during your session.

Your symbols have been updated

Edit watchlist.

• Type a symbol or company name. When the symbol you want to add appears, add it to Watchlist by selecting it and pressing Enter/Return.

Opt in to Smart Portfolio

Smart Portfolio is supported by our partner TipRanks. By connecting my portfolio to TipRanks Smart Portfolio I agree to their Terms of Use .