strengths and weaknesses of the case study method

10 Case Study Advantages and Disadvantages

A case study in academic research is a detailed and in-depth examination of a specific instance or event, generally conducted through a qualitative approach to data.

The most common case study definition that I come across is is Robert K. Yin’s (2003, p. 13) quote provided below:

“An empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.”

Researchers conduct case studies for a number of reasons, such as to explore complex phenomena within their real-life context, to look at a particularly interesting instance of a situation, or to dig deeper into something of interest identified in a wider-scale project.

While case studies render extremely interesting data, they have many limitations and are not suitable for all studies. One key limitation is that a case study’s findings are not usually generalizable to broader populations because one instance cannot be used to infer trends across populations.

Case Study Advantages and Disadvantages

1. in-depth analysis of complex phenomena.

Case study design allows researchers to delve deeply into intricate issues and situations.

By focusing on a specific instance or event, researchers can uncover nuanced details and layers of understanding that might be missed with other research methods, especially large-scale survey studies.

As Lee and Saunders (2017) argue,

“It allows that particular event to be studies in detail so that its unique qualities may be identified.”

This depth of analysis can provide rich insights into the underlying factors and dynamics of the studied phenomenon.

2. Holistic Understanding

Building on the above point, case studies can help us to understand a topic holistically and from multiple angles.

This means the researcher isn’t restricted to just examining a topic by using a pre-determined set of questions, as with questionnaires. Instead, researchers can use qualitative methods to delve into the many different angles, perspectives, and contextual factors related to the case study.

We can turn to Lee and Saunders (2017) again, who notes that case study researchers “develop a deep, holistic understanding of a particular phenomenon” with the intent of deeply understanding the phenomenon.

3. Examination of rare and Unusual Phenomena

We need to use case study methods when we stumble upon “rare and unusual” (Lee & Saunders, 2017) phenomena that would tend to be seen as mere outliers in population studies.

Take, for example, a child genius. A population study of all children of that child’s age would merely see this child as an outlier in the dataset, and this child may even be removed in order to predict overall trends.

So, to truly come to an understanding of this child and get insights into the environmental conditions that led to this child’s remarkable cognitive development, we need to do an in-depth study of this child specifically – so, we’d use a case study.

4. Helps Reveal the Experiences of Marginalzied Groups

Just as rare and unsual cases can be overlooked in population studies, so too can the experiences, beliefs, and perspectives of marginalized groups.

As Lee and Saunders (2017) argue, “case studies are also extremely useful in helping the expression of the voices of people whose interests are often ignored.”

Take, for example, the experiences of minority populations as they navigate healthcare systems. This was for many years a “hidden” phenomenon, not examined by researchers. It took case study designs to truly reveal this phenomenon, which helped to raise practitioners’ awareness of the importance of cultural sensitivity in medicine.

5. Ideal in Situations where Researchers cannot Control the Variables

Experimental designs – where a study takes place in a lab or controlled environment – are excellent for determining cause and effect . But not all studies can take place in controlled environments (Tetnowski, 2015).

When we’re out in the field doing observational studies or similar fieldwork, we don’t have the freedom to isolate dependent and independent variables. We need to use alternate methods.

Case studies are ideal in such situations.

A case study design will allow researchers to deeply immerse themselves in a setting (potentially combining it with methods such as ethnography or researcher observation) in order to see how phenomena take place in real-life settings.

6. Supports the generation of new theories or hypotheses

While large-scale quantitative studies such as cross-sectional designs and population surveys are excellent at testing theories and hypotheses on a large scale, they need a hypothesis to start off with!

This is where case studies – in the form of grounded research – come in. Often, a case study doesn’t start with a hypothesis. Instead, it ends with a hypothesis based upon the findings within a singular setting.

The deep analysis allows for hypotheses to emerge, which can then be taken to larger-scale studies in order to conduct further, more generalizable, testing of the hypothesis or theory.

7. Reveals the Unexpected

When a largescale quantitative research project has a clear hypothesis that it will test, it often becomes very rigid and has tunnel-vision on just exploring the hypothesis.

Of course, a structured scientific examination of the effects of specific interventions targeted at specific variables is extermely valuable.

But narrowly-focused studies often fail to shine a spotlight on unexpected and emergent data. Here, case studies come in very useful. Oftentimes, researchers set their eyes on a phenomenon and, when examining it closely with case studies, identify data and come to conclusions that are unprecedented, unforeseen, and outright surprising.

As Lars Meier (2009, p. 975) marvels, “where else can we become a part of foreign social worlds and have the chance to become aware of the unexpected?”

Disadvantages

1. not usually generalizable.

Case studies are not generalizable because they tend not to look at a broad enough corpus of data to be able to infer that there is a trend across a population.

As Yang (2022) argues, “by definition, case studies can make no claims to be typical.”

Case studies focus on one specific instance of a phenomenon. They explore the context, nuances, and situational factors that have come to bear on the case study. This is really useful for bringing to light important, new, and surprising information, as I’ve already covered.

But , it’s not often useful for generating data that has validity beyond the specific case study being examined.

2. Subjectivity in interpretation

Case studies usually (but not always) use qualitative data which helps to get deep into a topic and explain it in human terms, finding insights unattainable by quantitative data.

But qualitative data in case studies relies heavily on researcher interpretation. While researchers can be trained and work hard to focus on minimizing subjectivity (through methods like triangulation), it often emerges – some might argue it’s innevitable in qualitative studies.

So, a criticism of case studies could be that they’re more prone to subjectivity – and researchers need to take strides to address this in their studies.

3. Difficulty in replicating results

Case study research is often non-replicable because the study takes place in complex real-world settings where variables are not controlled.

So, when returning to a setting to re-do or attempt to replicate a study, we often find that the variables have changed to such an extent that replication is difficult. Furthermore, new researchers (with new subjective eyes) may catch things that the other readers overlooked.

Replication is even harder when researchers attempt to replicate a case study design in a new setting or with different participants.

Comprehension Quiz for Students

Question 1: What benefit do case studies offer when exploring the experiences of marginalized groups?

a) They provide generalizable data. b) They help express the voices of often-ignored individuals. c) They control all variables for the study. d) They always start with a clear hypothesis.

Question 2: Why might case studies be considered ideal for situations where researchers cannot control all variables?

a) They provide a structured scientific examination. b) They allow for generalizability across populations. c) They focus on one specific instance of a phenomenon. d) They allow for deep immersion in real-life settings.

Question 3: What is a primary disadvantage of case studies in terms of data applicability?

a) They always focus on the unexpected. b) They are not usually generalizable. c) They support the generation of new theories. d) They provide a holistic understanding.

Question 4: Why might case studies be considered more prone to subjectivity?

a) They always use quantitative data. b) They heavily rely on researcher interpretation, especially with qualitative data. c) They are always replicable. d) They look at a broad corpus of data.

Question 5: In what situations are experimental designs, such as those conducted in labs, most valuable?

a) When there’s a need to study rare and unusual phenomena. b) When a holistic understanding is required. c) When determining cause-and-effect relationships. d) When the study focuses on marginalized groups.

Question 6: Why is replication challenging in case study research?

a) Because they always use qualitative data. b) Because they tend to focus on a broad corpus of data. c) Due to the changing variables in complex real-world settings. d) Because they always start with a hypothesis.

Lee, B., & Saunders, M. N. K. (2017). Conducting Case Study Research for Business and Management Students. SAGE Publications.

Meir, L. (2009). Feasting on the Benefits of Case Study Research. In Mills, A. J., Wiebe, E., & Durepos, G. (Eds.). Encyclopedia of Case Study Research (Vol. 2). London: SAGE Publications.

Tetnowski, J. (2015). Qualitative case study research design.  Perspectives on fluency and fluency disorders ,  25 (1), 39-45. ( Source )

Yang, S. L. (2022). The War on Corruption in China: Local Reform and Innovation . Taylor & Francis.

Yin, R. (2003). Case Study research. Thousand Oaks, CA: Sage.

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Holistic Education: Definition, Benefits & Limitations
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Humanism in Education: Definition, Pros & Cons
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Behaviorism in Education: Definition, Pros and Cons
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 31 Major Learning Theories in Education, Explained!

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Strengths and Weaknesses of Case Studies

There is no doubt that case studies are a valuable and important form of research for all of the industries and fields that use them. However, along with all their advantages, they also have some disadvantages. In this article we are going to look at both.

Advantages of Case Studies

Intensive Study

Case study method is responsible for intensive study of a unit. It is the investigation and exploration of an event thoroughly and deeply. You get a very detailed and in-depth study of a person or event. This is especially the case with subjects that cannot be physically or ethically recreated.

This is one of the biggest advantages of the Genie case. You cannot lock up a child for 13 years and deprive them of everything. That would be morally and ethically wrong in every single way. So when the opportunity presented itself, researchers could not look away. It was a once in a lifetime opportunity to learn about feral children.

Genie was a feral child. She was raised in completed isolation, with little human contact. Because of the abuse she withstood, she was unable to develop cognitively. From infancy she was strapped to a potty chair, and therefore never acquired the physicality needed for walking, running and jumping.

If Genie made a noise, her father beat her. Therefore, she learned to not make a noise. Once she was found, researchers studied her language skills, and attempted to find ways to get her to communicate. They were successful. While she never gained the ability to speak, she did develop other ways to communicate. However, the public soon lost interest in her case, and with that, the funds to conduct the study.

However, her case was extremely important to child development psychology and linguistic theory. Because of her, we know that mental stimulation is needed for proper development. We also now know that there is a "critical period" for the learning of language.

Developing New Research

Case studies are one of the best ways to stimulate new research. A case study can be completed, and if the findings are valuable, they can lead to new and advanced research in the field. There has been a great deal of research done that wouldn't have been possible without case studies.

An example of this is the sociological study Nickel and Dimed. Nickel and Dimed is a book and study done by Barbara Ehrenreich. She wanted to study poverty in America, and did so by living and working as a person living on minimum wage.

Through her experiment, she discovered that poverty was almost inescapable. As soon as she saved a little money, she was hit with a crisis. She might get sick, or her car might break down, all occurrences that can be destructive when a person doesn't have a safety net to fall back on.

It didn't matter where she lived or what she did. Working a minimum wage job gave her no chances for advancement or improvement whatsoever. And she did the experiment as a woman with no children to support.

This study opened a lot of eyes to the problem of the working poor in America. By living and working as the experiment, Ehrenreich was able to show first-hand data regarding the issues surrounding poverty. The book didn't end with any solutions, just suggestions for the reader and points for them to think about.

Using this case study information, new studies could be organized to learn better ways to help people who are fighting poverty, or better ways to help the working poor.

Contradicting Established Ideas or Theories

Oftentimes there are theories that may be questioned with case studies. For example, in the John/John case study, it was believed that gender and sexual identity were a construct of nurture, not nature.

John-John focused on a set of twin boys, both of whom were circumcised at the age of 6 months. One of the twin's circumcisions failed, causing irreparable damage to the penis. His parents were concerned about the sexual health of their son, so they contacted Dr. John Money for a solution.

Dr. Money believed that sexuality came from nurture, not nature, and that the injured baby, Bruce, could be raised as a girl. His penis was removed and he was sexually reassigned to become a girl. Bruce's name was changed to Brenda, and his parents decided to raise him as a girl.

In this case, Dr. Money was dishonest. He believed that gender could be changed, which has since been proven false. Brenda's parents were also dishonest, stating that the surgery was a success, when in fact that wasn't the case.

As Brenda grew up, she always acted masculine and was teased for it at school. She did not socialize as a girl, and did not identify as a female. When Brenda was 13 she learned the truth, and was incredibly relieved. She changed her name to David, and lived the rest of her life as a male.

This case proved that the general theory was wrong, and is still valuable, even though the study author was dishonest.

Giving New Insight

Case studies have the ability to give insight into phenomena that cannot be learned in any other way. An example of this is the case study about Sidney Bradford. Bradford was blind from the age of 10 months old, and regained his sight at the age of 52 from a corneal transplant.

This unique situation allowed researchers to better learn how perception and motion changes when suddenly given sight. They were able to better understand how colors and dimensions affect the human process. For what it is worth, Bradford continued to live and work with his eyes closed, as he found sight too stimulating.

Another famous study was the sociological study of Milgram.

Stanley Milgram did a study from 1960 to 1974 in which he studied the effects of social pressure. The study was set up as an independent laboratory. A random person would walk in, and agree to be a part of the study. He was told to act as a teacher, and ask questions to another volunteer, who was the learner.

The teacher would ask the learner questions, and whenever he answered incorrectly, the teacher was instructed to give the learner an electric shock. Each time the learner was wrong, the shock would be increased by 15 volts. What the teacher didn't know was that the learner was a part of the experiment, and that no shocks were being given. However, the learner did act as if they were being shocked.

If the teachers tried to quit, they were strongly pushed to continue. The goal of the experiment was to see whether or not any of the teachers would go up to the highest voltage. As it turned out, 65% of the teachers did.

This study opened eyes when it comes to social pressure. If someone tells you it is okay to hurt someone, at what point will the person back off and say "this is not ok!" And in this study, the results were the same, regardless of income, race, gender or ethnicity.

This study opened up the sociological world of understanding the divide between social pressure and morality.

Disadvantages of Case Studies

Inability to Replicate

As demonstrated with the Genie case study, many studies cannot be replicated, and therefore, cannot be corroborated. Because the studies cannot be replicated, it means the data and results are only valid for that one person. Now, one could infer that that results of the Genie study would be the same with other feral children, without additional studies we can never be 100% certain.

Also, Genie was a white, American female. We do not know whether someone with a different gender, race or ethnicity would have a different result.

Key Term! Hawthorne Effect

The effect in which people change their behavior when they are aware they are being observed.

Researcher Bias

When conducting a case study, it is very possible for the author to form a bias. This bias can be for the subject; the form of data collection, or the way the data is interpreted. This is very common, since it is normal for humans to be subjective. It is well known that Sigmund Freud, the father of psychology, was often biased in his case histories and interpretations.

The researcher can become close to a study participant, or may learn to identify with the subject. When this happens the researcher loses their perspective as an outsider.

No Classification

Any classification is not possible due to studying a small unit. This generalization of results is limited, since the study is only focusing on one small group. However, this isn't always a problem, especially if generalization is not one of the study's goals.

Time Intensive

Case studies can be very time consuming. The data collection process can be very intensive and long, and this is something new researchers are not familiar with. It takes a long period of time to develop a case study, and develop a detailed analysis.

Many studies also require the authors to immerse themselves in the case. For example, in the Genie case, the lead researchers spent an abnormal amount of time with Genie, since so few people knew how to handle her. David Rigler, one of the lead researchers, actually had Genie live with him and his family for years. Because of this attachment, many questioned the veracity of the study data.

Possibility of Errors

Case study method may have errors of memory or judgment. Since reconstructing case history is based on memory, this can lead to errors. Also, how one person perceived the past could be different for another person, and this can and does lead to errors.

When considering various aspects of their lives, people tend to focus on issues that they find most important. This allows them to form a prejudice and can make them unaware of other possible options.

Ethical Issues

With small studies, there is always the question of ethics. At what point does a study become unethical? The Genie case was riddled with accusations of being unethical, and people still debate about it today.

Was it ethical to study Genie as deeply as she was studied?

Did Genie deserve to live out her life unbothered by researchers and academics trying to use her case to potentially further their careers?

At what point does the pursuit of scientific knowledge outweigh the right to a life free from research?

Also, because the researchers became so invested in the study, people questioned whether a researcher would report unethical behavior if they witnessed it.

Advantages and Disadvantages in Real-Life Studies

Two of these case studies are the Tylenol Scandal and the Genie language study.

Let's look at the advantages and disadvantages of these two studies.

Genie – Advantages

Uniqueness of study – Being able to study a feral child is a rare occurrence.

Genie – Disadvantages

Ethics - The lead researcher David Rigler provided a home for Genie, and was paid for being a foster parent. This is often seen as unethical, since Rigler had a financial interest in Genie and her case.

Tylenol – Advantages

Uniqueness of study – What happened to Tylenol was very unique and rare. While companies face crisis all the time, a public health crisis of this magnitude is very unique.

Tylenol – Disadvantages

• Course Catalog
• Group Discounts
• Gift Certificates
• For Libraries
• CEU Verification
• Medical Terminology
• Accounting Course
• Writing Basics
• QuickBooks Training
• Proofreading Class
• Sensitivity Training
• Excel Certificate
• Teach Online
• Terms of Service
• Privacy Policy

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews

Research question

• Conceptual framework
• Conceptual vs. theoretical framework

Data collection

• Qualitative research methods
• Focus groups
• Observational research

What is a case study?

Applications for case study research, what is a good case study, process of case study design, benefits and limitations of case studies.

• Ethnographical research
• Ethical considerations
• Confidentiality and privacy
• Power dynamics
• Reflexivity

Case studies

Case studies are essential to qualitative research , offering a lens through which researchers can investigate complex phenomena within their real-life contexts. This chapter explores the concept, purpose, applications, examples, and types of case studies and provides guidance on how to conduct case study research effectively.

Whereas quantitative methods look at phenomena at scale, case study research looks at a concept or phenomenon in considerable detail. While analyzing a single case can help understand one perspective regarding the object of research inquiry, analyzing multiple cases can help obtain a more holistic sense of the topic or issue. Let's provide a basic definition of a case study, then explore its characteristics and role in the qualitative research process.

Definition of a case study

A case study in qualitative research is a strategy of inquiry that involves an in-depth investigation of a phenomenon within its real-world context. It provides researchers with the opportunity to acquire an in-depth understanding of intricate details that might not be as apparent or accessible through other methods of research. The specific case or cases being studied can be a single person, group, or organization – demarcating what constitutes a relevant case worth studying depends on the researcher and their research question .

Among qualitative research methods , a case study relies on multiple sources of evidence, such as documents, artifacts, interviews , or observations , to present a complete and nuanced understanding of the phenomenon under investigation. The objective is to illuminate the readers' understanding of the phenomenon beyond its abstract statistical or theoretical explanations.

Characteristics of case studies

Case studies typically possess a number of distinct characteristics that set them apart from other research methods. These characteristics include a focus on holistic description and explanation, flexibility in the design and data collection methods, reliance on multiple sources of evidence, and emphasis on the context in which the phenomenon occurs.

Furthermore, case studies can often involve a longitudinal examination of the case, meaning they study the case over a period of time. These characteristics allow case studies to yield comprehensive, in-depth, and richly contextualized insights about the phenomenon of interest.

The role of case studies in research

Case studies hold a unique position in the broader landscape of research methods aimed at theory development. They are instrumental when the primary research interest is to gain an intensive, detailed understanding of a phenomenon in its real-life context.

In addition, case studies can serve different purposes within research - they can be used for exploratory, descriptive, or explanatory purposes, depending on the research question and objectives. This flexibility and depth make case studies a valuable tool in the toolkit of qualitative researchers.

Remember, a well-conducted case study can offer a rich, insightful contribution to both academic and practical knowledge through theory development or theory verification, thus enhancing our understanding of complex phenomena in their real-world contexts.

What is the purpose of a case study?

Case study research aims for a more comprehensive understanding of phenomena, requiring various research methods to gather information for qualitative analysis . Ultimately, a case study can allow the researcher to gain insight into a particular object of inquiry and develop a theoretical framework relevant to the research inquiry.

Why use case studies in qualitative research?

Using case studies as a research strategy depends mainly on the nature of the research question and the researcher's access to the data.

Conducting case study research provides a level of detail and contextual richness that other research methods might not offer. They are beneficial when there's a need to understand complex social phenomena within their natural contexts.

The explanatory, exploratory, and descriptive roles of case studies

Case studies can take on various roles depending on the research objectives. They can be exploratory when the research aims to discover new phenomena or define new research questions; they are descriptive when the objective is to depict a phenomenon within its context in a detailed manner; and they can be explanatory if the goal is to understand specific relationships within the studied context. Thus, the versatility of case studies allows researchers to approach their topic from different angles, offering multiple ways to uncover and interpret the data .

The impact of case studies on knowledge development

Case studies play a significant role in knowledge development across various disciplines. Analysis of cases provides an avenue for researchers to explore phenomena within their context based on the collected data.

This can result in the production of rich, practical insights that can be instrumental in both theory-building and practice. Case studies allow researchers to delve into the intricacies and complexities of real-life situations, uncovering insights that might otherwise remain hidden.

Types of case studies

In qualitative research , a case study is not a one-size-fits-all approach. Depending on the nature of the research question and the specific objectives of the study, researchers might choose to use different types of case studies. These types differ in their focus, methodology, and the level of detail they provide about the phenomenon under investigation.

Understanding these types is crucial for selecting the most appropriate approach for your research project and effectively achieving your research goals. Let's briefly look at the main types of case studies.

Exploratory case studies

Exploratory case studies are typically conducted to develop a theory or framework around an understudied phenomenon. They can also serve as a precursor to a larger-scale research project. Exploratory case studies are useful when a researcher wants to identify the key issues or questions which can spur more extensive study or be used to develop propositions for further research. These case studies are characterized by flexibility, allowing researchers to explore various aspects of a phenomenon as they emerge, which can also form the foundation for subsequent studies.

Descriptive case studies

Descriptive case studies aim to provide a complete and accurate representation of a phenomenon or event within its context. These case studies are often based on an established theoretical framework, which guides how data is collected and analyzed. The researcher is concerned with describing the phenomenon in detail, as it occurs naturally, without trying to influence or manipulate it.

Explanatory case studies

Explanatory case studies are focused on explanation - they seek to clarify how or why certain phenomena occur. Often used in complex, real-life situations, they can be particularly valuable in clarifying causal relationships among concepts and understanding the interplay between different factors within a specific context.

Intrinsic, instrumental, and collective case studies

These three categories of case studies focus on the nature and purpose of the study. An intrinsic case study is conducted when a researcher has an inherent interest in the case itself. Instrumental case studies are employed when the case is used to provide insight into a particular issue or phenomenon. A collective case study, on the other hand, involves studying multiple cases simultaneously to investigate some general phenomena.

Each type of case study serves a different purpose and has its own strengths and challenges. The selection of the type should be guided by the research question and objectives, as well as the context and constraints of the research.

The flexibility, depth, and contextual richness offered by case studies make this approach an excellent research method for various fields of study. They enable researchers to investigate real-world phenomena within their specific contexts, capturing nuances that other research methods might miss. Across numerous fields, case studies provide valuable insights into complex issues.

Critical information systems research

Case studies provide a detailed understanding of the role and impact of information systems in different contexts. They offer a platform to explore how information systems are designed, implemented, and used and how they interact with various social, economic, and political factors. Case studies in this field often focus on examining the intricate relationship between technology, organizational processes, and user behavior, helping to uncover insights that can inform better system design and implementation.

Health research

Health research is another field where case studies are highly valuable. They offer a way to explore patient experiences, healthcare delivery processes, and the impact of various interventions in a real-world context.

Case studies can provide a deep understanding of a patient's journey, giving insights into the intricacies of disease progression, treatment effects, and the psychosocial aspects of health and illness.

Asthma research studies

Specifically within medical research, studies on asthma often employ case studies to explore the individual and environmental factors that influence asthma development, management, and outcomes. A case study can provide rich, detailed data about individual patients' experiences, from the triggers and symptoms they experience to the effectiveness of various management strategies. This can be crucial for developing patient-centered asthma care approaches.

Other fields

Apart from the fields mentioned, case studies are also extensively used in business and management research, education research, and political sciences, among many others. They provide an opportunity to delve into the intricacies of real-world situations, allowing for a comprehensive understanding of various phenomena.

Case studies, with their depth and contextual focus, offer unique insights across these varied fields. They allow researchers to illuminate the complexities of real-life situations, contributing to both theory and practice.

Whatever field you're in, ATLAS.ti puts your data to work for you

Download a free trial of ATLAS.ti to turn your data into insights.

Understanding the key elements of case study design is crucial for conducting rigorous and impactful case study research. A well-structured design guides the researcher through the process, ensuring that the study is methodologically sound and its findings are reliable and valid. The main elements of case study design include the research question , propositions, units of analysis, and the logic linking the data to the propositions.

The research question is the foundation of any research study. A good research question guides the direction of the study and informs the selection of the case, the methods of collecting data, and the analysis techniques. A well-formulated research question in case study research is typically clear, focused, and complex enough to merit further detailed examination of the relevant case(s).

Propositions

Propositions, though not necessary in every case study, provide a direction by stating what we might expect to find in the data collected. They guide how data is collected and analyzed by helping researchers focus on specific aspects of the case. They are particularly important in explanatory case studies, which seek to understand the relationships among concepts within the studied phenomenon.

Units of analysis

The unit of analysis refers to the case, or the main entity or entities that are being analyzed in the study. In case study research, the unit of analysis can be an individual, a group, an organization, a decision, an event, or even a time period. It's crucial to clearly define the unit of analysis, as it shapes the qualitative data analysis process by allowing the researcher to analyze a particular case and synthesize analysis across multiple case studies to draw conclusions.

Argumentation

This refers to the inferential model that allows researchers to draw conclusions from the data. The researcher needs to ensure that there is a clear link between the data, the propositions (if any), and the conclusions drawn. This argumentation is what enables the researcher to make valid and credible inferences about the phenomenon under study.

Understanding and carefully considering these elements in the design phase of a case study can significantly enhance the quality of the research. It can help ensure that the study is methodologically sound and its findings contribute meaningful insights about the case.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Conducting a case study involves several steps, from defining the research question and selecting the case to collecting and analyzing data . This section outlines these key stages, providing a practical guide on how to conduct case study research.

Defining the research question

The first step in case study research is defining a clear, focused research question. This question should guide the entire research process, from case selection to analysis. It's crucial to ensure that the research question is suitable for a case study approach. Typically, such questions are exploratory or descriptive in nature and focus on understanding a phenomenon within its real-life context.

Selecting and defining the case

The selection of the case should be based on the research question and the objectives of the study. It involves choosing a unique example or a set of examples that provide rich, in-depth data about the phenomenon under investigation. After selecting the case, it's crucial to define it clearly, setting the boundaries of the case, including the time period and the specific context.

Previous research can help guide the case study design. When considering a case study, an example of a case could be taken from previous case study research and used to define cases in a new research inquiry. Considering recently published examples can help understand how to select and define cases effectively.

Developing a detailed case study protocol

A case study protocol outlines the procedures and general rules to be followed during the case study. This includes the data collection methods to be used, the sources of data, and the procedures for analysis. Having a detailed case study protocol ensures consistency and reliability in the study.

The protocol should also consider how to work with the people involved in the research context to grant the research team access to collecting data. As mentioned in previous sections of this guide, establishing rapport is an essential component of qualitative research as it shapes the overall potential for collecting and analyzing data.

Collecting data

Gathering data in case study research often involves multiple sources of evidence, including documents, archival records, interviews, observations, and physical artifacts. This allows for a comprehensive understanding of the case. The process for gathering data should be systematic and carefully documented to ensure the reliability and validity of the study.

Analyzing and interpreting data

The next step is analyzing the data. This involves organizing the data , categorizing it into themes or patterns , and interpreting these patterns to answer the research question. The analysis might also involve comparing the findings with prior research or theoretical propositions.

Writing the case study report

The final step is writing the case study report . This should provide a detailed description of the case, the data, the analysis process, and the findings. The report should be clear, organized, and carefully written to ensure that the reader can understand the case and the conclusions drawn from it.

Each of these steps is crucial in ensuring that the case study research is rigorous, reliable, and provides valuable insights about the case.

The type, depth, and quality of data in your study can significantly influence the validity and utility of the study. In case study research, data is usually collected from multiple sources to provide a comprehensive and nuanced understanding of the case. This section will outline the various methods of collecting data used in case study research and discuss considerations for ensuring the quality of the data.

Interviews are a common method of gathering data in case study research. They can provide rich, in-depth data about the perspectives, experiences, and interpretations of the individuals involved in the case. Interviews can be structured , semi-structured , or unstructured , depending on the research question and the degree of flexibility needed.

Observations

Observations involve the researcher observing the case in its natural setting, providing first-hand information about the case and its context. Observations can provide data that might not be revealed in interviews or documents, such as non-verbal cues or contextual information.

Documents and artifacts

Documents and archival records provide a valuable source of data in case study research. They can include reports, letters, memos, meeting minutes, email correspondence, and various public and private documents related to the case.

These records can provide historical context, corroborate evidence from other sources, and offer insights into the case that might not be apparent from interviews or observations.

Physical artifacts refer to any physical evidence related to the case, such as tools, products, or physical environments. These artifacts can provide tangible insights into the case, complementing the data gathered from other sources.

Ensuring the quality of data collection

Determining the quality of data in case study research requires careful planning and execution. It's crucial to ensure that the data is reliable, accurate, and relevant to the research question. This involves selecting appropriate methods of collecting data, properly training interviewers or observers, and systematically recording and storing the data. It also includes considering ethical issues related to collecting and handling data, such as obtaining informed consent and ensuring the privacy and confidentiality of the participants.

Data analysis

Analyzing case study research involves making sense of the rich, detailed data to answer the research question. This process can be challenging due to the volume and complexity of case study data. However, a systematic and rigorous approach to analysis can ensure that the findings are credible and meaningful. This section outlines the main steps and considerations in analyzing data in case study research.

Organizing the data

The first step in the analysis is organizing the data. This involves sorting the data into manageable sections, often according to the data source or the theme. This step can also involve transcribing interviews, digitizing physical artifacts, or organizing observational data.

Categorizing and coding the data

Once the data is organized, the next step is to categorize or code the data. This involves identifying common themes, patterns, or concepts in the data and assigning codes to relevant data segments. Coding can be done manually or with the help of software tools, and in either case, qualitative analysis software can greatly facilitate the entire coding process. Coding helps to reduce the data to a set of themes or categories that can be more easily analyzed.

Identifying patterns and themes

After coding the data, the researcher looks for patterns or themes in the coded data. This involves comparing and contrasting the codes and looking for relationships or patterns among them. The identified patterns and themes should help answer the research question.

Interpreting the data

Once patterns and themes have been identified, the next step is to interpret these findings. This involves explaining what the patterns or themes mean in the context of the research question and the case. This interpretation should be grounded in the data, but it can also involve drawing on theoretical concepts or prior research.

Verification of the data

The last step in the analysis is verification. This involves checking the accuracy and consistency of the analysis process and confirming that the findings are supported by the data. This can involve re-checking the original data, checking the consistency of codes, or seeking feedback from research participants or peers.

Like any research method , case study research has its strengths and limitations. Researchers must be aware of these, as they can influence the design, conduct, and interpretation of the study.

Understanding the strengths and limitations of case study research can also guide researchers in deciding whether this approach is suitable for their research question . This section outlines some of the key strengths and limitations of case study research.

Benefits include the following:

• Rich, detailed data: One of the main strengths of case study research is that it can generate rich, detailed data about the case. This can provide a deep understanding of the case and its context, which can be valuable in exploring complex phenomena.
• Flexibility: Case study research is flexible in terms of design , data collection , and analysis . A sufficient degree of flexibility allows the researcher to adapt the study according to the case and the emerging findings.
• Real-world context: Case study research involves studying the case in its real-world context, which can provide valuable insights into the interplay between the case and its context.
• Multiple sources of evidence: Case study research often involves collecting data from multiple sources , which can enhance the robustness and validity of the findings.

On the other hand, researchers should consider the following limitations:

• Generalizability: A common criticism of case study research is that its findings might not be generalizable to other cases due to the specificity and uniqueness of each case.
• Time and resource intensive: Case study research can be time and resource intensive due to the depth of the investigation and the amount of collected data.
• Complexity of analysis: The rich, detailed data generated in case study research can make analyzing the data challenging.
• Subjectivity: Given the nature of case study research, there may be a higher degree of subjectivity in interpreting the data , so researchers need to reflect on this and transparently convey to audiences how the research was conducted.

Being aware of these strengths and limitations can help researchers design and conduct case study research effectively and interpret and report the findings appropriately.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

The Advantages and Limitations of Single Case Study Analysis

As Andrew Bennett and Colin Elman have recently noted, qualitative research methods presently enjoy “an almost unprecedented popularity and vitality… in the international relations sub-field”, such that they are now “indisputably prominent, if not pre-eminent” (2010: 499). This is, they suggest, due in no small part to the considerable advantages that case study methods in particular have to offer in studying the “complex and relatively unstructured and infrequent phenomena that lie at the heart of the subfield” (Bennett and Elman, 2007: 171). Using selected examples from within the International Relations literature[1], this paper aims to provide a brief overview of the main principles and distinctive advantages and limitations of single case study analysis. Divided into three inter-related sections, the paper therefore begins by first identifying the underlying principles that serve to constitute the case study as a particular research strategy, noting the somewhat contested nature of the approach in ontological, epistemological, and methodological terms. The second part then looks to the principal single case study types and their associated advantages, including those from within the recent ‘third generation’ of qualitative International Relations (IR) research. The final section of the paper then discusses the most commonly articulated limitations of single case studies; while accepting their susceptibility to criticism, it is however suggested that such weaknesses are somewhat exaggerated. The paper concludes that single case study analysis has a great deal to offer as a means of both understanding and explaining contemporary international relations.

The term ‘case study’, John Gerring has suggested, is “a definitional morass… Evidently, researchers have many different things in mind when they talk about case study research” (2006a: 17). It is possible, however, to distil some of the more commonly-agreed principles. One of the most prominent advocates of case study research, Robert Yin (2009: 14) defines it as “an empirical enquiry that investigates a contemporary phenomenon in depth and within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident”. What this definition usefully captures is that case studies are intended – unlike more superficial and generalising methods – to provide a level of detail and understanding, similar to the ethnographer Clifford Geertz’s (1973) notion of ‘thick description’, that allows for the thorough analysis of the complex and particularistic nature of distinct phenomena. Another frequently cited proponent of the approach, Robert Stake, notes that as a form of research the case study “is defined by interest in an individual case, not by the methods of inquiry used”, and that “the object of study is a specific, unique, bounded system” (2008: 443, 445). As such, three key points can be derived from this – respectively concerning issues of ontology, epistemology, and methodology – that are central to the principles of single case study research.

First, the vital notion of ‘boundedness’ when it comes to the particular unit of analysis means that defining principles should incorporate both the synchronic (spatial) and diachronic (temporal) elements of any so-called ‘case’. As Gerring puts it, a case study should be “an intensive study of a single unit… a spatially bounded phenomenon – e.g. a nation-state, revolution, political party, election, or person – observed at a single point in time or over some delimited period of time” (2004: 342). It is important to note, however, that – whereas Gerring refers to a single unit of analysis – it may be that attention also necessarily be given to particular sub-units. This points to the important difference between what Yin refers to as an ‘holistic’ case design, with a single unit of analysis, and an ’embedded’ case design with multiple units of analysis (Yin, 2009: 50-52). The former, for example, would examine only the overall nature of an international organization, whereas the latter would also look to specific departments, programmes, or policies etc.

Secondly, as Tim May notes of the case study approach, “even the most fervent advocates acknowledge that the term has entered into understandings with little specification or discussion of purpose and process” (2011: 220). One of the principal reasons for this, he argues, is the relationship between the use of case studies in social research and the differing epistemological traditions – positivist, interpretivist, and others – within which it has been utilised. Philosophy of science concerns are obviously a complex issue, and beyond the scope of much of this paper. That said, the issue of how it is that we know what we know – of whether or not a single independent reality exists of which we as researchers can seek to provide explanation – does lead us to an important distinction to be made between so-called idiographic and nomothetic case studies (Gerring, 2006b). The former refers to those which purport to explain only a single case, are concerned with particularisation, and hence are typically (although not exclusively) associated with more interpretivist approaches. The latter are those focused studies that reflect upon a larger population and are more concerned with generalisation, as is often so with more positivist approaches[2]. The importance of this distinction, and its relation to the advantages and limitations of single case study analysis, is returned to below.

Thirdly, in methodological terms, given that the case study has often been seen as more of an interpretivist and idiographic tool, it has also been associated with a distinctly qualitative approach (Bryman, 2009: 67-68). However, as Yin notes, case studies can – like all forms of social science research – be exploratory, descriptive, and/or explanatory in nature. It is “a common misconception”, he notes, “that the various research methods should be arrayed hierarchically… many social scientists still deeply believe that case studies are only appropriate for the exploratory phase of an investigation” (Yin, 2009: 6). If case studies can reliably perform any or all three of these roles – and given that their in-depth approach may also require multiple sources of data and the within-case triangulation of methods – then it becomes readily apparent that they should not be limited to only one research paradigm. Exploratory and descriptive studies usually tend toward the qualitative and inductive, whereas explanatory studies are more often quantitative and deductive (David and Sutton, 2011: 165-166). As such, the association of case study analysis with a qualitative approach is a “methodological affinity, not a definitional requirement” (Gerring, 2006a: 36). It is perhaps better to think of case studies as transparadigmatic; it is mistaken to assume single case study analysis to adhere exclusively to a qualitative methodology (or an interpretivist epistemology) even if it – or rather, practitioners of it – may be so inclined. By extension, this also implies that single case study analysis therefore remains an option for a multitude of IR theories and issue areas; it is how this can be put to researchers’ advantage that is the subject of the next section.

Having elucidated the defining principles of the single case study approach, the paper now turns to an overview of its main benefits. As noted above, a lack of consensus still exists within the wider social science literature on the principles and purposes – and by extension the advantages and limitations – of case study research. Given that this paper is directed towards the particular sub-field of International Relations, it suggests Bennett and Elman’s (2010) more discipline-specific understanding of contemporary case study methods as an analytical framework. It begins however, by discussing Harry Eckstein’s seminal (1975) contribution to the potential advantages of the case study approach within the wider social sciences.

Eckstein proposed a taxonomy which usefully identified what he considered to be the five most relevant types of case study. Firstly were so-called configurative-idiographic studies, distinctly interpretivist in orientation and predicated on the assumption that “one cannot attain prediction and control in the natural science sense, but only understanding ( verstehen )… subjective values and modes of cognition are crucial” (1975: 132). Eckstein’s own sceptical view was that any interpreter ‘simply’ considers a body of observations that are not self-explanatory and “without hard rules of interpretation, may discern in them any number of patterns that are more or less equally plausible” (1975: 134). Those of a more post-modernist bent, of course – sharing an “incredulity towards meta-narratives”, in Lyotard’s (1994: xxiv) evocative phrase – would instead suggest that this more free-form approach actually be advantageous in delving into the subtleties and particularities of individual cases.

Eckstein’s four other types of case study, meanwhile, promote a more nomothetic (and positivist) usage. As described, disciplined-configurative studies were essentially about the use of pre-existing general theories, with a case acting “passively, in the main, as a receptacle for putting theories to work” (Eckstein, 1975: 136). As opposed to the opportunity this presented primarily for theory application, Eckstein identified heuristic case studies as explicit theoretical stimulants – thus having instead the intended advantage of theory-building. So-called p lausibility probes entailed preliminary attempts to determine whether initial hypotheses should be considered sound enough to warrant more rigorous and extensive testing. Finally, and perhaps most notably, Eckstein then outlined the idea of crucial case studies , within which he also included the idea of ‘most-likely’ and ‘least-likely’ cases; the essential characteristic of crucial cases being their specific theory-testing function.

Whilst Eckstein’s was an early contribution to refining the case study approach, Yin’s (2009: 47-52) more recent delineation of possible single case designs similarly assigns them roles in the applying, testing, or building of theory, as well as in the study of unique cases[3]. As a subset of the latter, however, Jack Levy (2008) notes that the advantages of idiographic cases are actually twofold. Firstly, as inductive/descriptive cases – akin to Eckstein’s configurative-idiographic cases – whereby they are highly descriptive, lacking in an explicit theoretical framework and therefore taking the form of “total history”. Secondly, they can operate as theory-guided case studies, but ones that seek only to explain or interpret a single historical episode rather than generalise beyond the case. Not only does this therefore incorporate ‘single-outcome’ studies concerned with establishing causal inference (Gerring, 2006b), it also provides room for the more postmodern approaches within IR theory, such as discourse analysis, that may have developed a distinct methodology but do not seek traditional social scientific forms of explanation.

Applying specifically to the state of the field in contemporary IR, Bennett and Elman identify a ‘third generation’ of mainstream qualitative scholars – rooted in a pragmatic scientific realist epistemology and advocating a pluralistic approach to methodology – that have, over the last fifteen years, “revised or added to essentially every aspect of traditional case study research methods” (2010: 502). They identify ‘process tracing’ as having emerged from this as a central method of within-case analysis. As Bennett and Checkel observe, this carries the advantage of offering a methodologically rigorous “analysis of evidence on processes, sequences, and conjunctures of events within a case, for the purposes of either developing or testing hypotheses about causal mechanisms that might causally explain the case” (2012: 10).

Harnessing various methods, process tracing may entail the inductive use of evidence from within a case to develop explanatory hypotheses, and deductive examination of the observable implications of hypothesised causal mechanisms to test their explanatory capability[4]. It involves providing not only a coherent explanation of the key sequential steps in a hypothesised process, but also sensitivity to alternative explanations as well as potential biases in the available evidence (Bennett and Elman 2010: 503-504). John Owen (1994), for example, demonstrates the advantages of process tracing in analysing whether the causal factors underpinning democratic peace theory are – as liberalism suggests – not epiphenomenal, but variously normative, institutional, or some given combination of the two or other unexplained mechanism inherent to liberal states. Within-case process tracing has also been identified as advantageous in addressing the complexity of path-dependent explanations and critical junctures – as for example with the development of political regime types – and their constituent elements of causal possibility, contingency, closure, and constraint (Bennett and Elman, 2006b).

Bennett and Elman (2010: 505-506) also identify the advantages of single case studies that are implicitly comparative: deviant, most-likely, least-likely, and crucial cases. Of these, so-called deviant cases are those whose outcome does not fit with prior theoretical expectations or wider empirical patterns – again, the use of inductive process tracing has the advantage of potentially generating new hypotheses from these, either particular to that individual case or potentially generalisable to a broader population. A classic example here is that of post-independence India as an outlier to the standard modernisation theory of democratisation, which holds that higher levels of socio-economic development are typically required for the transition to, and consolidation of, democratic rule (Lipset, 1959; Diamond, 1992). Absent these factors, MacMillan’s single case study analysis (2008) suggests the particularistic importance of the British colonial heritage, the ideology and leadership of the Indian National Congress, and the size and heterogeneity of the federal state.

Most-likely cases, as per Eckstein above, are those in which a theory is to be considered likely to provide a good explanation if it is to have any application at all, whereas least-likely cases are ‘tough test’ ones in which the posited theory is unlikely to provide good explanation (Bennett and Elman, 2010: 505). Levy (2008) neatly refers to the inferential logic of the least-likely case as the ‘Sinatra inference’ – if a theory can make it here, it can make it anywhere. Conversely, if a theory cannot pass a most-likely case, it is seriously impugned. Single case analysis can therefore be valuable for the testing of theoretical propositions, provided that predictions are relatively precise and measurement error is low (Levy, 2008: 12-13). As Gerring rightly observes of this potential for falsification:

“a positivist orientation toward the work of social science militates toward a greater appreciation of the case study format, not a denigration of that format, as is usually supposed” (Gerring, 2007: 247, emphasis added).

In summary, the various forms of single case study analysis can – through the application of multiple qualitative and/or quantitative research methods – provide a nuanced, empirically-rich, holistic account of specific phenomena. This may be particularly appropriate for those phenomena that are simply less amenable to more superficial measures and tests (or indeed any substantive form of quantification) as well as those for which our reasons for understanding and/or explaining them are irreducibly subjective – as, for example, with many of the normative and ethical issues associated with the practice of international relations. From various epistemological and analytical standpoints, single case study analysis can incorporate both idiographic sui generis cases and, where the potential for generalisation may exist, nomothetic case studies suitable for the testing and building of causal hypotheses. Finally, it should not be ignored that a signal advantage of the case study – with particular relevance to international relations – also exists at a more practical rather than theoretical level. This is, as Eckstein noted, “that it is economical for all resources: money, manpower, time, effort… especially important, of course, if studies are inherently costly, as they are if units are complex collective individuals ” (1975: 149-150, emphasis added).

Limitations

Single case study analysis has, however, been subject to a number of criticisms, the most common of which concern the inter-related issues of methodological rigour, researcher subjectivity, and external validity. With regard to the first point, the prototypical view here is that of Zeev Maoz (2002: 164-165), who suggests that “the use of the case study absolves the author from any kind of methodological considerations. Case studies have become in many cases a synonym for freeform research where anything goes”. The absence of systematic procedures for case study research is something that Yin (2009: 14-15) sees as traditionally the greatest concern due to a relative absence of methodological guidelines. As the previous section suggests, this critique seems somewhat unfair; many contemporary case study practitioners – and representing various strands of IR theory – have increasingly sought to clarify and develop their methodological techniques and epistemological grounding (Bennett and Elman, 2010: 499-500).

A second issue, again also incorporating issues of construct validity, concerns that of the reliability and replicability of various forms of single case study analysis. This is usually tied to a broader critique of qualitative research methods as a whole. However, whereas the latter obviously tend toward an explicitly-acknowledged interpretive basis for meanings, reasons, and understandings:

“quantitative measures appear objective, but only so long as we don’t ask questions about where and how the data were produced… pure objectivity is not a meaningful concept if the goal is to measure intangibles [as] these concepts only exist because we can interpret them” (Berg and Lune, 2010: 340).

The question of researcher subjectivity is a valid one, and it may be intended only as a methodological critique of what are obviously less formalised and researcher-independent methods (Verschuren, 2003). Owen (1994) and Layne’s (1994) contradictory process tracing results of interdemocratic war-avoidance during the Anglo-American crisis of 1861 to 1863 – from liberal and realist standpoints respectively – are a useful example. However, it does also rest on certain assumptions that can raise deeper and potentially irreconcilable ontological and epistemological issues. There are, regardless, plenty such as Bent Flyvbjerg (2006: 237) who suggest that the case study contains no greater bias toward verification than other methods of inquiry, and that “on the contrary, experience indicates that the case study contains a greater bias toward falsification of preconceived notions than toward verification”.

The third and arguably most prominent critique of single case study analysis is the issue of external validity or generalisability. How is it that one case can reliably offer anything beyond the particular? “We always do better (or, in the extreme, no worse) with more observation as the basis of our generalization”, as King et al write; “in all social science research and all prediction, it is important that we be as explicit as possible about the degree of uncertainty that accompanies out prediction” (1994: 212). This is an unavoidably valid criticism. It may be that theories which pass a single crucial case study test, for example, require rare antecedent conditions and therefore actually have little explanatory range. These conditions may emerge more clearly, as Van Evera (1997: 51-54) notes, from large-N studies in which cases that lack them present themselves as outliers exhibiting a theory’s cause but without its predicted outcome. As with the case of Indian democratisation above, it would logically be preferable to conduct large-N analysis beforehand to identify that state’s non-representative nature in relation to the broader population.

There are, however, three important qualifiers to the argument about generalisation that deserve particular mention here. The first is that with regard to an idiographic single-outcome case study, as Eckstein notes, the criticism is “mitigated by the fact that its capability to do so [is] never claimed by its exponents; in fact it is often explicitly repudiated” (1975: 134). Criticism of generalisability is of little relevance when the intention is one of particularisation. A second qualifier relates to the difference between statistical and analytical generalisation; single case studies are clearly less appropriate for the former but arguably retain significant utility for the latter – the difference also between explanatory and exploratory, or theory-testing and theory-building, as discussed above. As Gerring puts it, “theory confirmation/disconfirmation is not the case study’s strong suit” (2004: 350). A third qualification relates to the issue of case selection. As Seawright and Gerring (2008) note, the generalisability of case studies can be increased by the strategic selection of cases. Representative or random samples may not be the most appropriate, given that they may not provide the richest insight (or indeed, that a random and unknown deviant case may appear). Instead, and properly used , atypical or extreme cases “often reveal more information because they activate more actors… and more basic mechanisms in the situation studied” (Flyvbjerg, 2006). Of course, this also points to the very serious limitation, as hinted at with the case of India above, that poor case selection may alternatively lead to overgeneralisation and/or grievous misunderstandings of the relationship between variables or processes (Bennett and Elman, 2006a: 460-463).

As Tim May (2011: 226) notes, “the goal for many proponents of case studies […] is to overcome dichotomies between generalizing and particularizing, quantitative and qualitative, deductive and inductive techniques”. Research aims should drive methodological choices, rather than narrow and dogmatic preconceived approaches. As demonstrated above, there are various advantages to both idiographic and nomothetic single case study analyses – notably the empirically-rich, context-specific, holistic accounts that they have to offer, and their contribution to theory-building and, to a lesser extent, that of theory-testing. Furthermore, while they do possess clear limitations, any research method involves necessary trade-offs; the inherent weaknesses of any one method, however, can potentially be offset by situating them within a broader, pluralistic mixed-method research strategy. Whether or not single case studies are used in this fashion, they clearly have a great deal to offer.

References 

Bennett, A. and Checkel, J. T. (2012) ‘Process Tracing: From Philosophical Roots to Best Practice’, Simons Papers in Security and Development, No. 21/2012, School for International Studies, Simon Fraser University: Vancouver.

Bennett, A. and Elman, C. (2006a) ‘Qualitative Research: Recent Developments in Case Study Methods’, Annual Review of Political Science , 9, 455-476.

Bennett, A. and Elman, C. (2006b) ‘Complex Causal Relations and Case Study Methods: The Example of Path Dependence’, Political Analysis , 14, 3, 250-267.

Bennett, A. and Elman, C. (2007) ‘Case Study Methods in the International Relations Subfield’, Comparative Political Studies , 40, 2, 170-195.

Bennett, A. and Elman, C. (2010) Case Study Methods. In C. Reus-Smit and D. Snidal (eds) The Oxford Handbook of International Relations . Oxford University Press: Oxford. Ch. 29.

Berg, B. and Lune, H. (2012) Qualitative Research Methods for the Social Sciences . Pearson: London.

Bryman, A. (2012) Social Research Methods . Oxford University Press: Oxford.

David, M. and Sutton, C. D. (2011) Social Research: An Introduction . SAGE Publications Ltd: London.

Diamond, J. (1992) ‘Economic development and democracy reconsidered’, American Behavioral Scientist , 35, 4/5, 450-499.

Eckstein, H. (1975) Case Study and Theory in Political Science. In R. Gomm, M. Hammersley, and P. Foster (eds) Case Study Method . SAGE Publications Ltd: London.

Flyvbjerg, B. (2006) ‘Five Misunderstandings About Case-Study Research’, Qualitative Inquiry , 12, 2, 219-245.

Geertz, C. (1973) The Interpretation of Cultures: Selected Essays by Clifford Geertz . Basic Books Inc: New York.

Gerring, J. (2004) ‘What is a Case Study and What Is It Good for?’, American Political Science Review , 98, 2, 341-354.

Gerring, J. (2006a) Case Study Research: Principles and Practices . Cambridge University Press: Cambridge.

Gerring, J. (2006b) ‘Single-Outcome Studies: A Methodological Primer’, International Sociology , 21, 5, 707-734.

Gerring, J. (2007) ‘Is There a (Viable) Crucial-Case Method?’, Comparative Political Studies , 40, 3, 231-253.

King, G., Keohane, R. O. and Verba, S. (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research . Princeton University Press: Chichester.

Layne, C. (1994) ‘Kant or Cant: The Myth of the Democratic Peace’, International Security , 19, 2, 5-49.

Levy, J. S. (2008) ‘Case Studies: Types, Designs, and Logics of Inference’, Conflict Management and Peace Science , 25, 1-18.

Lipset, S. M. (1959) ‘Some Social Requisites of Democracy: Economic Development and Political Legitimacy’, The American Political Science Review , 53, 1, 69-105.

Lyotard, J-F. (1984) The Postmodern Condition: A Report on Knowledge . University of Minnesota Press: Minneapolis.

MacMillan, A. (2008) ‘Deviant Democratization in India’, Democratization , 15, 4, 733-749.

Maoz, Z. (2002) Case study methodology in international studies: from storytelling to hypothesis testing. In F. P. Harvey and M. Brecher (eds) Evaluating Methodology in International Studies . University of Michigan Press: Ann Arbor.

May, T. (2011) Social Research: Issues, Methods and Process . Open University Press: Maidenhead.

Owen, J. M. (1994) ‘How Liberalism Produces Democratic Peace’, International Security , 19, 2, 87-125.

Seawright, J. and Gerring, J. (2008) ‘Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options’, Political Research Quarterly , 61, 2, 294-308.

Stake, R. E. (2008) Qualitative Case Studies. In N. K. Denzin and Y. S. Lincoln (eds) Strategies of Qualitative Inquiry . Sage Publications: Los Angeles. Ch. 17.

Van Evera, S. (1997) Guide to Methods for Students of Political Science . Cornell University Press: Ithaca.

Verschuren, P. J. M. (2003) ‘Case study as a research strategy: some ambiguities and opportunities’, International Journal of Social Research Methodology , 6, 2, 121-139.

Yin, R. K. (2009) Case Study Research: Design and Methods . SAGE Publications Ltd: London.

[1] The paper follows convention by differentiating between ‘International Relations’ as the academic discipline and ‘international relations’ as the subject of study.

[2] There is some similarity here with Stake’s (2008: 445-447) notion of intrinsic cases, those undertaken for a better understanding of the particular case, and instrumental ones that provide insight for the purposes of a wider external interest.

[3] These may be unique in the idiographic sense, or in nomothetic terms as an exception to the generalising suppositions of either probabilistic or deterministic theories (as per deviant cases, below).

[4] Although there are “philosophical hurdles to mount”, according to Bennett and Checkel, there exists no a priori reason as to why process tracing (as typically grounded in scientific realism) is fundamentally incompatible with various strands of positivism or interpretivism (2012: 18-19). By extension, it can therefore be incorporated by a range of contemporary mainstream IR theories.

— Written by: Ben Willis Written at: University of Plymouth Written for: David Brockington Date written: January 2013

Further Reading on E-International Relations

• Identity in International Conflicts: A Case Study of the Cuban Missile Crisis
• Imperialism’s Legacy in the Study of Contemporary Politics: The Case of Hegemonic Stability Theory
• Recreating a Nation’s Identity Through Symbolism: A Chinese Case Study
• Ontological Insecurity: A Case Study on Israeli-Palestinian Conflict in Jerusalem
• Terrorists or Freedom Fighters: A Case Study of ETA
• A Critical Assessment of Eco-Marxism: A Ghanaian Case Study

Please Consider Donating

Before you download your free e-book, please consider donating to support open access publishing.

E-IR is an independent non-profit publisher run by an all volunteer team. Your donations allow us to invest in new open access titles and pay our bandwidth bills to ensure we keep our existing titles free to view. Any amount, in any currency, is appreciated. Many thanks!

Donations are voluntary and not required to download the e-book - your link to download is below.

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

What Is a Case Study?

Weighing the pros and cons of this method of research

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Cara Lustik is a fact-checker and copywriter.

Verywell / Colleen Tighe

• Pros and Cons

What Types of Case Studies Are Out There?

Where do you find data for a case study, how do i write a psychology case study.

A case study is an in-depth study of one person, group, or event. In a case study, nearly every aspect of the subject's life and history is analyzed to seek patterns and causes of behavior. Case studies can be used in many different fields, including psychology, medicine, education, anthropology, political science, and social work.

The point of a case study is to learn as much as possible about an individual or group so that the information can be generalized to many others. Unfortunately, case studies tend to be highly subjective, and it is sometimes difficult to generalize results to a larger population.

While case studies focus on a single individual or group, they follow a format similar to other types of psychology writing. If you are writing a case study, we got you—here are some rules of APA format to reference.  

At a Glance

A case study, or an in-depth study of a person, group, or event, can be a useful research tool when used wisely. In many cases, case studies are best used in situations where it would be difficult or impossible for you to conduct an experiment. They are helpful for looking at unique situations and allow researchers to gather a lot of˜ information about a specific individual or group of people. However, it's important to be cautious of any bias we draw from them as they are highly subjective.

What Are the Benefits and Limitations of Case Studies?

A case study can have its strengths and weaknesses. Researchers must consider these pros and cons before deciding if this type of study is appropriate for their needs.

One of the greatest advantages of a case study is that it allows researchers to investigate things that are often difficult or impossible to replicate in a lab. Some other benefits of a case study:

• Allows researchers to capture information on the 'how,' 'what,' and 'why,' of something that's implemented
• Gives researchers the chance to collect information on why one strategy might be chosen over another
• Permits researchers to develop hypotheses that can be explored in experimental research

On the other hand, a case study can have some drawbacks:

• It cannot necessarily be generalized to the larger population
• Cannot demonstrate cause and effect
• It may not be scientifically rigorous
• It can lead to bias

Researchers may choose to perform a case study if they want to explore a unique or recently discovered phenomenon. Through their insights, researchers develop additional ideas and study questions that might be explored in future studies.

It's important to remember that the insights from case studies cannot be used to determine cause-and-effect relationships between variables. However, case studies may be used to develop hypotheses that can then be addressed in experimental research.

Case Study Examples

There have been a number of notable case studies in the history of psychology. Much of  Freud's work and theories were developed through individual case studies. Some great examples of case studies in psychology include:

• Anna O : Anna O. was a pseudonym of a woman named Bertha Pappenheim, a patient of a physician named Josef Breuer. While she was never a patient of Freud's, Freud and Breuer discussed her case extensively. The woman was experiencing symptoms of a condition that was then known as hysteria and found that talking about her problems helped relieve her symptoms. Her case played an important part in the development of talk therapy as an approach to mental health treatment.
• Phineas Gage : Phineas Gage was a railroad employee who experienced a terrible accident in which an explosion sent a metal rod through his skull, damaging important portions of his brain. Gage recovered from his accident but was left with serious changes in both personality and behavior.
• Genie : Genie was a young girl subjected to horrific abuse and isolation. The case study of Genie allowed researchers to study whether language learning was possible, even after missing critical periods for language development. Her case also served as an example of how scientific research may interfere with treatment and lead to further abuse of vulnerable individuals.

Such cases demonstrate how case research can be used to study things that researchers could not replicate in experimental settings. In Genie's case, her horrific abuse denied her the opportunity to learn a language at critical points in her development.

This is clearly not something researchers could ethically replicate, but conducting a case study on Genie allowed researchers to study phenomena that are otherwise impossible to reproduce.

There are a few different types of case studies that psychologists and other researchers might use:

• Collective case studies : These involve studying a group of individuals. Researchers might study a group of people in a certain setting or look at an entire community. For example, psychologists might explore how access to resources in a community has affected the collective mental well-being of those who live there.
• Descriptive case studies : These involve starting with a descriptive theory. The subjects are then observed, and the information gathered is compared to the pre-existing theory.
• Explanatory case studies : These   are often used to do causal investigations. In other words, researchers are interested in looking at factors that may have caused certain things to occur.
• Exploratory case studies : These are sometimes used as a prelude to further, more in-depth research. This allows researchers to gather more information before developing their research questions and hypotheses .
• Instrumental case studies : These occur when the individual or group allows researchers to understand more than what is initially obvious to observers.
• Intrinsic case studies : This type of case study is when the researcher has a personal interest in the case. Jean Piaget's observations of his own children are good examples of how an intrinsic case study can contribute to the development of a psychological theory.

The three main case study types often used are intrinsic, instrumental, and collective. Intrinsic case studies are useful for learning about unique cases. Instrumental case studies help look at an individual to learn more about a broader issue. A collective case study can be useful for looking at several cases simultaneously.

The type of case study that psychology researchers use depends on the unique characteristics of the situation and the case itself.

There are a number of different sources and methods that researchers can use to gather information about an individual or group. Six major sources that have been identified by researchers are:

• Archival records : Census records, survey records, and name lists are examples of archival records.
• Direct observation : This strategy involves observing the subject, often in a natural setting . While an individual observer is sometimes used, it is more common to utilize a group of observers.
• Documents : Letters, newspaper articles, administrative records, etc., are the types of documents often used as sources.
• Interviews : Interviews are one of the most important methods for gathering information in case studies. An interview can involve structured survey questions or more open-ended questions.
• Participant observation : When the researcher serves as a participant in events and observes the actions and outcomes, it is called participant observation.
• Physical artifacts : Tools, objects, instruments, and other artifacts are often observed during a direct observation of the subject.

If you have been directed to write a case study for a psychology course, be sure to check with your instructor for any specific guidelines you need to follow. If you are writing your case study for a professional publication, check with the publisher for their specific guidelines for submitting a case study.

Here is a general outline of what should be included in a case study.

Section 1: A Case History

This section will have the following structure and content:

Background information : The first section of your paper will present your client's background. Include factors such as age, gender, work, health status, family mental health history, family and social relationships, drug and alcohol history, life difficulties, goals, and coping skills and weaknesses.

Description of the presenting problem : In the next section of your case study, you will describe the problem or symptoms that the client presented with.

Describe any physical, emotional, or sensory symptoms reported by the client. Thoughts, feelings, and perceptions related to the symptoms should also be noted. Any screening or diagnostic assessments that are used should also be described in detail and all scores reported.

Your diagnosis : Provide your diagnosis and give the appropriate Diagnostic and Statistical Manual code. Explain how you reached your diagnosis, how the client's symptoms fit the diagnostic criteria for the disorder(s), or any possible difficulties in reaching a diagnosis.

Section 2: Treatment Plan

This portion of the paper will address the chosen treatment for the condition. This might also include the theoretical basis for the chosen treatment or any other evidence that might exist to support why this approach was chosen.

• Cognitive behavioral approach : Explain how a cognitive behavioral therapist would approach treatment. Offer background information on cognitive behavioral therapy and describe the treatment sessions, client response, and outcome of this type of treatment. Make note of any difficulties or successes encountered by your client during treatment.
• Humanistic approach : Describe a humanistic approach that could be used to treat your client, such as client-centered therapy . Provide information on the type of treatment you chose, the client's reaction to the treatment, and the end result of this approach. Explain why the treatment was successful or unsuccessful.
• Psychoanalytic approach : Describe how a psychoanalytic therapist would view the client's problem. Provide some background on the psychoanalytic approach and cite relevant references. Explain how psychoanalytic therapy would be used to treat the client, how the client would respond to therapy, and the effectiveness of this treatment approach.
• Pharmacological approach : If treatment primarily involves the use of medications, explain which medications were used and why. Provide background on the effectiveness of these medications and how monotherapy may compare with an approach that combines medications with therapy or other treatments.

This section of a case study should also include information about the treatment goals, process, and outcomes.

When you are writing a case study, you should also include a section where you discuss the case study itself, including the strengths and limitiations of the study. You should note how the findings of your case study might support previous research. 

In your discussion section, you should also describe some of the implications of your case study. What ideas or findings might require further exploration? How might researchers go about exploring some of these questions in additional studies?

Need More Tips?

Here are a few additional pointers to keep in mind when formatting your case study:

• Never refer to the subject of your case study as "the client." Instead, use their name or a pseudonym.
• Read examples of case studies to gain an idea about the style and format.
• Remember to use APA format when citing references .

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach .  BMC Med Res Methodol . 2011;11:100.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach . BMC Med Res Methodol . 2011 Jun 27;11:100. doi:10.1186/1471-2288-11-100

Gagnon, Yves-Chantal.  The Case Study as Research Method: A Practical Handbook . Canada, Chicago Review Press Incorporated DBA Independent Pub Group, 2010.

Yin, Robert K. Case Study Research and Applications: Design and Methods . United States, SAGE Publications, 2017.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Case Study | Definition, Examples & Methods

Case Study | Definition, Examples & Methods

Published on 5 May 2022 by Shona McCombes . Revised on 30 January 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organisation, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating, and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyse the case.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Prevent plagiarism, run a free check.

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

Unlike quantitative or experimental research, a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

If you find yourself aiming to simultaneously investigate and solve an issue, consider conducting action research . As its name suggests, action research conducts research and takes action at the same time, and is highly iterative and flexible. 

However, you can also choose a more common or representative case to exemplify a particular category, experience, or phenomenon.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews, observations, and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data .

The aim is to gain as thorough an understanding as possible of the case and its context.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis, with separate sections or chapters for the methods , results , and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyse its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, January 30). Case Study | Definition, Examples & Methods. Scribbr. Retrieved 30 May 2024, from https://www.scribbr.co.uk/research-methods/case-studies/

Is this article helpful?

Shona McCombes

Other students also liked, correlational research | guide, design & examples, a quick guide to experimental design | 5 steps & examples, descriptive research design | definition, methods & examples.

Case Studies

This guide examines case studies, a form of qualitative descriptive research that is used to look at individuals, a small group of participants, or a group as a whole. Researchers collect data about participants using participant and direct observations, interviews, protocols, tests, examinations of records, and collections of writing samples. Starting with a definition of the case study, the guide moves to a brief history of this research method. Using several well documented case studies, the guide then looks at applications and methods including data collection and analysis. A discussion of ways to handle validity, reliability, and generalizability follows, with special attention to case studies as they are applied to composition studies. Finally, this guide examines the strengths and weaknesses of case studies.

Definition and Overview

Case study refers to the collection and presentation of detailed information about a particular participant or small group, frequently including the accounts of subjects themselves. A form of qualitative descriptive research, the case study looks intensely at an individual or small participant pool, drawing conclusions only about that participant or group and only in that specific context. Researchers do not focus on the discovery of a universal, generalizable truth, nor do they typically look for cause-effect relationships; instead, emphasis is placed on exploration and description.

Case studies typically examine the interplay of all variables in order to provide as complete an understanding of an event or situation as possible. This type of comprehensive understanding is arrived at through a process known as thick description, which involves an in-depth description of the entity being evaluated, the circumstances under which it is used, the characteristics of the people involved in it, and the nature of the community in which it is located. Thick description also involves interpreting the meaning of demographic and descriptive data such as cultural norms and mores, community values, ingrained attitudes, and motives.

Unlike quantitative methods of research, like the survey, which focus on the questions of who, what, where, how much, and how many, and archival analysis, which often situates the participant in some form of historical context, case studies are the preferred strategy when how or why questions are asked. Likewise, they are the preferred method when the researcher has little control over the events, and when there is a contemporary focus within a real life context. In addition, unlike more specifically directed experiments, case studies require a problem that seeks a holistic understanding of the event or situation in question using inductive logic--reasoning from specific to more general terms.

In scholarly circles, case studies are frequently discussed within the context of qualitative research and naturalistic inquiry. Case studies are often referred to interchangeably with ethnography, field study, and participant observation. The underlying philosophical assumptions in the case are similar to these types of qualitative research because each takes place in a natural setting (such as a classroom, neighborhood, or private home), and strives for a more holistic interpretation of the event or situation under study.

Unlike more statistically-based studies which search for quantifiable data, the goal of a case study is to offer new variables and questions for further research. F.H. Giddings, a sociologist in the early part of the century, compares statistical methods to the case study on the basis that the former are concerned with the distribution of a particular trait, or a small number of traits, in a population, whereas the case study is concerned with the whole variety of traits to be found in a particular instance" (Hammersley 95).

Case studies are not a new form of research; naturalistic inquiry was the primary research tool until the development of the scientific method. The fields of sociology and anthropology are credited with the primary shaping of the concept as we know it today. However, case study research has drawn from a number of other areas as well: the clinical methods of doctors; the casework technique being developed by social workers; the methods of historians and anthropologists, plus the qualitative descriptions provided by quantitative researchers like LePlay; and, in the case of Robert Park, the techniques of newspaper reporters and novelists.

Park was an ex-newspaper reporter and editor who became very influential in developing sociological case studies at the University of Chicago in the 1920s. As a newspaper professional he coined the term "scientific" or "depth" reporting: the description of local events in a way that pointed to major social trends. Park viewed the sociologist as "merely a more accurate, responsible, and scientific reporter." Park stressed the variety and value of human experience. He believed that sociology sought to arrive at natural, but fluid, laws and generalizations in regard to human nature and society. These laws weren't static laws of the kind sought by many positivists and natural law theorists, but rather, they were laws of becoming--with a constant possibility of change. Park encouraged students to get out of the library, to quit looking at papers and books, and to view the constant experiment of human experience. He writes, "Go and sit in the lounges of the luxury hotels and on the doorsteps of the flophouses; sit on the Gold Coast settees and on the slum shakedowns; sit in the Orchestra Hall and in the Star and Garter Burlesque. In short, gentlemen [sic], go get the seats of your pants dirty in real research."

But over the years, case studies have drawn their share of criticism. In fact, the method had its detractors from the start. In the 1920s, the debate between pro-qualitative and pro-quantitative became quite heated. Case studies, when compared to statistics, were considered by many to be unscientific. From the 1930's on, the rise of positivism had a growing influence on quantitative methods in sociology. People wanted static, generalizable laws in science. The sociological positivists were looking for stable laws of social phenomena. They criticized case study research because it failed to provide evidence of inter subjective agreement. Also, they condemned it because of the few number of cases studied and that the under-standardized character of their descriptions made generalization impossible. By the 1950s, quantitative methods, in the form of survey research, had become the dominant sociological approach and case study had become a minority practice.

Educational Applications

The 1950's marked the dawning of a new era in case study research, namely that of the utilization of the case study as a teaching method. "Instituted at Harvard Business School in the 1950s as a primary method of teaching, cases have since been used in classrooms and lecture halls alike, either as part of a course of study or as the main focus of the course to which other teaching material is added" (Armisted 1984). The basic purpose of instituting the case method as a teaching strategy was "to transfer much of the responsibility for learning from the teacher on to the student, whose role, as a result, shifts away from passive absorption toward active construction" (Boehrer 1990). Through careful examination and discussion of various cases, "students learn to identify actual problems, to recognize key players and their agendas, and to become aware of those aspects of the situation that contribute to the problem" (Merseth 1991). In addition, students are encouraged to "generate their own analysis of the problems under consideration, to develop their own solutions, and to practically apply their own knowledge of theory to these problems" (Boyce 1993). Along the way, students also develop "the power to analyze and to master a tangled circumstance by identifying and delineating important factors; the ability to utilize ideas, to test them against facts, and to throw them into fresh combinations" (Merseth 1991).

In addition to the practical application and testing of scholarly knowledge, case discussions can also help students prepare for real-world problems, situations and crises by providing an approximation of various professional environments (i.e. classroom, board room, courtroom, or hospital). Thus, through the examination of specific cases, students are given the opportunity to work out their own professional issues through the trials, tribulations, experiences, and research findings of others. An obvious advantage to this mode of instruction is that it allows students the exposure to settings and contexts that they might not otherwise experience. For example, a student interested in studying the effects of poverty on minority secondary student's grade point averages and S.A.T. scores could access and analyze information from schools as geographically diverse as Los Angeles, New York City, Miami, and New Mexico without ever having to leave the classroom.

The case study method also incorporates the idea that students can learn from one another "by engaging with each other and with each other's ideas, by asserting something and then having it questioned, challenged and thrown back at them so that they can reflect on what they hear, and then refine what they say" (Boehrer 1990). In summary, students can direct their own learning by formulating questions and taking responsibility for the study.

Types and Design Concerns

Researchers use multiple methods and approaches to conduct case studies.

Types of Case Studies

Under the more generalized category of case study exist several subdivisions, each of which is custom selected for use depending upon the goals and/or objectives of the investigator. These types of case study include the following:

Illustrative Case Studies These are primarily descriptive studies. They typically utilize one or two instances of an event to show what a situation is like. Illustrative case studies serve primarily to make the unfamiliar familiar and to give readers a common language about the topic in question.

Exploratory (or pilot) Case Studies These are condensed case studies performed before implementing a large scale investigation. Their basic function is to help identify questions and select types of measurement prior to the main investigation. The primary pitfall of this type of study is that initial findings may seem convincing enough to be released prematurely as conclusions.

Cumulative Case Studies These serve to aggregate information from several sites collected at different times. The idea behind these studies is the collection of past studies will allow for greater generalization without additional cost or time being expended on new, possibly repetitive studies.

Critical Instance Case Studies These examine one or more sites for either the purpose of examining a situation of unique interest with little to no interest in generalizability, or to call into question or challenge a highly generalized or universal assertion. This method is useful for answering cause and effect questions.

Identifying a Theoretical Perspective

Much of the case study's design is inherently determined for researchers, depending on the field from which they are working. In composition studies, researchers are typically working from a qualitative, descriptive standpoint. In contrast, physicists will approach their research from a more quantitative perspective. Still, in designing the study, researchers need to make explicit the questions to be explored and the theoretical perspective from which they will approach the case. The three most commonly adopted theories are listed below:

Individual Theories These focus primarily on the individual development, cognitive behavior, personality, learning and disability, and interpersonal interactions of a particular subject.

Organizational Theories These focus on bureaucracies, institutions, organizational structure and functions, or excellence in organizational performance.

Social Theories These focus on urban development, group behavior, cultural institutions, or marketplace functions.

Two examples of case studies are used consistently throughout this chapter. The first, a study produced by Berkenkotter, Huckin, and Ackerman (1988), looks at a first year graduate student's initiation into an academic writing program. The study uses participant-observer and linguistic data collecting techniques to assess the student's knowledge of appropriate discourse conventions. Using the pseudonym Nate to refer to the subject, the study sought to illuminate the particular experience rather than to generalize about the experience of fledgling academic writers collectively.

For example, in Berkenkotter, Huckin, and Ackerman's (1988) study we are told that the researchers are interested in disciplinary communities. In the first paragraph, they ask what constitutes membership in a disciplinary community and how achieving membership might affect a writer's understanding and production of texts. In the third paragraph they state that researchers must negotiate their claims "within the context of his sub specialty's accepted knowledge and methodology." In the next paragraph they ask, "How is literacy acquired? What is the process through which novices gain community membership? And what factors either aid or hinder students learning the requisite linguistic behaviors?" This introductory section ends with a paragraph in which the study's authors claim that during the course of the study, the subject, Nate, successfully makes the transition from "skilled novice" to become an initiated member of the academic discourse community and that his texts exhibit linguistic changes which indicate this transition. In the next section the authors make explicit the sociolinguistic theoretical and methodological assumptions on which the study is based (1988). Thus the reader has a good understanding of the authors' theoretical background and purpose in conducting the study even before it is explicitly stated on the fourth page of the study. "Our purpose was to examine the effects of the educational context on one graduate student's production of texts as he wrote in different courses and for different faculty members over the academic year 1984-85." The goal of the study then, was to explore the idea that writers must be initiated into a writing community, and that this initiation will change the way one writes.

The second example is Janet Emig's (1971) study of the composing process of a group of twelfth graders. In this study, Emig seeks to answer the question of what happens to the self as a result educational stimuli in terms of academic writing. The case study used methods such as protocol analysis, tape-recorded interviews, and discourse analysis.

In the case of Janet Emig's (1971) study of the composing process of eight twelfth graders, four specific hypotheses were made:

• Twelfth grade writers engage in two modes of composing: reflexive and extensive.
• These differences can be ascertained and characterized through having the writers compose aloud their composition process.
• A set of implied stylistic principles governs the writing process.
• For twelfth grade writers, extensive writing occurs chiefly as a school-sponsored activity, or reflexive, as a self-sponsored activity.

In this study, the chief distinction is between the two dominant modes of composing among older, secondary school students. The distinctions are:

• The reflexive mode, which focuses on the writer's thoughts and feelings.
• The extensive mode, which focuses on conveying a message.

Emig also outlines the specific questions which guided the research in the opening pages of her Review of Literature , preceding the report.

Designing a Case Study

After considering the different sub categories of case study and identifying a theoretical perspective, researchers can begin to design their study. Research design is the string of logic that ultimately links the data to be collected and the conclusions to be drawn to the initial questions of the study. Typically, research designs deal with at least four problems:

• What questions to study
• What data are relevant
• What data to collect
• How to analyze that data

In other words, a research design is basically a blueprint for getting from the beginning to the end of a study. The beginning is an initial set of questions to be answered, and the end is some set of conclusions about those questions.

Because case studies are conducted on topics as diverse as Anglo-Saxon Literature (Thrane 1986) and AIDS prevention (Van Vugt 1994), it is virtually impossible to outline any strict or universal method or design for conducting the case study. However, Robert K. Yin (1993) does offer five basic components of a research design:

• A study's questions.
• A study's propositions (if any).
• A study's units of analysis.
• The logic that links the data to the propositions.
• The criteria for interpreting the findings.

In addition to these five basic components, Yin also stresses the importance of clearly articulating one's theoretical perspective, determining the goals of the study, selecting one's subject(s), selecting the appropriate method(s) of collecting data, and providing some considerations to the composition of the final report.

Conducting Case Studies

To obtain as complete a picture of the participant as possible, case study researchers can employ a variety of approaches and methods. These approaches, methods, and related issues are discussed in depth in this section.

Method: Single or Multi-modal?

To obtain as complete a picture of the participant as possible, case study researchers can employ a variety of methods. Some common methods include interviews , protocol analyses, field studies, and participant-observations. Emig (1971) chose to use several methods of data collection. Her sources included conversations with the students, protocol analysis, discrete observations of actual composition, writing samples from each student, and school records (Lauer and Asher 1988).

Berkenkotter, Huckin, and Ackerman (1988) collected data by observing classrooms, conducting faculty and student interviews, collecting self reports from the subject, and by looking at the subject's written work.

A study that was criticized for using a single method model was done by Flower and Hayes (1984). In this study that explores the ways in which writers use different forms of knowing to create space, the authors used only protocol analysis to gather data. The study came under heavy fire because of their decision to use only one method.

Participant Selection

Case studies can use one participant, or a small group of participants. However, it is important that the participant pool remain relatively small. The participants can represent a diverse cross section of society, but this isn't necessary.

For example, the Berkenkotter, Huckin, and Ackerman (1988) study looked at just one participant, Nate. By contrast, in Janet Emig's (1971) study of the composition process of twelfth graders, eight participants were selected representing a diverse cross section of the community, with volunteers from an all-white upper-middle-class suburban school, an all-black inner-city school, a racially mixed lower-middle-class school, an economically and racially mixed school, and a university school.

Often, a brief "case history" is done on the participants of the study in order to provide researchers with a clearer understanding of their participants, as well as some insight as to how their own personal histories might affect the outcome of the study. For instance, in Emig's study, the investigator had access to the school records of five of the participants, and to standardized test scores for the remaining three. Also made available to the researcher was the information that three of the eight students were selected as NCTE Achievement Award winners. These personal histories can be useful in later stages of the study when data are being analyzed and conclusions drawn.

Data Collection

There are six types of data collected in case studies:

• Archival records.
• Interviews.
• Direct observation.
• Participant observation.

In the field of composition research, these six sources might be:

• A writer's drafts.
• School records of student writers.
• Transcripts of interviews with a writer.
• Transcripts of conversations between writers (and protocols).
• Videotapes and notes from direct field observations.
• Hard copies of a writer's work on computer.

Depending on whether researchers have chosen to use a single or multi-modal approach for the case study, they may choose to collect data from one or any combination of these sources.

Protocols, that is, transcriptions of participants talking aloud about what they are doing as they do it, have been particularly common in composition case studies. For example, in Emig's (1971) study, the students were asked, in four different sessions, to give oral autobiographies of their writing experiences and to compose aloud three themes in the presence of a tape recorder and the investigator.

In some studies, only one method of data collection is conducted. For example, the Flower and Hayes (1981) report on the cognitive process theory of writing depends on protocol analysis alone. However, using multiple sources of evidence to increase the reliability and validity of the data can be advantageous.

Case studies are likely to be much more convincing and accurate if they are based on several different sources of information, following a corroborating mode. This conclusion is echoed among many composition researchers. For example, in her study of predrafting processes of high and low-apprehensive writers, Cynthia Selfe (1985) argues that because "methods of indirect observation provide only an incomplete reflection of the complex set of processes involved in composing, a combination of several such methods should be used to gather data in any one study." Thus, in this study, Selfe collected her data from protocols, observations of students role playing their writing processes, audio taped interviews with the students, and videotaped observations of the students in the process of composing.

It can be said then, that cross checking data from multiple sources can help provide a multidimensional profile of composing activities in a particular setting. Sharan Merriam (1985) suggests "checking, verifying, testing, probing, and confirming collected data as you go, arguing that this process will follow in a funnel-like design resulting in less data gathering in later phases of the study along with a congruent increase in analysis checking, verifying, and confirming."

It is important to note that in case studies, as in any qualitative descriptive research, while researchers begin their studies with one or several questions driving the inquiry (which influence the key factors the researcher will be looking for during data collection), a researcher may find new key factors emerging during data collection. These might be unexpected patterns or linguistic features which become evident only during the course of the research. While not bearing directly on the researcher's guiding questions, these variables may become the basis for new questions asked at the end of the report, thus linking to the possibility of further research.

Data Analysis

As the information is collected, researchers strive to make sense of their data. Generally, researchers interpret their data in one of two ways: holistically or through coding. Holistic analysis does not attempt to break the evidence into parts, but rather to draw conclusions based on the text as a whole. Flower and Hayes (1981), for example, make inferences from entire sections of their students' protocols, rather than searching through the transcripts to look for isolatable characteristics.

However, composition researchers commonly interpret their data by coding, that is by systematically searching data to identify and/or categorize specific observable actions or characteristics. These observable actions then become the key variables in the study. Sharan Merriam (1988) suggests seven analytic frameworks for the organization and presentation of data:

• The role of participants.
• The network analysis of formal and informal exchanges among groups.
• Historical.
• Thematical.
• Ritual and symbolism.
• Critical incidents that challenge or reinforce fundamental beliefs, practices, and values.

There are two purposes of these frameworks: to look for patterns among the data and to look for patterns that give meaning to the case study.

As stated above, while most researchers begin their case studies expecting to look for particular observable characteristics, it is not unusual for key variables to emerge during data collection. Typical variables coded in case studies of writers include pauses writers make in the production of a text, the use of specific linguistic units (such as nouns or verbs), and writing processes (planning, drafting, revising, and editing). In the Berkenkotter, Huckin, and Ackerman (1988) study, for example, researchers coded the participant's texts for use of connectives, discourse demonstratives, average sentence length, off-register words, use of the first person pronoun, and the ratio of definite articles to indefinite articles.

Since coding is inherently subjective, more than one coder is usually employed. In the Berkenkotter, Huckin, and Ackerman (1988) study, for example, three rhetoricians were employed to code the participant's texts for off-register phrases. The researchers established the agreement among the coders before concluding that the participant used fewer off-register words as the graduate program progressed.

Composing the Case Study Report

In the many forms it can take, "a case study is generically a story; it presents the concrete narrative detail of actual, or at least realistic events, it has a plot, exposition, characters, and sometimes even dialogue" (Boehrer 1990). Generally, case study reports are extensively descriptive, with "the most problematic issue often referred to as being the determination of the right combination of description and analysis" (1990). Typically, authors address each step of the research process, and attempt to give the reader as much context as possible for the decisions made in the research design and for the conclusions drawn.

This contextualization usually includes a detailed explanation of the researchers' theoretical positions, of how those theories drove the inquiry or led to the guiding research questions, of the participants' backgrounds, of the processes of data collection, of the training and limitations of the coders, along with a strong attempt to make connections between the data and the conclusions evident.

Although the Berkenkotter, Huckin, and Ackerman (1988) study does not, case study reports often include the reactions of the participants to the study or to the researchers' conclusions. Because case studies tend to be exploratory, most end with implications for further study. Here researchers may identify significant variables that emerged during the research and suggest studies related to these, or the authors may suggest further general questions that their case study generated.

For example, Emig's (1971) study concludes with a section dedicated solely to the topic of implications for further research, in which she suggests several means by which this particular study could have been improved, as well as questions and ideas raised by this study which other researchers might like to address, such as: is there a correlation between a certain personality and a certain composing process profile (e.g. is there a positive correlation between ego strength and persistence in revising)?

Also included in Emig's study is a section dedicated to implications for teaching, which outlines the pedagogical ramifications of the study's findings for teachers currently involved in high school writing programs.

Sharan Merriam (1985) also offers several suggestions for alternative presentations of data:

• Prepare specialized condensations for appropriate groups.
• Replace narrative sections with a series of answers to open-ended questions.
• Present "skimmer's" summaries at beginning of each section.
• Incorporate headlines that encapsulate information from text.
• Prepare analytic summaries with supporting data appendixes.
• Present data in colorful and/or unique graphic representations.

Issues of Validity and Reliability

Once key variables have been identified, they can be analyzed. Reliability becomes a key concern at this stage, and many case study researchers go to great lengths to ensure that their interpretations of the data will be both reliable and valid. Because issues of validity and reliability are an important part of any study in the social sciences, it is important to identify some ways of dealing with results.

Multi-modal case study researchers often balance the results of their coding with data from interviews or writer's reflections upon their own work. Consequently, the researchers' conclusions become highly contextualized. For example, in a case study which looked at the time spent in different stages of the writing process, Berkenkotter concluded that her participant, Donald Murray, spent more time planning his essays than in other writing stages. The report of this case study is followed by Murray's reply, wherein he agrees with some of Berkenkotter's conclusions and disagrees with others.

As is the case with other research methodologies, issues of external validity, construct validity, and reliability need to be carefully considered.

Commentary on Case Studies

Researchers often debate the relative merits of particular methods, among them case study. In this section, we comment on two key issues. To read the commentaries, choose any of the items below:

Strengths and Weaknesses of Case Studies

Most case study advocates point out that case studies produce much more detailed information than what is available through a statistical analysis. Advocates will also hold that while statistical methods might be able to deal with situations where behavior is homogeneous and routine, case studies are needed to deal with creativity, innovation, and context. Detractors argue that case studies are difficult to generalize because of inherent subjectivity and because they are based on qualitative subjective data, generalizable only to a particular context.

Flexibility

The case study approach is a comparatively flexible method of scientific research. Because its project designs seem to emphasize exploration rather than prescription or prediction, researchers are comparatively freer to discover and address issues as they arise in their experiments. In addition, the looser format of case studies allows researchers to begin with broad questions and narrow their focus as their experiment progresses rather than attempt to predict every possible outcome before the experiment is conducted.

Emphasis on Context

By seeking to understand as much as possible about a single subject or small group of subjects, case studies specialize in "deep data," or "thick description"--information based on particular contexts that can give research results a more human face. This emphasis can help bridge the gap between abstract research and concrete practice by allowing researchers to compare their firsthand observations with the quantitative results obtained through other methods of research.

Inherent Subjectivity

"The case study has long been stereotyped as the weak sibling among social science methods," and is often criticized as being too subjective and even pseudo-scientific. Likewise, "investigators who do case studies are often regarded as having deviated from their academic disciplines, and their investigations as having insufficient precision (that is, quantification), objectivity and rigor" (Yin 1989). Opponents cite opportunities for subjectivity in the implementation, presentation, and evaluation of case study research. The approach relies on personal interpretation of data and inferences. Results may not be generalizable, are difficult to test for validity, and rarely offer a problem-solving prescription. Simply put, relying on one or a few subjects as a basis for cognitive extrapolations runs the risk of inferring too much from what might be circumstance.

High Investment

Case studies can involve learning more about the subjects being tested than most researchers would care to know--their educational background, emotional background, perceptions of themselves and their surroundings, their likes, dislikes, and so on. Because of its emphasis on "deep data," the case study is out of reach for many large-scale research projects which look at a subject pool in the tens of thousands. A budget request of $10,000 to examine 200 subjects sounds more efficient than a similar request to examine four subjects.

Ethical Considerations

Researchers conducting case studies should consider certain ethical issues. For example, many educational case studies are often financed by people who have, either directly or indirectly, power over both those being studied and those conducting the investigation (1985). This conflict of interests can hinder the credibility of the study.

The personal integrity, sensitivity, and possible prejudices and/or biases of the investigators need to be taken into consideration as well. Personal biases can creep into how the research is conducted, alternative research methods used, and the preparation of surveys and questionnaires.

A common complaint in case study research is that investigators change direction during the course of the study unaware that their original research design was inadequate for the revised investigation. Thus, the researchers leave unknown gaps and biases in the study. To avoid this, researchers should report preliminary findings so that the likelihood of bias will be reduced.

Concerns about Reliability, Validity, and Generalizability

Merriam (1985) offers several suggestions for how case study researchers might actively combat the popular attacks on the validity, reliability, and generalizability of case studies:

• Prolong the Processes of Data Gathering on Site: This will help to insure the accuracy of the findings by providing the researcher with more concrete information upon which to formulate interpretations.
• Employ the Process of "Triangulation": Use a variety of data sources as opposed to relying solely upon one avenue of observation. One example of such a data check would be what McClintock, Brannon, and Maynard (1985) refer to as a "case cluster method," that is, when a single unit within a larger case is randomly sampled, and that data treated quantitatively." For instance, in Emig's (1971) study, the case cluster method was employed, singling out the productivity of a single student named Lynn. This cluster profile included an advanced case history of the subject, specific examination and analysis of individual compositions and protocols, and extensive interview sessions. The seven remaining students were then compared with the case of Lynn, to ascertain if there are any shared, or unique dimensions to the composing process engaged in by these eight students.
• Conduct Member Checks: Initiate and maintain an active corroboration on the interpretation of data between the researcher and those who provided the data. In other words, talk to your subjects.
• Collect Referential Materials: Complement the file of materials from the actual site with additional document support. For example, Emig (1971) supports her initial propositions with historical accounts by writers such as T.S. Eliot, James Joyce, and D.H. Lawrence. Emig also cites examples of theoretical research done with regards to the creative process, as well as examples of empirical research dealing with the writing of adolescents. Specific attention is then given to the four stages description of the composing process delineated by Helmoltz, Wallas, and Cowley, as it serves as the focal point in this study.
• Engage in Peer Consultation: Prior to composing the final draft of the report, researchers should consult with colleagues in order to establish validity through pooled judgment.

Although little can be done to combat challenges concerning the generalizability of case studies, "most writers suggest that qualitative research should be judged as credible and confirmable as opposed to valid and reliable" (Merriam 1985). Likewise, it has been argued that "rather than transplanting statistical, quantitative notions of generalizability and thus finding qualitative research inadequate, it makes more sense to develop an understanding of generalization that is congruent with the basic characteristics of qualitative inquiry" (1985). After all, criticizing the case study method for being ungeneralizable is comparable to criticizing a washing machine for not being able to tell the correct time. In other words, it is unjust to criticize a method for not being able to do something which it was never originally designed to do in the first place.

Annotated Bibliography

Armisted, C. (1984). How Useful are Case Studies. Training and Development Journal, 38 (2), 75-77.

This article looks at eight types of case studies, offers pros and cons of using case studies in the classroom, and gives suggestions for successfully writing and using case studies.

Bardovi-Harlig, K. (1997). Beyond Methods: Components of Second Language Teacher Education . New York: McGraw-Hill.

A compilation of various research essays which address issues of language teacher education. Essays included are: "Non-native reading research and theory" by Lee, "The case for Psycholinguistics" by VanPatten, and "Assessment and Second Language Teaching" by Gradman and Reed.

Bartlett, L. (1989). A Question of Good Judgment; Interpretation Theory and Qualitative Enquiry Address. 70th Annual Meeting of the American Educational Research Association. San Francisco.

Bartlett selected "quasi-historical" methodology, which focuses on the "truth" found in case records, as one that will provide "good judgments" in educational inquiry. He argues that although the method is not comprehensive, it can try to connect theory with practice.

Baydere, S. et. al. (1993). Multimedia conferencing as a tool for collaborative writing: a case study in Computer Supported Collaborative Writing. New York: Springer-Verlag.

The case study by Baydere et. al. is just one of the many essays in this book found in the series "Computer Supported Cooperative Work." Denley, Witefield and May explore similar issues in their essay, "A case study in task analysis for the design of a collaborative document production system."

Berkenkotter, C., Huckin, T., N., & Ackerman J. (1988). Conventions, Conversations, and the Writer: Case Study of a Student in a Rhetoric Ph.D. Program. Research in the Teaching of English, 22, 9-44.

The authors focused on how the writing of their subject, Nate or Ackerman, changed as he became more acquainted or familiar with his field's discourse community.

Berninger, V., W., and Gans, B., M. (1986). Language Profiles in Nonspeaking Individuals of Normal Intelligence with Severe Cerebral Palsy. Augmentative and Alternative Communication, 2, 45-50.

Argues that generalizations about language abilities in patients with severe cerebral palsy (CP) should be avoided. Standardized tests of different levels of processing oral language, of processing written language, and of producing written language were administered to 3 male participants (aged 9, 16, and 40 yrs).

Bockman, J., R., and Couture, B. (1984). The Case Method in Technical Communication: Theory and Models. Texas: Association of Teachers of Technical Writing.

Examines the study and teaching of technical writing, communication of technical information, and the case method in terms of those applications.

Boehrer, J. (1990). Teaching With Cases: Learning to Question. New Directions for Teaching and Learning, 42 41-57.

This article discusses the origins of the case method, looks at the question of what is a case, gives ideas about learning in case teaching, the purposes it can serve in the classroom, the ground rules for the case discussion, including the role of the question, and new directions for case teaching.

Bowman, W. R. (1993). Evaluating JTPA Programs for Economically Disadvantaged Adults: A Case Study of Utah and General Findings . Washington: National Commission for Employment Policy.

"To encourage state-level evaluations of JTPA, the Commission and the State of Utah co-sponsored this report on the effectiveness of JTPA Title II programs for adults in Utah. The technique used is non-experimental and the comparison group was selected from registrants with Utah's Employment Security. In a step-by-step approach, the report documents how non-experimental techniques can be applied and several specific technical issues can be addressed."

Boyce, A. (1993) The Case Study Approach for Pedagogists. Annual Meeting of the American Alliance for Health, Physical Education, Recreation and Dance. (Address). Washington DC.

This paper addresses how case studies 1) bridge the gap between teaching theory and application, 2) enable students to analyze problems and develop solutions for situations that will be encountered in the real world of teaching, and 3) helps students to evaluate the feasibility of alternatives and to understand the ramifications of a particular course of action.

Carson, J. (1993) The Case Study: Ideal Home of WAC Quantitative and Qualitative Data. Annual Meeting of the Conference on College Composition and Communication. (Address). San Diego.

"Increasingly, one of the most pressing questions for WAC advocates is how to keep [WAC] programs going in the face of numerous difficulties. Case histories offer the best chance for fashioning rhetorical arguments to keep WAC programs going because they offer the opportunity to provide a coherent narrative that contextualizes all documents and data, including what is generally considered scientific data. A case study of the WAC program, . . . at Robert Morris College in Pittsburgh demonstrates the advantages of this research method. Such studies are ideal homes for both naturalistic and positivistic data as well as both quantitative and qualitative information."

---. (1991). A Cognitive Process Theory of Writing. College Composition and Communication. 32. 365-87.

No abstract available.

Cromer, R. (1994) A Case Study of Dissociations Between Language and Cognition. Constraints on Language Acquisition: Studies of Atypical Children . Hillsdale: Lawrence Erlbaum Associates, 141-153.

Crossley, M. (1983) Case Study in Comparative and International Education: An Approach to Bridging the Theory-Practice Gap. Proceedings of the 11th Annual Conference of the Australian Comparative and International Education Society. Hamilton, NZ.

Case study research, as presented here, helps bridge the theory-practice gap in comparative and international research studies of education because it focuses on the practical, day-to-day context rather than on the national arena. The paper asserts that the case study method can be valuable at all levels of research, formation, and verification of theories in education.

Daillak, R., H., and Alkin, M., C. (1982). Qualitative Studies in Context: Reflections on the CSE Studies of Evaluation Use . California: EDRS

The report shows how the Center of the Study of Evaluation (CSE) applied qualitative techniques to a study of evaluation information use in local, Los Angeles schools. It critiques the effectiveness and the limitations of using case study, evaluation, field study, and user interview survey methodologies.

Davey, L. (1991). The Application of Case Study Evaluations. ERIC/TM Digest.

This article examines six types of case studies, the type of evaluation questions that can be answered, the functions served, some design features, and some pitfalls of the method.

Deutch, C. E. (1996). A course in research ethics for graduate students. College Teaching, 44, 2, 56-60.

This article describes a one-credit discussion course in research ethics for graduate students in biology. Case studies are focused on within the four parts of the course: 1) major issues, 2 )practical issues in scholarly work, 3) ownership of research results, and 4) training and personal decisions.

DeVoss, G. (1981). Ethics in Fieldwork Research. RIE 27p. (ERIC)

This article examines four of the ethical problems that can happen when conducting case study research: acquiring permission to do research, knowing when to stop digging, the pitfalls of doing collaborative research, and preserving the integrity of the participants.

Driscoll, A. (1985). Case Study of a Research Intervention: the University of Utah’s Collaborative Approach . San Francisco: Far West Library for Educational Research Development.

Paper presented at the annual meeting of the American Association of Colleges of Teacher Education, Denver, CO, March 1985. Offers information of in-service training, specifically case studies application.

Ellram, L. M. (1996). The Use of the Case Study Method in Logistics Research. Journal of Business Logistics, 17, 2, 93.

This article discusses the increased use of case study in business research, and the lack of understanding of when and how to use case study methodology in business.

Emig, J. (1971) The Composing Processes of Twelfth Graders . Urbana: NTCE.

This case study uses observation, tape recordings, writing samples, and school records to show that writing in reflexive and extensive situations caused different lengths of discourse and different clusterings of the components of the writing process.

Feagin, J. R. (1991). A Case For the Case Study . Chapel Hill: The University of North Carolina Press.

This book discusses the nature, characteristics, and basic methodological issues of the case study as a research method.

Feldman, H., Holland, A., & Keefe, K. (1989) Language Abilities after Left Hemisphere Brain Injury: A Case Study of Twins. Topics in Early Childhood Special Education, 9, 32-47.

"Describes the language abilities of 2 twin pairs in which 1 twin (the experimental) suffered brain injury to the left cerebral hemisphere around the time of birth and1 twin (the control) did not. One pair of twins was initially assessed at age 23 mo. and the other at about 30 mo.; they were subsequently evaluated in their homes 3 times at about 6-mo intervals."

Fidel, R. (1984). The Case Study Method: A Case Study. Library and Information Science Research, 6.

The article describes the use of case study methodology to systematically develop a model of online searching behavior in which study design is flexible, subject manner determines data gathering and analyses, and procedures adapt to the study's progressive change.

Flower, L., & Hayes, J. R. (1984). Images, Plans and Prose: The Representation of Meaning in Writing. Written Communication, 1, 120-160.

Explores the ways in which writers actually use different forms of knowing to create prose.

Frey, L. R. (1992). Interpreting Communication Research: A Case Study Approach Englewood Cliffs, N.J.: Prentice Hall.

The book discusses research methodologies in the Communication field. It focuses on how case studies bridge the gap between communication research, theory, and practice.

Gilbert, V. K. (1981). The Case Study as a Research Methodology: Difficulties and Advantages of Integrating the Positivistic, Phenomenological and Grounded Theory Approaches . The Annual Meeting of the Canadian Association for the Study of Educational Administration. (Address) Halifax, NS, Can.

This study on an innovative secondary school in England shows how a "low-profile" participant-observer case study was crucial to the initial observation, the testing of hypotheses, the interpretive approach, and the grounded theory.

Gilgun, J. F. (1994). A Case for Case Studies in Social Work Research. Social Work, 39, 4, 371-381.

This article defines case study research, presents guidelines for evaluation of case studies, and shows the relevance of case studies to social work research. It also looks at issues such as evaluation and interpretations of case studies.

Glennan, S. L., Sharp-Bittner, M. A. & Tullos, D. C. (1991). Augmentative and Alternative Communication Training with a Nonspeaking Adult: Lessons from MH. Augmentative and Alternative Communication, 7, 240-7.

"A response-guided case study documented changes in a nonspeaking 36-yr-old man's ability to communicate using 3 trained augmentative communication modes. . . . Data were collected in videotaped interaction sessions between the nonspeaking adult and a series of adult speaking."

Graves, D. (1981). An Examination of the Writing Processes of Seven Year Old Children. Research in the Teaching of English, 15, 113-134.

Hamel, J. (1993). Case Study Methods . Newbury Park: Sage. .

"In a most economical fashion, Hamel provides a practical guide for producing theoretically sharp and empirically sound sociological case studies. A central idea put forth by Hamel is that case studies must "locate the global in the local" thus making the careful selection of the research site the most critical decision in the analytic process."

Karthigesu, R. (1986, July). Television as a Tool for Nation-Building in the Third World: A Post-Colonial Pattern, Using Malaysia as a Case-Study. International Television Studies Conference. (Address). London, 10-12.

"The extent to which Television Malaysia, as a national mass media organization, has been able to play a role in nation building in the post-colonial period is . . . studied in two parts: how the choice of a model of nation building determines the character of the organization; and how the character of the organization influences the output of the organization."

Kenny, R. (1984). Making the Case for the Case Study. Journal of Curriculum Studies, 16, (1), 37-51.

The article looks at how and why the case study is justified as a viable and valuable approach to educational research and program evaluation.

Knirk, F. (1991). Case Materials: Research and Practice. Performance Improvement Quarterly, 4 (1 ), 73-81.

The article addresses the effectiveness of case studies, subject areas where case studies are commonly used, recent examples of their use, and case study design considerations.

Klos, D. (1976). Students as Case Writers. Teaching of Psychology, 3.2, 63-66.

This article reviews a course in which students gather data for an original case study of another person. The task requires the students to design the study, collect the data, write the narrative, and interpret the findings.

Leftwich, A. (1981). The Politics of Case Study: Problems of Innovation in University Education. Higher Education Review, 13.2, 38-64.

The article discusses the use of case studies as a teaching method. Emphasis is on the instructional materials, interdisciplinarity, and the complex relationships within the university that help or hinder the method.

Mabrito, M. (1991, Oct.). Electronic Mail as a Vehicle for Peer Response: Conversations of High and Low Apprehensive Writers. Written Communication, 509-32.

McCarthy, S., J. (1955). The Influence of Classroom Discourse on Student Texts: The Case of Ella . East Lansing: Institute for Research on Teaching.

A look at how students of color become marginalized within traditional classroom discourse. The essay follows the struggles of one black student: Ella.

Matsuhashi, A., ed. (1987). Writing in Real Time: Modeling Production Processes Norwood, NJ: Ablex Publishing Corporation.

Investigates how writers plan to produce discourse for different purposes to report, to generalize, and to persuade, as well as how writers plan for sentence level units of language. To learn about planning, an observational measure of pause time was used" (ERIC).

Merriam, S. B. (1985). The Case Study in Educational Research: A Review of Selected Literature. Journal of Educational Thought, 19.3, 204-17.

The article examines the characteristics of, philosophical assumptions underlying the case study, the mechanics of conducting a case study, and the concerns about the reliability, validity, and generalizability of the method.

---. (1988). Case Study Research in Education: A Qualitative Approach San Francisco: Jossey Bass.

Merry, S. E., & Milner, N. eds. (1993). The Possibility of Popular Justice: A Case Study of Community Mediation in the United States . Ann Arbor: U of Michigan.

". . . this volume presents a case study of one experiment in popular justice, the San Francisco Community Boards. This program has made an explicit claim to create an alternative justice, or new justice, in the midst of a society ordered by state law. The contributors to this volume explore the history and experience of the program and compare it to other versions of popular justice in the United States, Europe, and the Third World."

Merseth, K. K. (1991). The Case for Cases in Teacher Education. RIE. 42p. (ERIC).

This monograph argues that the case method of instruction offers unique potential for revitalizing the field of teacher education.

Michaels, S. (1987). Text and Context: A New Approach to the Study of Classroom Writing. Discourse Processes, 10, 321-346.

"This paper argues for and illustrates an approach to the study of writing that integrates ethnographic analysis of classroom interaction with linguistic analysis of written texts and teacher/student conversational exchanges. The approach is illustrated through a case study of writing in a single sixth grade classroom during a single writing assignment."

Milburn, G. (1995). Deciphering a Code or Unraveling a Riddle: A Case Study in the Application of a Humanistic Metaphor to the Reporting of Social Studies Teaching. Theory and Research in Education, 13.

This citation serves as an example of how case studies document learning procedures in a senior-level economics course.

Milley, J. E. (1979). An Investigation of Case Study as an Approach to Program Evaluation. 19th Annual Forum of the Association for Institutional Research. (Address). San Diego.

The case study method merged a narrative report focusing on the evaluator as participant-observer with document review, interview, content analysis, attitude questionnaire survey, and sociogram analysis. Milley argues that case study program evaluation has great potential for widespread use.

Minnis, J. R. (1985, Sept.). Ethnography, Case Study, Grounded Theory, and Distance Education Research. Distance Education, 6.2.

This article describes and defines the strengths and weaknesses of ethnography, case study, and grounded theory.

Nunan, D. (1992). Collaborative language learning and teaching . New York: Cambridge University Press.

Included in this series of essays is Peter Sturman’s "Team Teaching: a case study from Japan" and David Nunan’s own "Toward a collaborative approach to curriculum development: a case study."

Nystrand, M., ed. (1982). What Writers Know: The Language, Process, and Structure of Written Discourse . New York: Academic Press.

Owenby, P. H. (1992). Making Case Studies Come Alive. Training, 29, (1), 43-46. (ERIC)

This article provides tips for writing more effective case studies.

---. (1981). Pausing and Planning: The Tempo of Writer Discourse Production. Research in the Teaching of English, 15 (2),113-34.

Perl, S. (1979). The Composing Processes of Unskilled College Writers. Research in the Teaching of English, 13, 317-336.

"Summarizes a study of five unskilled college writers, focusing especially on one of the five, and discusses the findings in light of current pedagogical practice and research design."

Pilcher J. and A. Coffey. eds. (1996). Gender and Qualitative Research . Brookfield: Aldershot, Hants, England.

This book provides a series of essays which look at gender identity research, qualitative research and applications of case study to questions of gendered pedagogy.

Pirie, B. S. (1993). The Case of Morty: A Four Year Study. Gifted Education International, 9 (2), 105-109.

This case study describes a boy from kindergarten through third grade with above average intelligence but difficulty in learning to read, write, and spell.

Popkewitz, T. (1993). Changing Patterns of Power: Social Regulation and Teacher Education Reform. Albany: SUNY Press.

Popkewitz edits this series of essays that address case studies on educational change and the training of teachers. The essays vary in terms of discipline and scope. Also, several authors include case studies of educational practices in countries other than the United States.

---. (1984). The Predrafting Processes of Four High- and Four Low Apprehensive Writers. Research in the Teaching of English, 18, (1), 45-64.

Rasmussen, P. (1985, March) A Case Study on the Evaluation of Research at the Technical University of Denmark. International Journal of Institutional Management in Higher Education, 9 (1).

This is an example of a case study methodology used to evaluate the chemistry and chemical engineering departments at the University of Denmark.

Roth, K. J. (1986). Curriculum Materials, Teacher Talk, and Student Learning: Case Studies in Fifth-Grade Science Teaching . East Lansing: Institute for Research on Teaching.

Roth offers case studies on elementary teachers, elementary school teaching, science studies and teaching, and verbal learning.

Selfe, C. L. (1985). An Apprehensive Writer Composes. When a Writer Can't Write: Studies in Writer's Block and Other Composing-Process Problems . (pp. 83-95). Ed. Mike Rose. NMY: Guilford.

Smith-Lewis, M., R. and Ford, A. (1987). A User's Perspective on Augmentative Communication. Augmentative and Alternative Communication, 3, 12-7.

"During a series of in-depth interviews, a 25-yr-old woman with cerebral palsy who utilized augmentative communication reflected on the effectiveness of the devices designed for her during her school career."

St. Pierre, R., G. (1980, April). Follow Through: A Case Study in Metaevaluation Research . 64th Annual Meeting of the American Educational Research Association. (Address).

The three approaches to metaevaluation are evaluation of primary evaluations, integrative meta-analysis with combined primary evaluation results, and re-analysis of the raw data from a primary evaluation.

Stahler, T., M. (1996, Feb.) Early Field Experiences: A Model That Worked. ERIC.

"This case study of a field and theory class examines a model designed to provide meaningful field experiences for preservice teachers while remaining consistent with the instructor's beliefs about the role of teacher education in preparing teachers for the classroom."

Stake, R. E. (1995). The Art of Case Study Research. Thousand Oaks: Sage Publications.

This book examines case study research in education and case study methodology.

Stiegelbauer, S. (1984) Community, Context, and Co-curriculum: Situational Factors Influencing School Improvements in a Study of High Schools. Presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Discussion of several case studies: one looking at high school environments, another examining educational innovations.

Stolovitch, H. (1990). Case Study Method. Performance And Instruction, 29, (9), 35-37.

This article describes the case study method as a form of simulation and presents guidelines for their use in professional training situations.

Thaller, E. (1994). Bibliography for the Case Method: Using Case Studies in Teacher Education. RIE. 37 p.

This bibliography presents approximately 450 citations on the use of case studies in teacher education from 1921-1993.

Thrane, T. (1986). On Delimiting the Senses of Near-Synonyms in Historical Semantics: A Case Study of Adjectives of 'Moral Sufficiency' in the Old English Andreas. Linguistics Across Historical and Geographical Boundaries: In Honor of Jacek Fisiak on the Occasion of his Fiftieth Birthday . Berlin: Mouton de Gruyter.

United Nations. (1975). Food and Agriculture Organization. Report on the FAO/UNFPA Seminar on Methodology, Research and Country: Case Studies on Population, Employment and Productivity . Rome: United Nations.

This example case study shows how the methodology can be used in a demographic and psychographic evaluation. At the same time, it discusses the formation and instigation of the case study methodology itself.

Van Vugt, J. P., ed. (1994). Aids Prevention and Services: Community Based Research . Westport: Bergin and Garvey.

"This volume has been five years in the making. In the process, some of the policy applications called for have met with limited success, such as free needle exchange programs in a limited number of American cities, providing condoms to prison inmates, and advertisements that depict same-sex couples. Rather than dating our chapters that deal with such subjects, such policy applications are verifications of the type of research demonstrated here. Furthermore, they indicate the critical need to continue community based research in the various communities threatened by acquired immuno-deficiency syndrome (AIDS) . . . "

Welch, W., ed. (1981, May). Case Study Methodology in Educational Evaluation. Proceedings of the Minnesota Evaluation Conference. Minnesota. (Address).

The four papers in these proceedings provide a comprehensive picture of the rationale, methodology, strengths, and limitations of case studies.

Williams, G. (1987). The Case Method: An Approach to Teaching and Learning in Educational Administration. RIE, 31p.

This paper examines the viability of the case method as a teaching and learning strategy in instructional systems geared toward the training of personnel of the administration of various aspects of educational systems.

Yin, R. K. (1993). Advancing Rigorous Methodologies: A Review of 'Towards Rigor in Reviews of Multivocal Literatures.' Review of Educational Research, 61, (3).

"R. T. Ogawa and B. Malen's article does not meet its own recommended standards for rigorous testing and presentation of its own conclusions. Use of the exploratory case study to analyze multivocal literatures is not supported, and the claim of grounded theory to analyze multivocal literatures may be stronger."

---. (1989). Case Study Research: Design and Methods. London: Sage Publications Inc.

This book discusses in great detail, the entire design process of the case study, including entire chapters on collecting evidence, analyzing evidence, composing the case study report, and designing single and multiple case studies.

Related Links

Consider the following list of related Web sites for more information on the topic of case study research. Note: although many of the links cover the general category of qualitative research, all have sections that address issues of case studies.

• Sage Publications on Qualitative Methodology: Search here for a comprehensive list of new books being published about "Qualitative Methodology" http://www.sagepub.co.uk/
• The International Journal of Qualitative Studies in Education: An on-line journal "to enhance the theory and practice of qualitative research in education." On-line submissions are welcome. http://www.tandf.co.uk/journals/tf/09518398.html
• Qualitative Research Resources on the Internet: From syllabi to home pages to bibliographies. All links relate somehow to qualitative research. http://www.nova.edu/ssss/QR/qualres.html

Citation Information

Bronwyn Becker, Patrick Dawson, Karen Devine, Carla Hannum, Steve Hill, Jon Leydens, Debbie Matuskevich, Carol Traver, and Mike Palmquist. (1994-2024). Case Studies. The WAC Clearinghouse. Colorado State University. Available at https://wac.colostate.edu/repository/writing/guides/.

Copyright Information

Copyright © 1994-2024 Colorado State University and/or this site's authors, developers, and contributors . Some material displayed on this site is used with permission.

• Privacy Policy

Home » Case Study – Methods, Examples and Guide

Case Study – Methods, Examples and Guide

Table of Contents

A case study is a research method that involves an in-depth examination and analysis of a particular phenomenon or case, such as an individual, organization, community, event, or situation.

It is a qualitative research approach that aims to provide a detailed and comprehensive understanding of the case being studied. Case studies typically involve multiple sources of data, including interviews, observations, documents, and artifacts, which are analyzed using various techniques, such as content analysis, thematic analysis, and grounded theory. The findings of a case study are often used to develop theories, inform policy or practice, or generate new research questions.

Types of Case Study

Types and Methods of Case Study are as follows:

Single-Case Study

A single-case study is an in-depth analysis of a single case. This type of case study is useful when the researcher wants to understand a specific phenomenon in detail.

For Example , A researcher might conduct a single-case study on a particular individual to understand their experiences with a particular health condition or a specific organization to explore their management practices. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a single-case study are often used to generate new research questions, develop theories, or inform policy or practice.

Multiple-Case Study

A multiple-case study involves the analysis of several cases that are similar in nature. This type of case study is useful when the researcher wants to identify similarities and differences between the cases.

For Example, a researcher might conduct a multiple-case study on several companies to explore the factors that contribute to their success or failure. The researcher collects data from each case, compares and contrasts the findings, and uses various techniques to analyze the data, such as comparative analysis or pattern-matching. The findings of a multiple-case study can be used to develop theories, inform policy or practice, or generate new research questions.

Exploratory Case Study

An exploratory case study is used to explore a new or understudied phenomenon. This type of case study is useful when the researcher wants to generate hypotheses or theories about the phenomenon.

For Example, a researcher might conduct an exploratory case study on a new technology to understand its potential impact on society. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as grounded theory or content analysis. The findings of an exploratory case study can be used to generate new research questions, develop theories, or inform policy or practice.

Descriptive Case Study

A descriptive case study is used to describe a particular phenomenon in detail. This type of case study is useful when the researcher wants to provide a comprehensive account of the phenomenon.

For Example, a researcher might conduct a descriptive case study on a particular community to understand its social and economic characteristics. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a descriptive case study can be used to inform policy or practice or generate new research questions.

Instrumental Case Study

An instrumental case study is used to understand a particular phenomenon that is instrumental in achieving a particular goal. This type of case study is useful when the researcher wants to understand the role of the phenomenon in achieving the goal.

For Example, a researcher might conduct an instrumental case study on a particular policy to understand its impact on achieving a particular goal, such as reducing poverty. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of an instrumental case study can be used to inform policy or practice or generate new research questions.

Case Study Data Collection Methods

Here are some common data collection methods for case studies:

Interviews involve asking questions to individuals who have knowledge or experience relevant to the case study. Interviews can be structured (where the same questions are asked to all participants) or unstructured (where the interviewer follows up on the responses with further questions). Interviews can be conducted in person, over the phone, or through video conferencing.

Observations

Observations involve watching and recording the behavior and activities of individuals or groups relevant to the case study. Observations can be participant (where the researcher actively participates in the activities) or non-participant (where the researcher observes from a distance). Observations can be recorded using notes, audio or video recordings, or photographs.

Documents can be used as a source of information for case studies. Documents can include reports, memos, emails, letters, and other written materials related to the case study. Documents can be collected from the case study participants or from public sources.

Surveys involve asking a set of questions to a sample of individuals relevant to the case study. Surveys can be administered in person, over the phone, through mail or email, or online. Surveys can be used to gather information on attitudes, opinions, or behaviors related to the case study.

Artifacts are physical objects relevant to the case study. Artifacts can include tools, equipment, products, or other objects that provide insights into the case study phenomenon.

How to conduct Case Study Research

Conducting a case study research involves several steps that need to be followed to ensure the quality and rigor of the study. Here are the steps to conduct case study research:

• Define the research questions: The first step in conducting a case study research is to define the research questions. The research questions should be specific, measurable, and relevant to the case study phenomenon under investigation.
• Select the case: The next step is to select the case or cases to be studied. The case should be relevant to the research questions and should provide rich and diverse data that can be used to answer the research questions.
• Collect data: Data can be collected using various methods, such as interviews, observations, documents, surveys, and artifacts. The data collection method should be selected based on the research questions and the nature of the case study phenomenon.
• Analyze the data: The data collected from the case study should be analyzed using various techniques, such as content analysis, thematic analysis, or grounded theory. The analysis should be guided by the research questions and should aim to provide insights and conclusions relevant to the research questions.
• Draw conclusions: The conclusions drawn from the case study should be based on the data analysis and should be relevant to the research questions. The conclusions should be supported by evidence and should be clearly stated.
• Validate the findings: The findings of the case study should be validated by reviewing the data and the analysis with participants or other experts in the field. This helps to ensure the validity and reliability of the findings.
• Write the report: The final step is to write the report of the case study research. The report should provide a clear description of the case study phenomenon, the research questions, the data collection methods, the data analysis, the findings, and the conclusions. The report should be written in a clear and concise manner and should follow the guidelines for academic writing.

Examples of Case Study

Here are some examples of case study research:

• The Hawthorne Studies : Conducted between 1924 and 1932, the Hawthorne Studies were a series of case studies conducted by Elton Mayo and his colleagues to examine the impact of work environment on employee productivity. The studies were conducted at the Hawthorne Works plant of the Western Electric Company in Chicago and included interviews, observations, and experiments.
• The Stanford Prison Experiment: Conducted in 1971, the Stanford Prison Experiment was a case study conducted by Philip Zimbardo to examine the psychological effects of power and authority. The study involved simulating a prison environment and assigning participants to the role of guards or prisoners. The study was controversial due to the ethical issues it raised.
• The Challenger Disaster: The Challenger Disaster was a case study conducted to examine the causes of the Space Shuttle Challenger explosion in 1986. The study included interviews, observations, and analysis of data to identify the technical, organizational, and cultural factors that contributed to the disaster.
• The Enron Scandal: The Enron Scandal was a case study conducted to examine the causes of the Enron Corporation’s bankruptcy in 2001. The study included interviews, analysis of financial data, and review of documents to identify the accounting practices, corporate culture, and ethical issues that led to the company’s downfall.
• The Fukushima Nuclear Disaster : The Fukushima Nuclear Disaster was a case study conducted to examine the causes of the nuclear accident that occurred at the Fukushima Daiichi Nuclear Power Plant in Japan in 2011. The study included interviews, analysis of data, and review of documents to identify the technical, organizational, and cultural factors that contributed to the disaster.

Application of Case Study

Case studies have a wide range of applications across various fields and industries. Here are some examples:

Business and Management

Case studies are widely used in business and management to examine real-life situations and develop problem-solving skills. Case studies can help students and professionals to develop a deep understanding of business concepts, theories, and best practices.

Case studies are used in healthcare to examine patient care, treatment options, and outcomes. Case studies can help healthcare professionals to develop critical thinking skills, diagnose complex medical conditions, and develop effective treatment plans.

Case studies are used in education to examine teaching and learning practices. Case studies can help educators to develop effective teaching strategies, evaluate student progress, and identify areas for improvement.

Social Sciences

Case studies are widely used in social sciences to examine human behavior, social phenomena, and cultural practices. Case studies can help researchers to develop theories, test hypotheses, and gain insights into complex social issues.

Law and Ethics

Case studies are used in law and ethics to examine legal and ethical dilemmas. Case studies can help lawyers, policymakers, and ethical professionals to develop critical thinking skills, analyze complex cases, and make informed decisions.

Purpose of Case Study

The purpose of a case study is to provide a detailed analysis of a specific phenomenon, issue, or problem in its real-life context. A case study is a qualitative research method that involves the in-depth exploration and analysis of a particular case, which can be an individual, group, organization, event, or community.

The primary purpose of a case study is to generate a comprehensive and nuanced understanding of the case, including its history, context, and dynamics. Case studies can help researchers to identify and examine the underlying factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and detailed understanding of the case, which can inform future research, practice, or policy.

Case studies can also serve other purposes, including:

• Illustrating a theory or concept: Case studies can be used to illustrate and explain theoretical concepts and frameworks, providing concrete examples of how they can be applied in real-life situations.
• Developing hypotheses: Case studies can help to generate hypotheses about the causal relationships between different factors and outcomes, which can be tested through further research.
• Providing insight into complex issues: Case studies can provide insights into complex and multifaceted issues, which may be difficult to understand through other research methods.
• Informing practice or policy: Case studies can be used to inform practice or policy by identifying best practices, lessons learned, or areas for improvement.

Advantages of Case Study Research

There are several advantages of case study research, including:

• In-depth exploration: Case study research allows for a detailed exploration and analysis of a specific phenomenon, issue, or problem in its real-life context. This can provide a comprehensive understanding of the case and its dynamics, which may not be possible through other research methods.
• Rich data: Case study research can generate rich and detailed data, including qualitative data such as interviews, observations, and documents. This can provide a nuanced understanding of the case and its complexity.
• Holistic perspective: Case study research allows for a holistic perspective of the case, taking into account the various factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and comprehensive understanding of the case.
• Theory development: Case study research can help to develop and refine theories and concepts by providing empirical evidence and concrete examples of how they can be applied in real-life situations.
• Practical application: Case study research can inform practice or policy by identifying best practices, lessons learned, or areas for improvement.
• Contextualization: Case study research takes into account the specific context in which the case is situated, which can help to understand how the case is influenced by the social, cultural, and historical factors of its environment.

Limitations of Case Study Research

There are several limitations of case study research, including:

• Limited generalizability : Case studies are typically focused on a single case or a small number of cases, which limits the generalizability of the findings. The unique characteristics of the case may not be applicable to other contexts or populations, which may limit the external validity of the research.
• Biased sampling: Case studies may rely on purposive or convenience sampling, which can introduce bias into the sample selection process. This may limit the representativeness of the sample and the generalizability of the findings.
• Subjectivity: Case studies rely on the interpretation of the researcher, which can introduce subjectivity into the analysis. The researcher’s own biases, assumptions, and perspectives may influence the findings, which may limit the objectivity of the research.
• Limited control: Case studies are typically conducted in naturalistic settings, which limits the control that the researcher has over the environment and the variables being studied. This may limit the ability to establish causal relationships between variables.
• Time-consuming: Case studies can be time-consuming to conduct, as they typically involve a detailed exploration and analysis of a specific case. This may limit the feasibility of conducting multiple case studies or conducting case studies in a timely manner.
• Resource-intensive: Case studies may require significant resources, including time, funding, and expertise. This may limit the ability of researchers to conduct case studies in resource-constrained settings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Survey Research – Types, Methods, Examples

2.2 Approaches to Research

Learning objectives.

By the end of this section, you will be able to:

• Describe the different research methods used by psychologists
• Discuss the strengths and weaknesses of case studies, naturalistic observation, surveys, and archival research
• Compare longitudinal and cross-sectional approaches to research
• Compare and contrast correlation and causation

There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it. Some methods rely on observational techniques. Other approaches involve interactions between the researcher and the individuals who are being studied—ranging from a series of simple questions to extensive, in-depth interviews—to well-controlled experiments.

Each of these research methods has unique strengths and weaknesses, and each method may only be appropriate for certain types of research questions. For example, studies that rely primarily on observation produce incredible amounts of information, but the ability to apply this information to the larger population is somewhat limited because of small sample sizes. Survey research, on the other hand, allows researchers to easily collect data from relatively large samples. While this allows for results to be generalized to the larger population more easily, the information that can be collected on any given survey is somewhat limited and subject to problems associated with any type of self-reported data. Some researchers conduct archival research by using existing records. While this can be a fairly inexpensive way to collect data that can provide insight into a number of research questions, researchers using this approach have no control on how or what kind of data was collected. All of the methods described thus far are correlational in nature. This means that researchers can speak to important relationships that might exist between two or more variables of interest. However, correlational data cannot be used to make claims about cause-and-effect relationships.

Correlational research can find a relationship between two variables, but the only way a researcher can claim that the relationship between the variables is cause and effect is to perform an experiment. In experimental research, which will be discussed later in this chapter, there is a tremendous amount of control over variables of interest. While this is a powerful approach, experiments are often conducted in artificial settings. This calls into question the validity of experimental findings with regard to how they would apply in real-world settings. In addition, many of the questions that psychologists would like to answer cannot be pursued through experimental research because of ethical concerns.

Clinical or Case Studies

In 2011, the New York Times published a feature story on Krista and Tatiana Hogan, Canadian twin girls. These particular twins are unique because Krista and Tatiana are conjoined twins, connected at the head. There is evidence that the two girls are connected in a part of the brain called the thalamus, which is a major sensory relay center. Most incoming sensory information is sent through the thalamus before reaching higher regions of the cerebral cortex for processing.

Link to Learning

Watch this CBC video about Krista's and Tatiana's lives to learn more.

The implications of this potential connection mean that it might be possible for one twin to experience the sensations of the other twin. For instance, if Krista is watching a particularly funny television program, Tatiana might smile or laugh even if she is not watching the program. This particular possibility has piqued the interest of many neuroscientists who seek to understand how the brain uses sensory information.

These twins represent an enormous resource in the study of the brain, and since their condition is very rare, it is likely that as long as their family agrees, scientists will follow these girls very closely throughout their lives to gain as much information as possible (Dominus, 2011).

Over time, it has become clear that while Krista and Tatiana share some sensory experiences and motor control, they remain two distinct individuals, which provides invaluable insight for researchers interested in the mind and the brain (Egnor, 2017).

In observational research, scientists are conducting a clinical or case study when they focus on one person or just a few individuals. Indeed, some scientists spend their entire careers studying just 10–20 individuals. Why would they do this? Obviously, when they focus their attention on a very small number of people, they can gain a precious amount of insight into those cases. The richness of information that is collected in clinical or case studies is unmatched by any other single research method. This allows the researcher to have a very deep understanding of the individuals and the particular phenomenon being studied.

If clinical or case studies provide so much information, why are they not more frequent among researchers? As it turns out, the major benefit of this particular approach is also a weakness. As mentioned earlier, this approach is often used when studying individuals who are interesting to researchers because they have a rare characteristic. Therefore, the individuals who serve as the focus of case studies are not like most other people. If scientists ultimately want to explain all behavior, focusing attention on such a special group of people can make it difficult to generalize any observations to the larger population as a whole. Generalizing refers to the ability to apply the findings of a particular research project to larger segments of society. Again, case studies provide enormous amounts of information, but since the cases are so specific, the potential to apply what’s learned to the average person may be very limited.

Naturalistic Observation

If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances are that almost everyone in the classroom will raise their hand, but do you think hand washing after every trip to the restroom is really that universal?

This is very similar to the phenomenon mentioned earlier in this chapter: many individuals do not feel comfortable answering a question honestly. But if we are committed to finding out the facts about hand washing, we have other options available to us.

Suppose we send a classmate into the restroom to actually watch whether everyone washes their hands after using the restroom. Will our observer blend into the restroom environment by wearing a white lab coat, sitting with a clipboard, and staring at the sinks? We want our researcher to be inconspicuous—perhaps standing at one of the sinks pretending to put in contact lenses while secretly recording the relevant information. This type of observational study is called naturalistic observation : observing behavior in its natural setting. To better understand peer exclusion, Suzanne Fanger collaborated with colleagues at the University of Texas to observe the behavior of preschool children on a playground. How did the observers remain inconspicuous over the duration of the study? They equipped a few of the children with wireless microphones (which the children quickly forgot about) and observed while taking notes from a distance. Also, the children in that particular preschool (a “laboratory preschool”) were accustomed to having observers on the playground (Fanger, Frankel, & Hazen, 2012).

It is critical that the observer be as unobtrusive and as inconspicuous as possible: when people know they are being watched, they are less likely to behave naturally. If you have any doubt about this, ask yourself how your driving behavior might differ in two situations: In the first situation, you are driving down a deserted highway during the middle of the day; in the second situation, you are being followed by a police car down the same deserted highway ( Figure 2.7 ).

It should be pointed out that naturalistic observation is not limited to research involving humans. Indeed, some of the best-known examples of naturalistic observation involve researchers going into the field to observe various kinds of animals in their own environments. As with human studies, the researchers maintain their distance and avoid interfering with the animal subjects so as not to influence their natural behaviors. Scientists have used this technique to study social hierarchies and interactions among animals ranging from ground squirrels to gorillas. The information provided by these studies is invaluable in understanding how those animals organize socially and communicate with one another. The anthropologist Jane Goodall , for example, spent nearly five decades observing the behavior of chimpanzees in Africa ( Figure 2.8 ). As an illustration of the types of concerns that a researcher might encounter in naturalistic observation, some scientists criticized Goodall for giving the chimps names instead of referring to them by numbers—using names was thought to undermine the emotional detachment required for the objectivity of the study (McKie, 2010).

The greatest benefit of naturalistic observation is the validity , or accuracy, of information collected unobtrusively in a natural setting. Having individuals behave as they normally would in a given situation means that we have a higher degree of ecological validity, or realism, than we might achieve with other research approaches. Therefore, our ability to generalize the findings of the research to real-world situations is enhanced. If done correctly, we need not worry about people or animals modifying their behavior simply because they are being observed. Sometimes, people may assume that reality programs give us a glimpse into authentic human behavior. However, the principle of inconspicuous observation is violated as reality stars are followed by camera crews and are interviewed on camera for personal confessionals. Given that environment, we must doubt how natural and realistic their behaviors are.

The major downside of naturalistic observation is that they are often difficult to set up and control. In our restroom study, what if you stood in the restroom all day prepared to record people’s hand washing behavior and no one came in? Or, what if you have been closely observing a troop of gorillas for weeks only to find that they migrated to a new place while you were sleeping in your tent? The benefit of realistic data comes at a cost. As a researcher you have no control of when (or if) you have behavior to observe. In addition, this type of observational research often requires significant investments of time, money, and a good dose of luck.

Sometimes studies involve structured observation. In these cases, people are observed while engaging in set, specific tasks. An excellent example of structured observation comes from Strange Situation by Mary Ainsworth (you will read more about this in the chapter on lifespan development). The Strange Situation is a procedure used to evaluate attachment styles that exist between an infant and caregiver. In this scenario, caregivers bring their infants into a room filled with toys. The Strange Situation involves a number of phases, including a stranger coming into the room, the caregiver leaving the room, and the caregiver’s return to the room. The infant’s behavior is closely monitored at each phase, but it is the behavior of the infant upon being reunited with the caregiver that is most telling in terms of characterizing the infant’s attachment style with the caregiver.

Another potential problem in observational research is observer bias . Generally, people who act as observers are closely involved in the research project and may unconsciously skew their observations to fit their research goals or expectations. To protect against this type of bias, researchers should have clear criteria established for the types of behaviors recorded and how those behaviors should be classified. In addition, researchers often compare observations of the same event by multiple observers, in order to test inter-rater reliability : a measure of reliability that assesses the consistency of observations by different observers.

Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally ( Figure 2.9 ). Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.

Surveys allow researchers to gather data from larger samples than may be afforded by other research methods . A sample is a subset of individuals selected from a population , which is the overall group of individuals that the researchers are interested in. Researchers study the sample and seek to generalize their findings to the population. Generally, researchers will begin this process by calculating various measures of central tendency from the data they have collected. These measures provide an overall summary of what a typical response looks like. There are three measures of central tendency: mode, median, and mean. The mode is the most frequently occurring response, the median lies at the middle of a given data set, and the mean is the arithmetic average of all data points. Means tend to be most useful in conducting additional analyses like those described below; however, means are very sensitive to the effects of outliers, and so one must be aware of those effects when making assessments of what measures of central tendency tell us about a data set in question.

There is both strength and weakness of the survey in comparison to case studies. By using surveys, we can collect information from a larger sample of people. A larger sample is better able to reflect the actual diversity of the population, thus allowing better generalizability. Therefore, if our sample is sufficiently large and diverse, we can assume that the data we collect from the survey can be generalized to the larger population with more certainty than the information collected through a case study. However, given the greater number of people involved, we are not able to collect the same depth of information on each person that would be collected in a case study.

Another potential weakness of surveys is something we touched on earlier in this chapter: People don't always give accurate responses. They may lie, misremember, or answer questions in a way that they think makes them look good. For example, people may report drinking less alcohol than is actually the case.

Any number of research questions can be answered through the use of surveys. One real-world example is the research conducted by Jenkins, Ruppel, Kizer, Yehl, and Griffin (2012) about the backlash against the US Arab-American community following the terrorist attacks of September 11, 2001. Jenkins and colleagues wanted to determine to what extent these negative attitudes toward Arab-Americans still existed nearly a decade after the attacks occurred. In one study, 140 research participants filled out a survey with 10 questions, including questions asking directly about the participant’s overt prejudicial attitudes toward people of various ethnicities. The survey also asked indirect questions about how likely the participant would be to interact with a person of a given ethnicity in a variety of settings (such as, “How likely do you think it is that you would introduce yourself to a person of Arab-American descent?”). The results of the research suggested that participants were unwilling to report prejudicial attitudes toward any ethnic group. However, there were significant differences between their pattern of responses to questions about social interaction with Arab-Americans compared to other ethnic groups: they indicated less willingness for social interaction with Arab-Americans compared to the other ethnic groups. This suggested that the participants harbored subtle forms of prejudice against Arab-Americans, despite their assertions that this was not the case (Jenkins et al., 2012).

Archival Research

Some researchers gain access to large amounts of data without interacting with a single research participant. Instead, they use existing records to answer various research questions. This type of research approach is known as archival research . Archival research relies on looking at past records or data sets to look for interesting patterns or relationships.

For example, a researcher might access the academic records of all individuals who enrolled in college within the past ten years and calculate how long it took them to complete their degrees, as well as course loads, grades, and extracurricular involvement. Archival research could provide important information about who is most likely to complete their education, and it could help identify important risk factors for struggling students ( Figure 2.10 ).

In comparing archival research to other research methods, there are several important distinctions. For one, the researcher employing archival research never directly interacts with research participants. Therefore, the investment of time and money to collect data is considerably less with archival research. Additionally, researchers have no control over what information was originally collected. Therefore, research questions have to be tailored so they can be answered within the structure of the existing data sets. There is also no guarantee of consistency between the records from one source to another, which might make comparing and contrasting different data sets problematic.

Longitudinal and Cross-Sectional Research

Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again at age 40.

Another approach is cross-sectional research. In cross-sectional research , a researcher compares multiple segments of the population at the same time. Using the dietary habits example above, the researcher might directly compare different groups of people by age. Instead of studying a group of people for 20 years to see how their dietary habits changed from decade to decade, the researcher would study a group of 20-year-old individuals and compare them to a group of 30-year-old individuals and a group of 40-year-old individuals. While cross-sectional research requires a shorter-term investment, it is also limited by differences that exist between the different generations (or cohorts) that have nothing to do with age per se, but rather reflect the social and cultural experiences of different generations of individuals that make them different from one another.

To illustrate this concept, consider the following survey findings. In recent years there has been significant growth in the popular support of same-sex marriage. Many studies on this topic break down survey participants into different age groups. In general, younger people are more supportive of same-sex marriage than are those who are older (Jones, 2013). Does this mean that as we age we become less open to the idea of same-sex marriage, or does this mean that older individuals have different perspectives because of the social climates in which they grew up? Longitudinal research is a powerful approach because the same individuals are involved in the research project over time, which means that the researchers need to be less concerned with differences among cohorts affecting the results of their study.

Often longitudinal studies are employed when researching various diseases in an effort to understand particular risk factors. Such studies often involve tens of thousands of individuals who are followed for several decades. Given the enormous number of people involved in these studies, researchers can feel confident that their findings can be generalized to the larger population. The Cancer Prevention Study-3 (CPS-3) is one of a series of longitudinal studies sponsored by the American Cancer Society aimed at determining predictive risk factors associated with cancer. When participants enter the study, they complete a survey about their lives and family histories, providing information on factors that might cause or prevent the development of cancer. Then every few years the participants receive additional surveys to complete. In the end, hundreds of thousands of participants will be tracked over 20 years to determine which of them develop cancer and which do not.

Clearly, this type of research is important and potentially very informative. For instance, earlier longitudinal studies sponsored by the American Cancer Society provided some of the first scientific demonstrations of the now well-established links between increased rates of cancer and smoking (American Cancer Society, n.d.) ( Figure 2.11 ).

As with any research strategy, longitudinal research is not without limitations. For one, these studies require an incredible time investment by the researcher and research participants. Given that some longitudinal studies take years, if not decades, to complete, the results will not be known for a considerable period of time. In addition to the time demands, these studies also require a substantial financial investment. Many researchers are unable to commit the resources necessary to see a longitudinal project through to the end.

Research participants must also be willing to continue their participation for an extended period of time, and this can be problematic. People move, get married and take new names, get ill, and eventually die. Even without significant life changes, some people may simply choose to discontinue their participation in the project. As a result, the attrition rates, or reduction in the number of research participants due to dropouts, in longitudinal studies are quite high and increase over the course of a project. For this reason, researchers using this approach typically recruit many participants fully expecting that a substantial number will drop out before the end. As the study progresses, they continually check whether the sample still represents the larger population, and make adjustments as necessary.

As an Amazon Associate we earn from qualifying purchases.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/psychology-2e/pages/1-introduction

• Authors: Rose M. Spielman, William J. Jenkins, Marilyn D. Lovett
• Publisher/website: OpenStax
• Book title: Psychology 2e
• Publication date: Apr 22, 2020
• Location: Houston, Texas
• Book URL: https://openstax.org/books/psychology-2e/pages/1-introduction
• Section URL: https://openstax.org/books/psychology-2e/pages/2-2-approaches-to-research

© Jan 6, 2024 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

• About this Book
• Learning Theories
• Behaviorism
• Cognitivism
• Constructivism
• Socioculturalism
• Research Methods

Case Studies

• Design-Based Research
• Inferential Statistics
• Learning Analytics
• Psychometrics
• Design and Development
• Continuous Improvement Dashboards
• Design Layers
• Gamification
• Informal Learning
• Instructional Design Methods
• Personalized and Blended Learning
• Project-Based Learning
• Secure Web Application Development
• Socratic Seminar
• Visual Aesthetics
• List of Authors
• Translations

Choose a Sign-in Option

Tools and Settings

Questions and Tasks

Citation and Embed Code

Imagine your childhood neighborhood. Retrace the steps that you would take each day to play outside, visit friends, or simply explore. Perhaps there are many experiences that you dwell on, such as the place where you crashed your bike, the smell of cookies at your neighbor’s house, or the distance you ran when you raced your friends around the block. If you were to write a short story, in order to provide enough depth you would likely need to focus on just one aspect of your childhood experience in that neighborhood. Wilson (1996) remarks that an environment as rich as this has varying dimensions such as constancy and change, simplicity and complexity; you knew your way around in the neighborhood, but there was always more to find. The same is true for case studies in educational research. Case studies are a qualitative research method that focus on one unit of study (Merriam, 1998). This chapter seeks to clearly define case studies, explore their weaknesses and strengths, and discuss when and for what research questions they are most appropriate to use as an educational research method.

Defining a Case Study

In the book The Art of Case Studies, Robert E. Stake (1995) defines case study as “the study of the particularity and complexity of a single case, coming to understand its activity within important circumstances” (p. xi). Like other qualitative research methods, case studies provide a holistic view of their context. Case studies use a variety of qualitative research methods, such as observations and interviews, to provide rich detail. This rich detail makes case studies a useful tool for instruction and discussion in many subjects, such as business, law, and the social sciences, which includes education. Just like the example of your childhood neighborhood, case studies look closely at a slice of life.

Types of Case Studies

According to Merriam (1998), the types of case studies in educational research can be separated into four main categories: (1) ethnographic, (2) historical, (3) psychological and (4) sociological. In short, ethnographic case studies focus on how people behave in cultural settings, such as the culture within a classroom. Historical case studies use a variety of evidences to understand a context over time, such as the founding and development of a private school. Psychological case studies, such as studies by Piaget on his own children, look at individuals and analyze their behavior. Sociological case studies focus on social constructs and use demographics to analyze the case, such as socioeconomic differences within a school (Merriam, 1998).

Importance of Boundaries

In her widely-cited book Qualitative Research and Case Study Applications in Education, Sharan B. Merriam (1998) remarks that “the most single defining characteristic of case study research lies in delimiting the object of study, the case” (p. 27). Boundaries require researchers to scope their study. Researchers choose a bounded context which can contain a person, an organization, a class, a policy, or any given unit of study. Boundaries also help a researcher to define what will not be included in the study. If a researcher cannot state a limit to the number of participants or the amount of time their research requires, then it does not qualify as a case study (Merriam, 1998). Continuing with the example of your childhood neighborhood, you would need to decide what phenomenon in the neighborhood to focus on. If you chose to study the types of interactive play that occur in the neighborhood park, you would need to specify a length of time for the study and limit your observations to only what occurs in the boundaries of the park.

Weaknesses of Case Studies

Many critiques of case studies align with critiques of qualitative research methods in general. These include the time-consuming nature of data collection and analysis, the increased risk of researcher bias, and the lack of generalizability that could influence credibility (Johnson & Onwuegbuzie, 2004). As mentioned previously, case studies use a variety of qualitative research techniques, which often require a researcher to spend large amounts of time collecting data, finding the appropriate way to code and organize data, and analyzing the data to make sound conclusions. Some critics remark that qualitative research methods are more susceptible to data cherry-picking, when a researcher only presents evidence that matches their own position. Some are concerned that qualitative research is more susceptible to a researcher’s assumptions and biases. Frequently, policy-makers, administrators, and other leaders look to quantitative data for decision-making and view qualitative data as being too specific to just one context. This is perhaps the weakness that is most relevant to case studies.

Due to the boundaries that define a case study, the sample size for research is often small. Over-simplification and exaggeration can mislead a reader to think that a case study represents a greater part of the whole than is true (Merriam, 1998). Research with a smaller scope and sample size cannot find patterns across a wide sampling of cases, making it less generalizable. Data from a small sampling of participants may be dismissed as an outlier or as being unique to that specific group (Johnson & Onwuegbuzie, 2004). In contrast, quantitative data uses inferential statistics to find patterns and generalizable cases, which often speak to decision-makers because they appear to be more applicable to their own situation.

Strengths of Case Studies

Despite the common critiques of case studies outlined previously, the rich and holistic detail provided by case study has many advantages for researchers and other stakeholders. Complexities of a phenomenon within one case or context should be analyzed in depth, which requires time to observe, describe, and analyze. Other research methods would not provide this depth and detail, because they have a larger scope, which may limit them to collect more superficial data. Provided that a researcher is using appropriate techniques to collect and analyze data, the time is well spent to understand the context, because the resulting detail increases usefulness and transferability. Additionally, proper qualitative research always uses multiple methods to establish trustworthiness that acknowledges and reduces bias in a study, such as member checking or triangulation from multiple data sources. Like other qualitative methods, case studies are responsive to changes during the course of study and to the needs of the stakeholders (Johnson & Onwuegbuzie, 2004). This is especially true in case study, because the researcher is often immersed in the context, giving them a greater understanding of how to adapt. Additionally, case studies are frequently used to improve their own context, such as an evaluation of an educational program.

Case studies are not only valuable to the stakeholders within its bounded context; their rich detail makes them transferable to other contexts. Sometimes the generalizable knowledge produced from quantitative research is so broad and abstract that it is not useful to specific contexts (Johnson & Onwuegbuzie, 2004). Qualitative research as a whole does not seek to be generalizable; its value lies in its transferability. Generalizable research aims to apply its findings to the population at large, whereas transferable research must be applied by the reader as they make connections between the research and their own experiences. Transferability to other contexts can come from descriptions of decisions, structures, findings, and other principles found in the case. A reader may see connections to their own context that inform their thinking or decision-making. Perhaps they decide to avoid a course of action, because the case study showed that it did not work for one context, or they see how they can adapt their approach based on positive results in the case study. Rich detail is necessary for this kind of transferability.

Cross-Case Analysis

Also called collective, multicase, or comparative case studies, a cross-case analysis looks for similarities and differences between multiple case studies (Merriam 1998). Although a standard case study may have subunits, such as multiple students in a classroom, a cross-case analysis generally takes on a larger scope, such as multiple classrooms or multiple schools. A compelling cross-case analysis includes more cases and greater variety between the cases. Cross-case analysis requires rigorous comparison and interpretation, which strengthens the preciseness and stability of the research (Merriam 1998). The external validity, or generalizability, increases when patterns are found across cases because the sample size increases and the case results either confirm or negate each other. Thus, a researcher may choose to conduct a cross-case analysis if they plan to select and research multiple cases. Though the goal of the cross-case analysis is still transferability, this will increase the generalizability of the research results.

When to Use Case Studies

With the strengths and weaknesses of case studies in mind, I will now discuss when it is most appropriate to use case study as a qualitative research method in education. Recall that the most defining characteristic of a case study is its boundaries. It follows that a researcher should use case study as their research method when it is feasible and advantageous to set clear limits around their research. A case study is a method that suits many beginning researchers, because the scale is small and the context is focused. However, case studies should not be overly simplistic nor a mere description of what happens; like any research in education, they should be a worthwhile addition to the current literature (Rowley, 2002). This requires the researcher to know what is currently in the literature regarding the topic and where stronger evidence is needed or gaps in knowledge exist.

A research question should not be altered to fit a chosen research method; rather, a research method such as case study should be based on the research question. Case studies are particularly useful as preliminary research that provides a fresh perspective and sets the stage for future, related research. However, case studies can stand alone by rigorously describing and explaining a phenomenon (Rowley, 2002). Case studies answer “how” and “why” research questions with a high degree of detail. More specifically, case studies fit well when “a how or why question is being asked about a contemporary set of events over which the investigator has little or no control” (Yin, 1994, p. 9). For example, in my current case study research the questions I seek to answer have to do with how an organization has adopted a specific instructional design method to their context, why they have adopted it to that degree, and how that method has influenced the perceived quality of courses, the speed at which they are produced, and employee satisfaction. Note that the scope in these questions refers to only what is going on in the organization. Case study is a suitable method to answer these bounded research questions.

Just as it is compelling to attempt to describe one aspect of your childhood neighborhood, case studies are a valuable way of looking at the world, because they allow a researcher to set boundaries and focus on one unit of study. Although case studies are susceptible to common criticisms of qualitative research methods like small sample size, the rich detail they provide help to make them a learning tool that produces knowledge that is transferable to other contexts. Conducting a cross-case analysis would increase generalizability, because it seeks to find patterns across multiple cases. Case study should be selected for research questions that have an appropriate, bounded scope and seek to answer “how” and “why questions.” The fact that case studies are well-suited for beginning researchers does not diminish the importance of rigor or their value in educational research. Case studies are a useful research method in many fields, particularly education, because a holistic view within a bounded context brings about rich detail, which enhances the understanding of the researcher and reader alike.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.

Merriam, S. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.

Rowley, J. (2002). Using case studies in research. Management Research News, 25(1), 16-27.

Stake, R. (1995). The art of case study research. Thousand Oaks, CA: Sage Publications, Inc.

Wilson, B. G. (1996). Constructivist learning environments: Case studies in instructional design. Englewood Cliffs, NJ: Educational Technology Publications, Inc.

Yin, R.K. (1994). Case study research: design and methods (2nd ed.). Thousand Oaks, CA: Sage.

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/studentguide/case_studies .

Home » Pros and Cons » 12 Case Study Method Advantages and Disadvantages

12 Case Study Method Advantages and Disadvantages

A case study is an investigation into an individual circumstance. The investigation may be of a single person, business, event, or group. The investigation involves collecting in-depth data about the individual entity through the use of several collection methods. Interviews and observation are two of the most common forms of data collection used.

The case study method was originally developed in the field of clinical medicine. It has expanded since to other industries to examine key results, either positive or negative, that were received through a specific set of decisions. This allows for the topic to be researched with great detail, allowing others to glean knowledge from the information presented.

Here are the advantages and disadvantages of using the case study method.

List of the Advantages of the Case Study Method

1. it turns client observations into useable data..

Case studies offer verifiable data from direct observations of the individual entity involved. These observations provide information about input processes. It can show the path taken which led to specific results being generated. Those observations make it possible for others, in similar circumstances, to potentially replicate the results discovered by the case study method.

2. It turns opinion into fact.

Case studies provide facts to study because you’re looking at data which was generated in real-time. It is a way for researchers to turn their opinions into information that can be verified as fact because there is a proven path of positive or negative development. Singling out a specific incident also provides in-depth details about the path of development, which gives it extra credibility to the outside observer.

3. It is relevant to all parties involved.

Case studies that are chosen well will be relevant to everyone who is participating in the process. Because there is such a high level of relevance involved, researchers are able to stay actively engaged in the data collection process. Participants are able to further their knowledge growth because there is interest in the outcome of the case study. Most importantly, the case study method essentially forces people to make a decision about the question being studied, then defend their position through the use of facts.

4. It uses a number of different research methodologies.

The case study method involves more than just interviews and direct observation. Case histories from a records database can be used with this method. Questionnaires can be distributed to participants in the entity being studies. Individuals who have kept diaries and journals about the entity being studied can be included. Even certain experimental tasks, such as a memory test, can be part of this research process.

5. It can be done remotely.

Researchers do not need to be present at a specific location or facility to utilize the case study method. Research can be obtained over the phone, through email, and other forms of remote communication. Even interviews can be conducted over the phone. That means this method is good for formative research that is exploratory in nature, even if it must be completed from a remote location.

6. It is inexpensive.

Compared to other methods of research, the case study method is rather inexpensive. The costs associated with this method involve accessing data, which can often be done for free. Even when there are in-person interviews or other on-site duties involved, the costs of reviewing the data are minimal.

7. It is very accessible to readers.

The case study method puts data into a usable format for those who read the data and note its outcome. Although there may be perspectives of the researcher included in the outcome, the goal of this method is to help the reader be able to identify specific concepts to which they also relate. That allows them to discover unusual features within the data, examine outliers that may be present, or draw conclusions from their own experiences.

List of the Disadvantages of the Case Study Method

1. it can have influence factors within the data..

Every person has their own unconscious bias. Although the case study method is designed to limit the influence of this bias by collecting fact-based data, it is the collector of the data who gets to define what is a “fact” and what is not. That means the real-time data being collected may be based on the results the researcher wants to see from the entity instead. By controlling how facts are collected, a research can control the results this method generates.

2. It takes longer to analyze the data.

The information collection process through the case study method takes much longer to collect than other research options. That is because there is an enormous amount of data which must be sifted through. It’s not just the researchers who can influence the outcome in this type of research method. Participants can also influence outcomes by given inaccurate or incomplete answers to questions they are asked. Researchers must verify the information presented to ensure its accuracy, and that takes time to complete.

3. It can be an inefficient process.

Case study methods require the participation of the individuals or entities involved for it to be a successful process. That means the skills of the researcher will help to determine the quality of information that is being received. Some participants may be quiet, unwilling to answer even basic questions about what is being studied. Others may be overly talkative, exploring tangents which have nothing to do with the case study at all. If researchers are unsure of how to manage this process, then incomplete data is often collected.

4. It requires a small sample size to be effective.

The case study method requires a small sample size for it to yield an effective amount of data to be analyzed. If there are different demographics involved with the entity, or there are different needs which must be examined, then the case study method becomes very inefficient.

5. It is a labor-intensive method of data collection.

The case study method requires researchers to have a high level of language skills to be successful with data collection. Researchers must be personally involved in every aspect of collecting the data as well. From reviewing files or entries personally to conducting personal interviews, the concepts and themes of this process are heavily reliant on the amount of work each researcher is willing to put into things.

These case study method advantages and disadvantages offer a look at the effectiveness of this research option. With the right skill set, it can be used as an effective tool to gather rich, detailed information about specific entities. Without the right skill set, the case study method becomes inefficient and inaccurate.

Related Posts:

• 25 Best Ways to Overcome the Fear of Failure
• Monroe's Motivated Sequence Explained [with Examples]
• 21 Most Effective Bundle Pricing Strategies with Examples
• Force Field Analysis Explained with Examples

Case Study Method – 18 Advantages and Disadvantages

The case study method uses investigatory research as a way to collect data about specific demographics. This approach can apply to individuals, businesses, groups, or events. Each participant receives an equal amount of participation, offering information for collection that can then find new insights into specific trends, ideas, of hypotheses.

Interviews and research observation are the two standard methods of data collection used when following the case study method.

Researchers initially developed the case study method to develop and support hypotheses in clinical medicine. The benefits found in these efforts led the approach to transition to other industries, allowing for the examination of results through proposed decisions, processes, or outcomes. Its unique approach to information makes it possible for others to glean specific points of wisdom that encourage growth.

Several case study method advantages and disadvantages can appear when researchers take this approach.

List of the Advantages of the Case Study Method

1. It requires an intensive study of a specific unit. Researchers must document verifiable data from direct observations when using the case study method. This work offers information about the input processes that go into the hypothesis under consideration. A casual approach to data-gathering work is not effective if a definitive outcome is desired. Each behavior, choice, or comment is a critical component that can verify or dispute the ideas being considered.

Intensive programs can require a significant amount of work for researchers, but it can also promote an improvement in the data collected. That means a hypothesis can receive immediate verification in some situations.

2. No sampling is required when following the case study method. This research method studies social units in their entire perspective instead of pulling individual data points out to analyze them. That means there is no sampling work required when using the case study method. The hypothesis under consideration receives support because it works to turn opinions into facts, verifying or denying the proposals that outside observers can use in the future.

Although researchers might pay attention to specific incidents or outcomes based on generalized behaviors or ideas, the study itself won’t sample those situations. It takes a look at the “bigger vision” instead.

3. This method offers a continuous analysis of the facts. The case study method will look at the facts continuously for the social group being studied by researchers. That means there aren’t interruptions in the process that could limit the validity of the data being collected through this work. This advantage reduces the need to use assumptions when drawing conclusions from the information, adding validity to the outcome of the study over time. That means the outcome becomes relevant to both sides of the equation as it can prove specific suppositions or invalidate a hypothesis under consideration.

This advantage can lead to inefficiencies because of the amount of data being studied by researchers. It is up to the individuals involved in the process to sort out what is useful and meaningful and what is not.

4. It is a useful approach to take when formulating a hypothesis. Researchers will use the case study method advantages to verify a hypothesis under consideration. It is not unusual for the collected data to lead people toward the formulation of new ideas after completing this work. This process encourages further study because it allows concepts to evolve as people do in social or physical environments. That means a complete data set can be gathered based on the skills of the researcher and the honesty of the individuals involved in the study itself.

Although this approach won’t develop a societal-level evaluation of a hypothesis, it can look at how specific groups will react in various circumstances. That information can lead to a better decision-making process in the future for everyone involved.

5. It provides an increase in knowledge. The case study method provides everyone with analytical power to increase knowledge. This advantage is possible because it uses a variety of methodologies to collect information while evaluating a hypothesis. Researchers prefer to use direct observation and interviews to complete their work, but it can also advantage through the use of questionnaires. Participants might need to fill out a journal or diary about their experiences that can be used to study behaviors or choices.

Some researchers incorporate memory tests and experimental tasks to determine how social groups will interact or respond in specific situations. All of this data then works to verify the possibilities that a hypothesis proposes.

6. The case study method allows for comparisons. The human experience is one that is built on individual observations from group situations. Specific demographics might think, act, or respond in particular ways to stimuli, but each person in that group will also contribute a small part to the whole. You could say that people are sponges that collect data from one another every day to create individual outcomes.

The case study method allows researchers to take the information from each demographic for comparison purposes. This information can then lead to proposals that support a hypothesis or lead to its disruption.

7. Data generalization is possible using the case study method. The case study method provides a foundation for data generalization, allowing researches to illustrate their statistical findings in meaningful ways. It puts the information into a usable format that almost anyone can use if they have the need to evaluate the hypothesis under consideration. This process makes it easier to discover unusual features, unique outcomes, or find conclusions that wouldn’t be available without this method. It does an excellent job of identifying specific concepts that relate to the proposed ideas that researchers were verifying through their work.

Generalization does not apply to a larger population group with the case study method. What researchers can do with this information is to suggest a predictable outcome when similar groups are placed in an equal situation.

8. It offers a comprehensive approach to research. Nothing gets ignored when using the case study method to collect information. Every person, place, or thing involved in the research receives the complete attention of those seeking data. The interactions are equal, which means the data is comprehensive and directly reflective of the group being observed.

This advantage means that there are fewer outliers to worry about when researching an idea, leading to a higher level of accuracy in the conclusions drawn by the researchers.

9. The identification of deviant cases is possible with this method. The case study method of research makes it easier to identify deviant cases that occur in each social group. These incidents are units (people) that behave in ways that go against the hypothesis under consideration. Instead of ignoring them like other options do when collecting data, this approach incorporates the “rogue” behavior to understand why it exists in the first place.

This advantage makes the eventual data and conclusions gathered more reliable because it incorporates the “alternative opinion” that exists. One might say that the case study method places as much emphasis on the yin as it does the yang so that the whole picture becomes available to the outside observer.

10. Questionnaire development is possible with the case study method. Interviews and direct observation are the preferred methods of implementing the case study method because it is cheap and done remotely. The information gathered by researchers can also lead to farming questionnaires that can farm additional data from those being studied. When all of the data resources come together, it is easier to formulate a conclusion that accurately reflects the demographics.

Some people in the case study method may try to manipulate the results for personal reasons, but this advantage makes it possible to identify this information readily. Then researchers can look into the thinking that goes into the dishonest behaviors observed.

List of the Disadvantages of the Case Study Method

1. The case study method offers limited representation. The usefulness of the case study method is limited to a specific group of representatives. Researchers are looking at a specific demographic when using this option. That means it is impossible to create any generalization that applies to the rest of society, an organization, or a larger community with this work. The findings can only apply to other groups caught in similar circumstances with the same experiences.

It is useful to use the case study method when attempting to discover the specific reasons why some people behave in a specific way. If researchers need something more generalized, then a different method must be used.

2. No classification is possible with the case study method. This disadvantage is also due to the sample size in the case study method. No classification is possible because researchers are studying such a small unit, group, or demographic. It can be an inefficient process since the skills of the researcher help to determine the quality of the data being collected to verify the validity of a hypothesis. Some participants may be unwilling to answer or participate, while others might try to guess at the outcome to support it.

Researchers can get trapped in a place where they explore more tangents than the actual hypothesis with this option. Classification can occur within the units being studied, but this data cannot extrapolate to other demographics.

3. The case study method still offers the possibility of errors. Each person has an unconscious bias that influences their behaviors and choices. The case study method can find outliers that oppose a hypothesis fairly easily thanks to its emphasis on finding facts, but it is up to the researchers to determine what information qualifies for this designation. If the results from the case study method are surprising or go against the opinion of participating individuals, then there is still the possibility that the information will not be 100% accurate.

Researchers must have controls in place that dictate how data gathering work occurs. Without this limitation in place, the results of the study cannot be guaranteed because of the presence of bias.

4. It is a subjective method to use for research. Although the purpose of the case study method of research is to gather facts, the foundation of what gets gathered is still based on opinion. It uses the subjective method instead of the objective one when evaluating data, which means there can be another layer of errors in the information to consider.

Imagine that a researcher interprets someone’s response as “angry” when performing direct observation, but the individual was feeling “shame” because of a decision they made. The difference between those two emotions is profound, and it could lead to information disruptions that could be problematic to the eventual work of hypothesis verification.

5. The processes required by the case study method are not useful for everyone. The case study method uses a person’s memories, explanations, and records from photographs and diaries to identify interactions on influences on psychological processes. People are given the chance to describe what happens in the world around them as a way for researchers to gather data. This process can be an advantage in some industries, but it can also be a worthless approach to some groups.

If the social group under study doesn’t have the information, knowledge, or wisdom to provide meaningful data, then the processes are no longer useful. Researchers must weigh the advantages and disadvantages of the case study method before starting their work to determine if the possibility of value exists. If it does not, then a different method may be necessary.

6. It is possible for bias to form in the data. It’s not just an unconscious bias that can form in the data when using the case study method. The narrow study approach can lead to outright discrimination in the data. Researchers can decide to ignore outliers or any other information that doesn’t support their hypothesis when using this method. The subjective nature of this approach makes it difficult to challenge the conclusions that get drawn from this work, and the limited pool of units (people) means that duplication is almost impossible.

That means unethical people can manipulate the results gathered by the case study method to their own advantage without much accountability in the process.

7. This method has no fixed limits to it. This method of research is highly dependent on situational circumstances rather than overarching societal or corporate truths. That means the researcher has no fixed limits of investigation. Even when controls are in place to limit bias or recommend specific activities, the case study method has enough flexibility built into its structures to allow for additional exploration. That means it is possible for this work to continue indefinitely, gathering data that never becomes useful.

Scientists began to track the health of 268 sophomores at Harvard in 1938. The Great Depression was in its final years at that point, so the study hoped to reveal clues that lead to happy and healthy lives. It continues still today, now incorporating the children of the original participants, providing over 80 years of information to sort through for conclusions.

8. The case study method is time-consuming and expensive. The case study method can be affordable in some situations, but the lack of fixed limits and the ability to pursue tangents can make it a costly process in most situations. It takes time to gather the data in the first place, and then researchers must interpret the information received so that they can use it for hypothesis evaluation. There are other methods of data collection that can be less expensive and provide results faster.

That doesn’t mean the case study method is useless. The individualization of results can help the decision-making process advance in a variety of industries successfully. It just takes more time to reach the appropriate conclusion, and that might be a resource that isn’t available.

The advantages and disadvantages of the case study method suggest that the helpfulness of this research option depends on the specific hypothesis under consideration. When researchers have the correct skills and mindset to gather data accurately, then it can lead to supportive data that can verify ideas with tremendous accuracy.

This research method can also be used unethically to produce specific results that can be difficult to challenge.

When bias enters into the structure of the case study method, the processes become inefficient, inaccurate, and harmful to the hypothesis. That’s why great care must be taken when designing a study with this approach. It might be a labor-intensive way to develop conclusions, but the outcomes are often worth the investments needed.

Live revision! Join us for our free exam revision livestreams Watch now →

Reference Library

Collections

• See what's new
• All Resources
• Student Resources
• Assessment Resources
• Teaching Resources
• CPD Courses
• Livestreams

Study notes, videos, interactive activities and more!

Psychology news, insights and enrichment

Currated collections of free resources

Browse resources by topic

• All Psychology Resources

Resource Selections

Currated lists of resources

Study Notes

Case Studies

Last updated 22 Mar 2021

• Share on Facebook
• Share on Twitter
• Share by Email

Case studies are very detailed investigations of an individual or small group of people, usually regarding an unusual phenomenon or biographical event of interest to a research field. Due to a small sample, the case study can conduct an in-depth analysis of the individual/group.

Evaluation of case studies:

- Case studies create opportunities for a rich yield of data, and the depth of analysis can in turn bring high levels of validity (i.e. providing an accurate and exhaustive measure of what the study is hoping to measure).

- Studying abnormal psychology can give insight into how something works when it is functioning correctly, such as brain damage on memory (e.g. the case study of patient KF, whose short-term memory was impaired following a motorcycle accident but left his long-term memory intact, suggesting there might be separate physical stores in the brain for short and long-term memory).

- The detail collected on a single case may lead to interesting findings that conflict with current theories, and stimulate new paths for research.

- There is little control over a number of variables involved in a case study, so it is difficult to confidently establish any causal relationships between variables.

- Case studies are unusual by nature, so will have poor reliability as replicating them exactly will be unlikely.

- Due to the small sample size, it is unlikely that findings from a case study alone can be generalised to a whole population.

- The case study’s researcher may become so involved with the study that they exhibit bias in their interpretation and presentation of the data, making it challenging to distinguish what is truly objective/factual.

• Case Studies

You might also like

A level psychology topic quiz - research methods.

Quizzes & Activities

Case Studies: Example Answer Video for A Level SAM 3, Paper 1, Q4 (5 Marks)

Topic Videos

Research Methods: MCQ Revision Test 1 for AQA A Level Psychology

Example answers for research methods: a level psychology, paper 2, june 2018 (aqa).

Exam Support

Our subjects

• › Criminology
• › Economics
• › Geography
• › Health & Social Care
• › Psychology
• › Sociology
• › Teaching & learning resources
• › Student revision workshops
• › Online student courses
• › CPD for teachers
• › Livestreams
• › Teaching jobs

Boston House, 214 High Street, Boston Spa, West Yorkshire, LS23 6AD Tel: 01937 848885

• › Contact us
• › Terms of use
• › Privacy & cookies

© 2002-2024 Tutor2u Limited. Company Reg no: 04489574. VAT reg no 816865400.

Phineas Gage

In this post

You may have already heard of Phineas Gage, such is his infamous history with psychology. He was working on a railway line in the USA when there was an explosion, which resulted in an iron rod being fired through his head. He survived the accident even though there were serious injuries to his face and brain but it was soon discovered that in terms of his personality, he was completely different after the accident than he was before it.

Before the accident he was described as a very calm man who was very popular, but afterwards he was considered to be rude and irresponsible.

Gage died 12 years after the accident and after hearing of his death, his doctor, John Harlow, who had worked with him at the time of his accident, asked for his body to be exhumed so that he could look at his skull and try to identify how this caused the change in his personality.

Many years later, Damasio and her colleagues were able to make use of much better technology to further investigate the damage that had been caused to Phineas Gage’s brain and the effects that this had on his personality.

Damasio et al. aimed to build a replica model of Gage’s skull (using the actual skull as a guide) so that they could show exactly where the iron rod entered and exited Gage’s head.

A 3D representation of the skull and the injuries it received meant that it was much clearer which parts of his brain would have been affected by the accident and Damasio et al. wanted to see if any other areas of the brain had also been damaged.

• Pictures and measurements of Gage’s skull were taken
• A 3D replica model was built based on the information from the skull
• Information was also taken from the iron rod (which had been buried with Gage!)
• Information from the rod and the skull together meant that the trajectory of the iron rod could be accurately mapped
• Altogether 20 different points of entry and 16 points of exit were identified and the five most likely paths were chosen
• Each of these five paths were explored to map out which areas of Gage’s brain would have been damaged by each path.

It was thought that damage to both the left and right hemispheres of the brain were likely and that no other area than the frontal lobe would have been affected.

The iron rod would have gone through Gage’s left eye socket and then upwards in its trajectory. This means that rather than affecting the right frontal lobe, only the white matter (tissue containing nerve fibres) in the brain’s left hemisphere would have been affected. However, this meant that neural messages in this area of the brain would not have been transmitted because white matter is where neurons pass messages along axon fibres.

The findings from the 3D model and its implications for the parts of the brain that were thought to be damaged were compared to reports of the changes in Gage’s personality. It was concluded that a specific area of the frontal lobe (the ventromedial area) is responsible for making controlled decisions, regulating impulses and urges and dealing with emotions in a proper way.

These findings were compared to 12 other individuals who had experienced similar brain injuries and the same problems with control and impulse were found, showing that it is likely possible to predict the behaviour of people who have sustained this kind of brain injury.

Strengths of the study

• Modern-day technology is very reliable and therefore the 3D model that was created would have been very accurate and information could be ‘seen’ rather than just guessed at from written reports
• Predictions can now be made about people’s behaviour when they have experienced injuries in specific areas of the brain; this can help people to adjust to new lifestyles and may help in treating them as well.

Weaknesses of the study

• Information about the change in Gage’s personality were gleaned from details written more than a century ago, meaning its accuracy is questionable
• As this was a case study, it is difficult to generalise the findings to a wider population so predictions about possible changes in behaviour may not be applicable to everyone.

Interested in a Psychology GCSE?

We offer the Edexcel GCSE in Psychology through our online campus.

Learn more about our Psychology GCSE courses

Read another one of our posts

Understanding dementia: types, symptoms, and care needs.

A Comprehensive Guide to Health and Social Care Training

Understanding and Supporting Mental Health Conditions

The Role of Diet in Managing Chronic Diseases

Effective Communication Skills in Health and Social Care

Understanding Animal Training – Positive Reinforcement Techniques

Key Skills for Successful Care Home Management

Save your cart?

Estimating the proportions and latencies of reaction time outliers: A pooling method and case study of lexical decision tasks

• Original Manuscript
• Open access
• Published: 29 May 2024

Cite this article

You have full access to this open access article

• Jeff Miller   ORCID: orcid.org/0000-0003-2718-3153 1  

A methodological problem in most reaction time (RT) studies is that some measured RTs may be outliers—that is, they may be very fast or very slow for reasons unconnected to the task-related processing of interest. Numerous ad hoc methods have been suggested to discriminate between such outliers and the valid RTs of interest, but it is extremely difficult to determine how well these methods work in practice because virtually nothing is known about the actual characteristics of outliers in real RT datasets. This article proposes a new method of pooling cumulative distribution function values for examining empirical RT distributions to assess both the proportions of outliers and their latencies relative to those of the valid RTs. As the method is developed, its strengths and weaknesses are examined using simulations based on previously suggested ad hoc models for RT outliers with particular assumed proportions and distributions of valid RTs and outliers. The method is then applied to several large RT datasets from lexical decision tasks, and the results provide the first empirically based description of outlier RTs. For these datasets, fewer than 1% of the RTs seem to be outliers, and the median outlier latency appears to be approximately 4–6 standard deviations of RT above the mean of the valid RT distribution.

Avoid common mistakes on your manuscript.

Since the earliest days of experimental psychology, reaction time (RT) has been one of the most common measures of mental processes (Boring, 1929 ; Brebner & Welford, 1980 ). This measure is objective, is available cheaply and unobtrusively in a wide variety of tasks, and bears an obvious relation to the mental processes being measured. It also provides a convenient foundation for detailed quantitative models of the processes underlying specific tasks (Luce, 1986 ).

At the same time, RT presents challenges to researchers because of its well-documented large variability. Substantially different RTs will be obtained in different trials with seemingly identical conditions—that is, trials from a single participant, tested in the same task, with the same stimulus and response. The distributions of such within-condition RTs typically have standard deviations that are approximately 10% or more of their means and are generally skewed, with long tails of slow RTs often extending out past 2–3 times the mean (Luce, 1986 ). RT researchers have become increasingly interested in exploring the complex characteristics of RT distributions because these can provide useful information beyond what is available in simple summary measures like means and medians (e.g., Balota & Yap, 2011 ; Heathcote, Popiel, & Mewhort, 1991 ; Miller, 1982 ; Ruthruff, 1996 ).

A practical problem in the study of RT distributions is that some of the more extreme RTs may actually be outliers that do not reflect relevant task processing. Although most observed RTs are presumably valid ones determined by the normal processes involved in performing the required task, a few of the observed RTs may be generated by anomalous processes in which the researcher is not interested. For example, RTs may be too short (i.e., “fast outliers”) if participants accidentally trigger the response apparatus before performing the task or if they simply anticipate the stimulus in an attempt to perform quickly (Ollman, 1966 ). Likewise, measured RTs may be inappropriately long (i.e., “slow outliers”) if participants are momentarily distracted from the task (e.g., by coughing, fatigue, extraneous thoughts) or if a mechanical response does not register and has to be repeated because of some motor glitch. Naturally, to obtain a more realistic picture of the true distributions of processing times, RT researchers should identify and exclude such outlier RTs.

In most tasks, fast outliers are relatively easy to recognize. There is general agreement that most participants have a physiological lower limit of RT on the order of 100–150 ms (e.g., Luce, 1986 ), so RTs less than this can be excluded as fast outliers on theoretical grounds. Even in choice RT tasks where this physiological lower limit is not approached, the lower tails of empirical RT distributions often fall off sharply and accuracy is at chance for the fastest RTs, making it relatively easy to detect fast outliers that are distinct from the rest of the distribution (Cousineau et al., 2023 ).

Slow outliers are more difficult to identify, because they can normally only be detected when they are unrealistically large relative to the valid RTs. Yet, most stochastic latency mechanisms predict that quite long valid RTs will occasionally be generated by genuine task-related processes (e.g., Luce, 1986 ), which makes it very difficult to say just how large is “unrealistically large.” For example, with the long tails commonly found at the high end of skewed theoretical RT distributions, the slowest valid RT in a given set of trials (i.e., for a given participant in a given condition) could be several hundred milliseconds slower than the second-slowest valid RT. This makes it very difficult to distinguish between those RTs that are part of the real distribution (i.e., valid RTs) and those that are slow outliers. Obviously, researchers studying task-related processing mechanisms should retain slow valid RTs, because these provide potentially informative clues about those mechanisms. Thus, RTs should not be excluded as outliers simply because they are slow, but only when they are so slow that they were unlikely to have been valid RTs. The question is, just how slow is that? Unfortunately, no empirically-based description of these outliers has yet been developed (Berger & Kiefer, 2021 ; Ulrich & Miller, 1994 ), and it is difficult to develop methods for discriminating between the valid RTs and the slow outliers without knowing anything about how the outliers are distributed.

The purpose of this paper was to investigate the empirical characteristics of outliers in typical RT tasks. Since outliers are expected to be rare, very large datasets are needed to get an accurate picture of them. Fortunately, as will be described later, four large lexical decision task mega-studies have been conducted that can be used for this purpose. Using these datasets, a new method of pooling cumulative distribution function (CDF) values was developed in order to obtain information about both the proportions of outliers present in the empirical datasets and about the overall latencies of those outliers. It should be emphasized that the immediate goal of this work was not to obtain an improved method of classifying individual RTs as valid versus outliers. Instead, the method provides information about the characteristics of outliers in aggregate , somewhat analogous to techniques for estimating the proportions of people who would answer “yes” to sensitive questions without obtaining a definitive answer for any particular individual (e.g., asking elite athletes “Have you knowingly violated anti-doping regulations by using a prohibited substance or method in the past 12 months?”; Ulrich et al., 2018 ). Aggregate information about the proportions and latencies of outliers has implications regarding the effects of outliers on various methods of RT analysis and the utility of current methods of outlier exclusion. Eventually, it should also contribute to the development of improved methods of excluding individual RTs as outliers.

The organization of this paper is as follows. The next section describes several ad hoc models of outliers that have been suggested—with no specific empirical basis—in previous studies examining the efficiency of various RT data analysis procedures (e.g., procedures for outlier exclusion). These models are useful test beds in which to evaluate the proposed method for investigating outliers, because in their cases the truth about outliers is known a priori from the models’ assumptions. Following that, I propose a new method of CDF pooling across multiple participants to obtain information about the distributions of valid RTs and outliers—especially high-resolution information about the distribution tails where the outliers are most likely to be discernible. In the subsequent sections, I apply the new method both to RTs simulated from the previously suggested ad hoc outlier models, with known probabilities and distributions of valid and slow outlier RTs, and to real RTs from lexical decision tasks. Applying the method to simulated RTs helps to illustrate the method and more importantly to investigate its accuracy in uncovering the known characteristics of the simulated outliers. These analyses show that the pooling method can provide specific insights about the outliers present in a dataset. I then apply the method to real RTs from the lexical decision mega-studies to develop a picture of the outliers that can be expected in this important type of RT experiment. To preview the results, it appears that slow outliers are rare—less than 1% of trials—and that their latencies overlap with approximately the top 1–1.5% of the distribution of valid RTs.

Previous ad hoc models of outliers

Various intuitively appealing procedures for identifying outliers have been proposed and are often used (e.g., Berger & Kiefer, 2021 ; Cousineau & Chartier, 2010 ). The performance of these procedures has generally been assessed within the context of simulated datasets, assuming specific proportions of outliers and distributions of their latencies (see Miller, 2023 , for an exception). Given that no empirically based description of RT outliers has yet been developed (Berger & Kiefer, 2021 ; Ulrich & Miller, 1994 ), these assumptions have been somewhat arbitrary. Moreover, the divergence among the assumed outlier distributions illustrates the lack of consensus about the characteristics of these distributions.

Figure 1 illustrates representative examples of some previously assumed outlier distributions. The figure shows examples of hypothetical valid and outlier RT distributions for a single participant in a single condition; some simulation studies have compared different analysis options with a single set of fixed distributions like these, whereas other studies have allowed participant-to-participant variation in the parameters of the valid and outlier distributions.

Examples of probability density functions (PDFs) of valid reaction times (RTs), fast outliers, and slow outliers assumed by different researchers performing simulations examining RT analysis methods. The total areas of the different outlier distributions have been scaled upwards to facilitate visual comparisons among the different assumptions. a Gamma distribution of valid RTs. Fast outliers (2%) are 3 SDs below the mean valid RT and slow outliers (1%) are 6 SDs above that mean. b Ex-Gaussian distribution of valid RTs with slow outliers formed by adding a uniform 0–2000 random number to a valid RT. c Ex-Gaussian distributions of valid RTs, fast outliers, and slow outliers. d Ex-Gaussian distributions of valid RTs and slow outliers. e Ex-Gaussian distributions of valid RTs, fast outliers, and slow outliers. f Ex-Gaussian distribution of valid RTs. Fast outliers are uniformly distributed between 100 ms and the fastest valid RT in the sample. Slow outliers are slower than the slowest valid RT in the sample by an amount that varies uniformly from 0–2000 ms. Distributions of the fastest and slowest RTs in the samples were computed for a sample size of 60 trials

For example, Bush et al. ( 1993 ; henceforth, BHW) carried out a simulation with valid RTs coming from a chi-square distribution, plus 2% fast outliers and 1% slow outliers. Their outliers were constant values three standard deviations below the valid RT mean or six standard deviations above it Footnote 1 . In contrast, other researchers have generally assumed that valid RTs come from the ex-Gaussian distribution, which has parameters µ , σ , and τ (e.g., Balota & Yap, 2011 ; Hohle, 1965 ; Luce, 1986 ), but they have assumed different distributions for outliers. For example, in some simulations Ratcliff ( 1993 ; henceforth, RAT) included 10% slow outliers that were modelled as regular RTs with an added start-up delay uniformly distributed from 0–2000 ms (see also, Vankov, 2023 ). As can be seen in Fig. 1 b, this produces a very wide range of slow outliers and a substantial overlap of the distributions of valid RTs and slow outliers. Slow outliers can be nearly as fast as the fastest valid RTs (i.e., fast valid RTs with a near-zero start-up delay), but they can also be 2 s slower than the slowest valid RTs (i.e., slow valid RTs with a nearly 2 s delay)—quite different from the outlier distributions assumed by Bush et al. ( 1993 ). In various conditions, Ulrich and Miller ( 1994 ; henceforth, UM) and Van Selst and Jolicœur ( 1994 ; henceforth, VSJ) adopted specific ex-Gaussian distributions for the fast and slow outliers, with smaller or larger differences between the distributions of valid RTs and outliers. Berger and Kiefer ( 2021 ) generated valid RTs as ex-Gaussians whose parameters varied randomly across simulated participants, and they generated outliers using one of two distinct and somewhat complicated methods known as “overlap” and “tails” (henceforth, BK overlap and BK tails ; see Appendix 1 for details) Footnote 2 .

It is now well established by simulation studies that the effectiveness of different RT analysis procedures depends—among other things—on the frequencies and distributions of the simulated outliers (e.g., Berger & Kiefer, 2021 ; Bush et al., 1993 ; Ulrich & Miller, 1994 ). Thus, information about real outlier frequencies and latencies would clearly be very useful for deciding how researchers can most effectively analyze their real RT data. The next section describes a new method developed to obtain such information.

Pooling CDF values

In order to get a fine-grained picture of an RT distribution—and especially of its relatively rare outlier RTs—it is essential to have a frequency distribution tabulated across many thousands of observed trials. This presents a practical problem, because few participants are willing to provide that many trials per condition and—even if they did—there might be day-to-day variation due to practice or other factors. In order to get enough trials to study outliers, it would be helpful to have a method of pooling RTs from different distributions (i.e., from multiple participants and/or sessions) into a single overall distribution. As is described in detail in this section, the method proposed here is based on pooling estimated CDF values of the RTs from different distributions. Recall that the CDF value of a given score is the proportion of scores in its distribution that are smaller. For example, for the value of X = 1 . 96 from the standard normal distribution, the CDF value is  Y = F (1 . 96) = 0 . 975 because 97.5% of this distribution lies below the value of 1.96.

The rationale for the present CDF pooling is based on the following statistical fact that is central to the current approach: For any continuous random variable X , the distribution of the CDF values Y = F ( X ) is uniform over the interval 0–1, where F is the cumulative distribution function of X . Intuitively, this simply means that any score X is equally likely to come from every percentile of its distribution, from the lowest 1% [i.e., 0 < F ( X ) ≤ 0 . 01], up to the highest 1% [0 . 99 < F ( X ) ≤ 1].

Figure 2 illustrates this fact. In Fig. 2 a, the solid line shows a normal distribution with µ = 400 and σ = 100 superimposed over a histogram of 100,000 random X values sampled from this distribution. To construct the histogram in Fig. 2 b, the CDF value Y = F ( X ) of each random X value was computed relative to that true underlying normal distribution with its known µ and σ . The resulting histogram of these Y values is approximately uniform, and there are slight deviations from uniformity—just as there are slight deviations from the normal curve in Fig. 2 a—only because the sample size was finite. Because the X values in the tails of the distribution are less likely than those in the center (Fig. 2 a), the 1% ranges of X values must be correspondingly wider in the tails than in the center so that it will be equally likely (i.e., 1%) to get a score in any given 1% range of CDF values, by definition.

Theoretical probability density functions (lines) and frequency histograms of simulated observations (bars) with the PDFs scaled to cover the same areas as the histograms. a A theoretical normal distribution and a histogram of simulated values from it. b The theoretical distribution of the cumulative distribution function (CDF) values of scores from the normal distribution in a , and a histogram of the CDF values of the simulated observations in a . c A theoretical ex-Gaussian distribution and a histogram of simulated values from it. d The theoretical distribution of the CDF values of scores from that ex-Gaussian distribution and a histogram of the CDF values of the simulated observations in c . e The theoretical distribution of values pooled across the distributions in a and c , and the histogram of pooled simulated values. f The theoretical distribution of the pooled CDF values from b and d and the histogram of pooled simulated values

Figure 2 c and d show a parallel example starting with an ex-Gaussian distribution, which is a distribution typically found to provide a good approximation of empirical RTs (e.g., Balota & Yap, 2011 ; Hohle, 1965 ; Luce, 1986 ). Although the underlying ex-Gaussian is skewed (Fig. 2 c), the distribution of the CDF values computed for observations from that distribution is again uniform.

The fact that CDF values are always uniformly distributed from 0–1 means that the CDF values of scores from different distributions can be tabulated within a single combined distribution (i.e., “pooled”) without changing the shape of the pooled CDF distribution, even though the shapes of the original distributions are not preserved if the raw values from the original distributions are pooled. For example, Fig. 2 e shows the pooled frequency distribution tabulated across all 200,000 scores from the normal and ex-Gaussian distributions shown in Fig. 2 a and c. This pooled distribution looks very different from either of the underlying distributions from which it was obtained—it is obviously neither normal nor ex-Gaussian. Nonetheless, as shown in Fig. 2 f, the 200,000 pooled CDF values of the scores from these different distributions show the expected uniform distribution over the 0–1 interval. This must be the case, because the CDF values of the X values of the scores in different underlying distributions—relative to their own distributions—necessarily have the same uniform distribution even though the distributions of the underlying X values differ.

The presence of outliers would perturb the expected uniform 0–1 CDF distribution, however, if the CDF values of the outliers were computed relative to the valid distribution. By definition, fast and slow outliers would be concentrated, respectively, in the lowest and highest 1% or so of the valid distribution, so their CDFs would all be near zero or 1. To the extent that fast and slow outliers were present in the dataset, they would tend to increase the frequency of near-zero and near-1 CDFs, which would produce increased frequency “spikes” in the lowest and highest bins of a CDF histogram. Although this increase might not be visible in a single participant’s frequency distribution because of the limited number of trials available, it could well become clear if CDF values were pooled across participants. Figure 3 illustrates this in a simulated dataset consisting of five participants with various true ex-Gaussian distributions of valid RTs, depicted by the solid lines in Fig. 3 a–e. The µ , σ , and τ of each participant’s valid RT distribution are indicated in the figure, and 985 RTs were randomly selected from this distribution for each participant. To model the presence of 1.5% slow outliers, 15 additional RTs were sampled specifically from the top 1% of the participant’s true distribution. The histograms of the 1000 RTs from each participant are shown in Fig. 3 a–e, and it is impossible to see a clear indication of outliers in these histograms. This illustrates the difficulty of identifying outliers in the long slow tails of observed RT distributions, which was mentioned earlier. The CDFs of each participant’s 1000 RTs were then computed relative to that participant’s true distribution of valid RTs . The histogram of these 5000 pooled CDF values is shown in Fig. 3 f. The presence of the simulated slow outliers is clearly visible in the spike for the highest CDF bin, reflecting the fact that the 15 outlier RTs for each participant were all selected from that bin in their participant’s own valid RT distribution. Thus, pooling of CDF values across participants may provide a view of the outliers that cannot be seen by looking at the participants’ individual distributions of observed RTs.

Histograms (bars) and associated theoretical probability density functions (lines) scaled to cover the same areas as the histograms. a – e Histograms of 1000 randomly sampled reaction times (RTs) for each of five hypothetical participants (Participants 1–5). For each participant, 985 valid RTs were sampled from an ex-Gaussian distribution with the parameters indicated, and 15 slow outliers were sampled from the top 1% of that distribution. f Pooled frequency distribution of the CDF values of all participants’ RTs scored within their individual ex-Gaussian valid RT distributions (bin size = 2.5%)

Evaluation of pooling method in theory

To see how well the CDF pooling method can reveal the presence of various types of RT outliers, I first checked its performance on simulated datasets with known outliers and known distributions of valid RTs. This is a best-case scenario for the method in the sense that all CDFs can be computed relative to known true valid RT distributions. None of these computations could be carried out with real data, however, because in that case the true underlying distributions are unknown. Thus, the subsequent section evaluates how well the method would work in practice, where the unknown valid RT distributions must be estimated from the observed RTs, which could themselves include outliers.

The theoretical outlier models depicted in Fig. 1 were used for this purpose, and separate simulations were conducted with each model using relatively low, medium, and high proportions of outliers. For each combination of model and outlier proportion, 200 RTs were randomly generated for each of 1000 simulated participants. For all models, the valid RTs were randomly generated from ex-Gaussian distributions to simulate trial-to-trial variability. To simulate participant-to-participant variability, the values of each simulated participant’s valid RT ex-Gaussian parameters µ , σ , and τ were randomly chosen from separate hyper-parameter distributions suggested by Berger and Kiefer ( 2021 ). Each participant’s true proportion of slow outliers was also chosen randomly, and in different simulations these proportions were chosen from distributions with relatively low, medium, or high proportions of outliers overall (i.e., mean proportions of 1%, 2.5%, or 5%). The numerical values of the outlier RTs were generated differently for each outlier model in accordance with the assumptions of that model (Fig. 1 ; see Appendix 1 for additional details). After the simulated 200 RTs were randomly generated for each participant, the CDFs of all RTs—including any outliers—were computed relative to that participant’s valid RT distribution with its true values of µ , σ , and τ , and these CDF values were pooled across all simulated participants. Since all of the slow outlier CDF values would be near 1 within the valid RT distributions, they should produce a spike at the high end of each CDF distribution.

The black dots in Fig. 4 summarize the most important results of these simulations. These dots represent the tops of histograms showing the frequencies of the different CDF values pooled across simulated participants for each model and each outlier proportion, and these show the expected pattern. CDF values are almost uniformly distributed across most of the possible 0–1 range, with only small fluctuations due to the limited sample of 1000 × 200 = 200 , 000 trials. In contrast, there is clearly a frequency spike indicated by the rising black dots in the largest CDF bin due to the presence of slow outliers. Furthermore, this spike is larger when the proportion of outliers is higher. Thus, these frequencies support the idea that the CDF pooling method can provide a window on RT outliers. Small differences among models can also be seen in the second- and third-highest bins, with elevated frequencies in these bins mainly for models allowing some overlap of the valid and outlier RT distributions (cf. Fig. 1 ), suggesting that empirical data might even provide evidence concerning the amount of overlap between the distributions of valid and outlier RTs, as will be considered later.

Relative frequencies of the cumulative distribution function (CDF and \(\widehat{{\text{CDF}}}\) ) values of reaction times (RTs) simulated from the outlier models depicted in Fig. 1 . The dots show the frequencies of CDF values computed from each simulated participant’s true underlying ex-Gaussian RT distribution. The histograms show the frequencies of \(\widehat{{\text{CDF}}}\) values computed from ex-Gaussian distributions with parameters estimated from each participant’s simulated RTs and with the proportion of outliers randomly selected for each participant from uniform distributions ranging from zero to 0.02, 0.05, or 0.1 in simulations with low, medium, and high proportions of outliers, respectively

Evaluation of pooling method with estimated parameter values

Unfortunately, in actual practice with real datasets, researchers cannot tabulate the frequencies of CDF values analogous to those shown by the black dots in Fig. 4 . Even under the assumption that every participant’s valid RTs come from an ex-Gaussian distribution, the true values of each participant’s underlying µ , σ , and τ parameters are needed in order to compute these CDF values, but with real participants the true values of these parameters are unknown. At best, the researcher can only estimate the values of these parameters from the participant’s observed RTs, and then compute an estimated CDF, \(\widehat{CDF}\) , of each observed RT relative to the ex-Gaussian distribution with the estimated parameter values Footnote 3 . The problem of unknown parameter values is of course ubiquitous in model fitting, since it is always necessary to estimate the true values of hypothesized parameters from observed data. For the present CDF pooling method, parameter estimation seems problematic not only because the µ , σ , and τ estimates obtained from a sample of RTs will deviate from the true values due to sampling but also—and more seriously—because the outliers within an RT sample can distort the estimates of the valid RT distribution’s parameters from that sample.

Fortunately, it is possible to see how seriously the CDF pooling method would be compromised by the use of estimated rather than true µ , σ , and τ values. For each of the participants simulated in creating Fig. 4 , the 200 simulated RTs were used to compute maximum likelihood estimates of \(\widehat{\mu }\) , \(\widehat{\sigma }\) , and \(\widehat{\tau }\) , just as a researcher could do with real observed RTs (e.g., Lacouture & Cousineau, 2008 ). The estimated \(\widehat{CDF}\) of each simulated RT from each participant was then computed from the ex-Gaussian distribution with the estimated values \(\widehat{\mu }\) , \(\widehat{\sigma }\) , and \(\widehat{\tau }\) . The gray histograms in Fig. 4 show the relative frequencies of these 200,000 \(\widehat{CDF}\) values pooled across simulated participants for each model and proportion of outliers.

The effects of the previously mentioned distortions of the pooled \(\widehat{CDF}\) values due to estimation of the ex-Gaussian parameters can be seen as the difference between the black dots (CDFs) and the gray histogram bars ( \(\widehat{CDF}\) s) shown in Fig. 4 . There is very little distortion in the simulations with low probabilities of outliers, as indicated by nearly uniform \(\widehat{CDF}\) frequencies across most of the 0–1 range, with a spike in the highest \(\widehat{CDF}\) bin, similar to the pattern with CDFs. The distortion increases as the outlier probability increases, with clear departures from a uniform \(\widehat{CDF}\) distribution, especially for \(\widehat{CDF}\) values greater than 0.8 for some of the outlier models (e.g., VSJ and BK tails ). In most cases it is still possible to see an increased frequency of \(\widehat{CDF}\) values in the highest bin, however, suggesting again that the pooling method may be able to reveal the presence of outliers using estimated rather than true CDF values and thus that it might be informative with real data.

The high-end spikes shown in Fig. 4 are consistently smaller for \(\widehat{CDF}\) values computed from estimated parameters than for CDFs computed from true parameters, especially when the proportion of outliers is high. This must be kept in mind when applying the method to real RT datasets, because it suggests that the method is likely to underestimate the true proportion of outliers. This is to be expected, because the presence of slow outliers in the RT samples biases the estimates of the ex-Gaussian parameters—particularly τ (see Table 5 )—and thereby distorts the estimated \(\widehat{CDF}\) values relative to the true CDFs. Fortunately, the distortion appears relatively small when the proportions of outliers are low or medium, as would be expected in most carefully conducted RT studies.

In summary, the main message of Fig. 4 is that slow outliers can generally be seen as high-end spikes in the distributions of \(\widehat{CDF}\) values computed and pooled across participants in the proposed manner. Depending on the exact relationships between the valid and slow outlier distributions and the proportion of outliers, the spikes can be limited to the most extreme \(\widehat{CDF}\) values or can extend down to slightly lower \(\widehat{CDF}\) values. As expected, these spikes are larger when there is a larger proportion of outliers. Together, these simulation results indicate that the proposed pooling method can provide some information about the presence of outliers in real RT samples. The following two sections report results of applying this method to real data from four mega-studies of RTs in lexical decision tasks.

The Semantic Priming Project dataset

Given the preceding simulation-based evidence that the CDF pooling method provides useful information about RT outliers, I next used the method to investigate outliers in real datasets. I used the data from mega-studies of the lexical decision task because this is a well-known and important RT task within experimental psychology and because these datasets are unusually large and thus provide rich information for characterizing the distributions of both valid RTs and occasional outliers. I first applied the method to the publicly available Semantic Priming Project (SPP) dataset of Hutchison et al. ( 2013 ), which includes approximately 1600 trial-by-trial RTs recorded from each of 503 participants categorizing individual letter strings as words versus nonwords in a lexical decision task. For completeness, this section provides a detailed description of the application of the method to this dataset. As a check on the generality of the conclusions from the SPP dataset, parallel investigations were also carried out for several other lexical decision task datasets for comparison purposes, and their results are reported in the following section.

Recorded RTs in the SPP dataset range from 0–2999 ms, with the upper limit reflecting the maximum interval allowed for responding before the trial ended. For the present analysis, RTs less than 150 ms ( ≈ 0 . 5%) were excluded as unrealistically fast, because this value is less than that typically regarded as a lower estimate for simple RT and the present choice-task RTs should be even slower than simple RTs Footnote 4 . It may seem strange to eliminate obviously too-fast RTs from an analysis designed to identify outliers, but retaining them would increase noise and make it more difficult to assess the presence of outliers among the RTs whose validity is in doubt. Thus, the present method should be viewed as a way of investigating what proportion of the questionable RTs are outliers, after the obvious outliers have been removed. No RT upper bound was set because participants might conceivably need several seconds to decide whether a letter string is a word or not, and the maximum RT recording interval of 2999 ms eliminates the possibility of any obviously too-long RTs (e.g., 30 s).

Following the exclusion of the unrealistically fast RTs, a preliminary ANOVA on individual-participant mean correct RTs showed highly significant effects of word versus nonword trial, response repetition versus response alternation (i.e., whether the preceding trial’s response had been the same or different), and the interaction of these two factors ( p < 0 . 001 and η p 2 > 0 . 34). In view of the different mean RTs in these four conditions, it seemed appropriate to examine their RT distributions separately. Thus, the correct RTs of each participant were approximately equally divided across four conditions defined by whether the current stimulus was a word versus nonword and by response repetitions versus alternations, resulting in approximately 400 RTs per participant in each condition. The CDF pooling method was then applied separately and independently to the RTs in each of these four conditions, providing a type of within-experiment replication of the analysis.

As mentioned earlier, the ex-Gaussian distribution is typically used as a model for RT distributions. Given the crucial status of the assumed distribution model for the computation of \(\widehat{CDF}\) values, however, it is important to confirm that this distribution is actually a good model for the 2012 RT distributions observed in the SPP study (i.e., 503 participants times four conditions). To that end, as described in detail in Appendix 2 , a preliminary analysis was carried out to compare the fits of the observed RTs to the ex-Gaussian distribution versus their fits to 29 other candidate RT distributions. Each of 30 candidate RT distributions described in Appendix 2 was fit separately to each of 2012 observed RT distributions, and the ex-Gaussian emerged as the best-fitting distribution. Thus, for each of the 2012 combinations of participant and condition, ex-Gaussian parameter estimates \(\widehat{\mu }\) , \(\widehat{\sigma }\) , and \(\widehat{\tau }\) were determined by maximum likelihood, and the \(\widehat{CDF}\) of each individual RT was computed relative to the ex-Gaussian distribution with parameters equal to those estimates.

Figure 5 shows the pooled distributions of \(\widehat{CDF}\) values for correct responses in each of four conditions from all participants in the SPP dataset (comprising a total of 796,066 individual RTs). Even though the four pooled distributions were computed from the RTs of separate sets of trials, the results are remarkably similar across conditions. In all four conditions, the distributions are fairly uniform over all but the highest bin, which has a much higher frequency than expected. For example, in their 0.99–1 bins, the four conditions contain 1.49%, 1.40%, 1.41%, and 1.41% of the \(\widehat{CDF}\) values, and each of these percentages is significantly greater than the 1% that would be expected if there were no outliers ( p < 0 . 00001 in each condition, by binomial tests). It seems plausible that the highest bins are over-represented in frequency precisely because they include occasional slow outliers. The lack of analogous spikes at the low ends of the four distributions suggests that fast outliers above the 150 ms cut-off were non-existent or extremely rare.

Histogram of pooled estimated cumulative distribution function ( \(\widehat{{\text{CDF}}}\) ) values of RTs for the Semantic Priming Project dataset

The fairly uniform distribution across the lower bins in Fig. 5 also provides further evidence that the ex-Gaussian is a reasonably good distributional model for the observed RTs. If each RT’s \(\widehat{CDF}\) value was computed relative to a distribution substantially different from the true one—for example, a distribution from a different family or a distribution from the same family but with parameter values far from those estimated—then the pooled \(\widehat{CDF}\) values would not generally be uniformly distributed. Thus, the reasonably uniform distributions of pooled \(\widehat{CDF}\) values in Fig. 5 —apart from the highest bin—provide reassurance both that the ex-Gaussian is a good distributional family for modelling the RTs in this task and that there were sufficient RTs per participant in each condition to get good estimates of the ex-Gaussian parameter values for each participant/condition combination. There is a small but consistent non-uniformity across the four frequency distributions in Fig. 5 , however: there were slightly fewer RTs than expected with \(\widehat{CDF}\) values in the range of 80–98%. Similar trends will also be seen in some of the datasets presented later, so this is probably not a chance finding. The explanation could well be the same as that for the analogous pattern seen in Fig. 4 ; the presence of slow outliers inflates the estimated values of τ , so there would not be quite as many RTs in the upper tail of the distribution as expected based on that inflated τ .

Other lexical decision datasets

Analyses parallel to those carried out with the SPP dataset were also carried out with three other large lexical decision task datasets in order to check the robustness of the method and the generality of the conclusions. These datasets came from the Dutch Lexicon Project (DLP; Keuleers et al., 2010 ), the English Lexicon Project (ELP; Balota et al., 2007 ), and the French Lexicon Project (FLP; Ferrand et al., 2010 ). The DLP had extensive testing per person, with approximately 25,000 correct RTs from each of 39 participants, whereas the ELP and FLP datasets were more similar to the SPP dataset, including approximately 800–900 participants with 2000–3000 trials each. The maximum RT was 2000 ms in the DLP and FLP datasets due to a limited response interval, whereas it was 4000 ms in the ELP dataset. As with the SPP dataset, preliminary analyses of all datasets showed strong response repetition effects, so RT distributions were estimated for the same four conditions as the SPP dataset. For two of these three datasets, the ex-Gaussian provided the best fit to the individual participant and condition distributions, and for one it provided the second-best fit (Fig. B1), so it was again used for computation of the pooled \(\widehat{CDF}\) values.

Figures 6 , 7 and 8 show the pooled \(\widehat{CDF}\) distributions for the DLP, ELP, and FLP datasets, computed with the same method as the distributions for the SPP dataset shown in Fig. 5 . The distributions for the DLP and ELP datasets are similar to those obtained with the SPP dataset. They show reasonably flat distributions of \(\widehat{CDF}\) values across most of the 0–1 range except for spikes of increased frequency for \(\widehat{CDF}\) values near 1, exactly as would be expected if the observed RTs came from ex-Gaussian distributions with occasional slow outliers. For the DLP and ELP datasets, the eight different 0.99–1 bins contain 1.17–1.35% of the trials, and all of these values are significantly greater than the 1% expected if there were no outliers (all p < 0 . 00001). For the FLP dataset, however, there are only small spikes at the highest \(\widehat{CDF}\) values for the two word conditions (1.07% and 1.10%, both p < 0 . 001), with no such spikes in the distributions for nonwords. The weaker evidence for high outliers in the FLP dataset may be due to a procedural aspect of this study. As mentioned earlier, its maximum RT interval was only 2 s, so it may be that outliers were eliminated by trial termination. The maximum RT interval was also only 2 s in the DLP study, but the limitation on response time may have had less effect in this case because the participants had so much more practice with the task (i.e., approximately 25,000 versus 1800 trials per participant) and because the mean RT was correspondingly approximately 100 ms less.

Histogram of pooled estimated cumulative distribution function ( \(\widehat{{\text{CDF}}}\) ) values of RTs for the Dutch Lexicon Project dataset

Histogram of pooled estimated cumulative distribution function ( \(\widehat{{\text{CDF}}}\) ) values of RTs for the English Lexicon Project dataset

Histogram of pooled estimated cumulative distribution function ( \(\widehat{{\text{CDF}}}\) ) values of RTs for the French Lexicon Project dataset

Estimating outlier proportions: Known outlier models

If the \(\widehat{CDF}\) distributions shown in Figs. 5 , 6 , 7 , and 8 do reflect mixtures of the \(\widehat{CDF}\) values of valid RTs and those of occasional slow outliers, then it should be possible to estimate, for each dataset, the proportion of outliers and their \(\widehat{CDF}\) values with a simple mixture model. Let p o be the probability of an outlier within the dataset under consideration. Then 1 − p o is the proportion of valid RTs, and the \(\widehat{CDF}\) values of these are approximately uniform 0–1, as already explained. The \(\widehat{CDF}\) values for the slow outliers follow some other distribution whose exact shape is uncertain. Since the slow outliers are by definition quite large relative to valid RTs, this distribution should be concentrated near the maximum of the estimated valid distribution (i.e., the \(\widehat{CDF}\) values of the outliers should be near 1.0), with decreasing frequency for smaller outlier \(\widehat{CDF}\) values (e.g., 0.998, 0.997, and so on). Thus, it seems reasonable to model the \(\widehat{CDF}\) values of the outliers as having a wedge-like half-triangular distribution with its mode at the maximum possible \(\widehat{CDF}\) of 1.0 and a density function decreasing linearly to 0.0 at some minimum outlier \(\widehat{CDF}\) , ω , which is a free parameter of the mixture model. For example, Fig. 9 shows an overall predicted \(\widehat{CDF}\) distribution for this mixture model, combining \(\widehat{CDF}\) values from valid RTs and outliers, with p o = 0 . 03 and ω = 0 . 98. The distribution of \(\widehat{CDF}\) values is perfectly flat for \(\widehat{CDF}\) values from zero to ω , which can only come from valid RTs. It then increases linearly for \(\widehat{CDF}\) values from ω to 1, which can come from both valid RTs and slow outliers, because of the increasing half-triangular density of outlier \(\widehat{CDF}\) values. The linear increase that is evident for the largest \(\widehat{CDF}\) values in Fig. 9 may not be visible in a pooled histogram of \(\widehat{CDF}\) values, however, if ω falls within the histogram’s largest bin (e.g., with ω = 0 . 991 for the bin size of 0.01 shown in Fig. 4 ). In that case the top bin will show a frequency spike due to the outliers, but a narrower bin width would be needed to show the linear increase in frequencies of the different \(\widehat{CDF}\) values within the highest bin.

\(\widehat{{\text{CDF}}}\) distribution predicted by the mixture model with p o = 0.03 and ω = 0.98

In summary, the proposed mixture model has two free parameters: the outlier probability, p o , and the \(\widehat{CDF}\) value (relative to the distribution of valid RTs) of the fastest possible slow outlier, ω . For both the simulated and real datasets explored previously, these parameters were estimated by maximum likelihood. The results of these fits for the histograms in Fig. 4 are summarized in Table 1 . The parallel analyses for the real datasets (Figs. 5 , 6 , 7 , and 8 ) are shown in Table 2 .

Since the true properties of the underlying distributions were known in the simulations using the models in Fig. 1 , the parameter estimates in Table 1 can be evaluated relative to the known values of those parameters. For example, the true mean proportions of outliers were 0.01, 0.025, and 0.05 for the low, medium, and high conditions, respectively (see Appendix 1 ). The corresponding estimates of p o appropriately increased with the true mean proportion of outliers for all models, suggesting that the model would be useful in comparing the proportions of outliers across different participant groups or conditions. Note, however, that p o underestimates the true outlier probabilities in all cases, which must be kept in mind when interpreting obtained p o values—the true values could be 1.5–2 times as large as the estimates. The estimate of p o was most accurate with the BHW and VSJ models, and it was least accurate with the RAT and UM models. This suggests that the model’s estimate of p o is especially likely to be too low when the distribution of outlier RTs overlaps with the distribution of valid RTs, as it does most strongly for the RAT and UM (see Fig. 1 ).

Across the six different outlier models, the estimates of ω clearly reflect the differing assumed degrees of overlap between the distributions of valid and outlier RTs. The estimates of ω are in the range of approximately 0.93–0.97 for the UM and BK overlap models, correctly indicating that with these models the outlier distribution overlaps approximately 3–7% of the valid RT distribution at the slow end. For the BHW, VSJ, and BK tails models, whose outliers are more extreme relative to the valid RTs, the estimates of ω were all greater than 0.99, correctly indicating that these outliers were larger than all but the very largest valid RTs. Thus, in these five cases the value of ω reflects outlier speeds fairly well.

Another clear trend in the ω estimates is that they decrease as the proportion of outliers increases. This is a consequence of the increasing bias in the estimates of τ for larger outlier proportions (see Table 5 ). As the valid distribution’s τ is increasingly overestimated with a larger outlier proportion, a given outlier RT will be located less far out in the tail of the fitted valid RT distribution and thus have a less extreme \(\widehat{CDF}\) .

The ω estimates for the RAT model were badly incorrect, however. The estimates for this model were greater than 0.99 with all three outlier proportions, although the actual distribution of outlier RTs overlaps most of the valid RT distribution with this model (Fig. 1 b). This misestimation of ω may be caused by the mixture model’s inability to identify the presence of many outliers of this type in the first place (i.e., small estimates of p o ). Since the model does not recognize that outliers are present, it cannot provide good estimates of their \(\widehat{CDF}\) values. It is difficult to see how any RT-based method could identify the proportion of outliers under this model in view of the extensive overlap of the distributions of valid and outlier RTs Footnote 5 .

In sum, the analyses of this section suggest that the pooled CDF approach and associated mixture model provide useful information about the proportions and relative latencies of outliers in RT data, especially when there is little overlap between the distributions of valid and outlier RTs, as could plausibly be expected in well-controlled experimental studies. Where there is only a small true proportion of outliers, the pooled \(\widehat{CDF}\) values may be adequately described by a mixture model, and the true proportion of outliers seems to be approximately 1.5–2 times the size of the proportion estimated from this model. In addition, the ω parameter of the model gives a rough indication of the relative extremity of the outlier RTs within the distribution of valid RTs.

Estimating outlier proportions: Lexical decision datasets

The mixture model developed in the previous section can be applied to the pooled \(\widehat{CDF}\) values from the lexical decision tasks shown in Figs. 5 , 6 , 7 , and 8 to estimate the proportions and latencies of outliers in these real datasets. The results obtained with simulated data in the previous section indicate that these estimates—although they will not be perfect—will provide useful approximations of the two mixture model parameters.

Table 2 shows the results of fitting of the mixture model to each of the real datasets of lexical decision RTs, and several points are noteworthy. First, the estimated outlier probabilities, p o , are low for all datasets and conditions. These estimated p o values are slightly less than those obtained in simulations of the models of Fig. 1 with low (i.e., 0–2%) outlier probabilities, which suggests that there were probably fewer than 1% outliers overall in any of these datasets. Second, the large (i.e., nearly 1) ω values in the SPP, DLP, and ELP datasets suggest that these outliers fall within the top 1% of the valid RT distributions. Notably, the ω values are much lower for the FLP dataset, especially in the nonword conditions. Since there is little evidence of a slow outlier peak of \(\widehat{CDF}\) values for this dataset in the first place (Fig. 8 ), it is not surprising that the mixture model is unable to provide stable estimates of the outliers’ locations within these \(\widehat{CDF}\) distributions.

Estimating the distribution of outlier RTs

The mixture model developed in the previous section provides an estimate of the distribution of \(\widehat{CDF}\) values of the outlier RTs. It is possible to use this hypothesized distribution to estimate numerical outlier RT values in milliseconds relative to any researcher-selected reference distribution. Specifically, for an assumed reference distribution of valid RTs, one can work backwards from the estimated distribution of outlier \(\widehat{CDF}\) values to compute the corresponding outlier RTs in milliseconds by “looking up” each of the outlier \(\widehat{CDF}\) values from the half-triangular outlier component of the mixture model. The RT with a given \(\widehat{CDF}\) value within the valid RT reference distribution is the corresponding numerical value of the outlier with that \(\widehat{CDF}\) value. For example, for the UM outlier model and the low true proportion of outliers, the fitted mixture model includes outliers with a half-triangular distribution of \(\widehat{CDF}\) values ranging from 0.971–1. The median of this distribution can be computed to be a \(\widehat{CDF}\) of 0.9915.

What does this mean in terms of RTs? To have a concrete example, assume arbitrarily that the ex-Gaussian ( µ = 300, σ = 30, τ = 100) distribution is chosen as the reference distribution of valid RTs. The mean and standard deviation of this distribution can be computed to be 400 ms and 104 ms, respectively. Relative to this reference distribution, the median outlier RT must be the RT with the CDF value of 0.9915—the previously computed median outlier \(\widehat{CDF}\) . Computations with this reference distribution show that this median outlier is an RT of 781 ms—a value which is approximately 3.8 standard deviations above the reference distribution’s mean. Thus, the mixture model provides not only an estimate of the proportion of outliers but also an estimate of the actual numerical values of those outliers relative to any given assumed reference distribution of valid RTs. Of course, these numerical values (e.g., 781, 3.8) are specific to this particular assumed reference distribution rather than values characteristic of RT tasks in general, but they illustrate how corresponding values could be obtained for a reference distribution regarded as typical for any particular task.

Figure 10 illustrates the accuracy of this procedure for estimating median outlier RTs using the simulated RTs from the models with known outliers in Fig. 1 and the ex-Gaussian ( µ = 300, σ = 30, τ = 100) reference distribution shown for comparison in the violin plots. For each model and proportion of outliers, I first computed the median outlier \(\widehat{CDF}\) within the half-triangular outlier distribution from the fitted mixture model, as explained above. The value of RT with that CDF value within the reference distribution is then the mixture model’s estimate of the median outlier RT relative to the reference distribution, and these values are plotted with the dashed lines. For comparison, the true CDF value of each simulated outlier RT was computed relative to each participant’s true valid RT distribution, and the median of these CDF values was determined. The value of the reference distribution with this median true CDF value, shown as the solid line, would be a better estimate of the median outlier RT, but it cannot be computed in practice because researchers can only compute estimated \(\widehat{CDF}\) values—not true CDF values. Note that the true CDFs of the outlier RTs and their corresponding median in the reference distribution do not depend on the proportion of slow outliers. This proportion influences the estimation process through its impact on parameter estimation (e.g., Table 5 ), but it does not influence the true values of the outlier RTs or their true CDFs within the valid RT distributions.

Median RTs of outliers relative to the ex-Gaussian (µ = 300, σ = 30, τ = 100) reference distribution, as computed from the true CDFs of the outliers and from the \(\widehat{{\text{CDF}}}\) values estimated from the mixture model. The reference distribution of valid RTs shown as a violin plot was the same in all conditions

The results shown in Fig. 10 suggest that the mixture model provides a reasonable estimate of the median outlier RT relative to the reference distribution. The medians estimated from the mixture model are not exactly equal to those computed from the true CDFs, of course, due to the estimation bias and random error inherent in that model. The estimated medians are generally similar to the true medians, though, at least when compared to the range of values in the reference distribution shown by the violin plots. Looking across the different panels of Fig. 10 , it is clear that the mixture model generally reproduces the differences among models in the sizes of the outliers relative to the valid RTs. The mixture model’s predicted median outlier RTs are relatively fast (i.e., approximately 800 ms; Fig. 10 c and e) for the outlier models whose outliers are only slightly slower than the slowest valid RTs (Fig. 1 c and e), whereas the mixture model’s predicted median outlier RTs are distinctly slower (i.e., approximately 1000 ms or more; Fig. 10 a, b, d, and f) for the outlier models whose slow outliers are more extreme (Fig. 1 a, b, d, and f). The mixture model’s predicted median outlier RTs are generally less extreme than they should be, and this underestimation increases when there are more slow outliers. This pattern of underestimation is presumably a further consequence of the bias in τ estimation, which makes the estimated \(\widehat{CDF}\) values less extreme than their true values. Interestingly, the RAT model is an exception to the general pattern of underestimation, as can be seen in Fig. 10 b. In this case the mixture model’s estimate of the median outlier RT tends to be somewhat larger than the true value, unlike the estimates for the other outlier models. The explanation for this difference is not clear, but it is not surprising that the mixture approach works differently for this outlier model given that its outliers are qualitatively different than those of the other models and that the approach so badly overestimates this model’s minimum outlier \(\widehat{CDF}\) , ω , in the first place (see Table 1 ).

It is even more interesting to see what this type of analysis suggests about the latencies of outliers in real RT data. Table 3 shows the median outlier RTs estimated as described above using \(\widehat{CDF}\) values from the four real lexical decision task datasets, again using the ex-Gaussian ( µ = 300, σ = 30, τ = 100) as the reference distribution. The results look reasonable for all of the datasets except FLP. The absolute times are plausible, with median outlier RTs approximately 4–6 standard deviations of RT above the mean of the valid RT distribution. In addition, the median outlier RTs are consistent across the four conditions within each experiment, as would be expected if the outliers were randomly distributed across conditions. The fact that the outliers were faster in the DLP dataset than in the SPP and ELP datasets could be due to the use of the shorter 2 s RT interval in the former study, as opposed to the 3 s and 4 s intervals in the latter studies. However, the median outlier RTs do not seem plausible in the FLP dataset—particularly for the nonword conditions—because they are not nearly slow enough. This is not unexpected given the weak evidence of slow outlier spikes in this experiment (Fig. 8 ) and the small ω estimates shown in Table 2 , and it provides a further indication that slow outliers were rare or non-existent in this study. Again, if there is little or no evidence that slow outliers are present in the data, then there is little or no hope of estimating the minimum or median of the slow outlier latencies.

Robustness of the method

Although the new CDF pooling method appears to work quite well with the lexical decision task mega-study datasets examined here, it is reasonable to question how well it would work with other datasets Footnote 6 . For one thing, these datasets had large numbers of trials per participant in each condition, and it seems worthwhile to investigate how well the technique would work with fewer trials. Likewise, these datasets had relatively low proportions of outliers—seemingly only slow ones—and it is possible that the method would break down with larger proportions of outliers, outliers at both extremes, or both. This section reports separate sets of simulations investigating these two issues. Both sets of simulations used large numbers of simulated participants, so the results illustrate the long-term biases that can be expected to emerge when the method is used with fewer trials or more outliers.

Number of trials

To investigate the number of trials per participant and condition needed for the method to work, I used the real RTs for the word, repetition condition of the ELP dataset (Fig. 7 a), selecting this dataset because of its large RT cut-off (4 s) as well as its large numbers of participants and trials. In different simulations, I applied the method to randomly selected subsets of N = 25, 50, 75, 100, 125, or 150 trials from this condition for each participant. That is, the ex-Gaussian was fit separately to the RTs in each random subset of N trials, and the \(\widehat{CDF}\) values of the RTs in that subset were computed relative to its fitted ex-Gaussian. For each N, this process was repeated enough times to get 3000 \(\widehat{CDF}\) values for each of the 791 participants so that there would be a large number of \(\widehat{CDF}\) values for that N, and finally all of the \(\widehat{CDF}\) values were pooled together into a histogram for that N.

The results of these simulations are shown in Fig. 11 . To the extent that the method still works properly with a given sample size of N trials, the resulting histograms should show approximately the same high-end spikes as the original histograms. Not surprisingly, the method does break down with smaller numbers of trials. In particular, smaller N’s result in diminished high-end spikes and also produce low-end spikes that would suggest the presence of fast outliers, even though there was no evidence of fast outliers when all RTs were analyzed. These simulations suggest that at least 75 trials would be needed per participant in each condition for the method to be at all usable, but more trials would obviously be better. Inaccuracies in the estimation of the ex-Gaussian parameters are presumably responsible for the distortion of the pooled \(\widehat{CDF}\) histograms with smaller Ns. Table 4 shows the means and standard deviations of the parameter estimates obtained with each of the different Ns, and parameter estimates appear to be less biased and more stable with larger Ns, as expected.

Histogram of pooled estimated cumulative distribution function ( \(\widehat{{\text{CDF}}}\) ) values of RTs for random subsets of different numbers of trials for the word, repetition condition of the English Lexicon Project dataset

Proportion of outliers

An additional set of simulations looked at how the CDF pooling method fares with increases in the proportion of fast outliers, slow outliers, or both. The potential problem is that larger proportions of outliers could distort the estimates of the valid RT distribution’s ex-Gaussian parameters so seriously that the pooled \(\widehat{CDF}\) values would no longer produce the discernible low- and high-end spikes that tend to indicate the presence of outliers.

To allow precise control over the true proportion of outliers, in these simulations the valid RTs were generated from true underlying ex-Gaussian distributions, whereas the fast and slow outliers were generated in accordance with a mixture model using half-triangular CDF distributions like the one shown in Fig. 9 . To ensure realistic variation across simulated participants, the parameter values of the valid RT ex-Gaussians were sampled from the individual-participant parameter values estimated for the word, repetition condition from the full ELP dataset. The probabilities of fast and slow outliers, p o , were varied across simulations. In some simulations there were only fast outliers, in others there were only slow outliers, and in others there were both fast and slow outliers, each with probability p o , so that the probability of a valid RT was 1 − 2 × p o in these latter simulations. Each slow outlier was an RT whose CDF within the valid ex-Gaussian came from a half-triangular distribution with ω = 0 . 99, and each fast outlier was an RT whose CDF came from a mirror-reflected half-triangular distribution in the range of 0–0.01 (i.e., maximum density at CDF equal to 0, decreasing linearly to 0.01).

Figure 12 shows the results of the simulations in which there were both fast and slow outliers. The frequency distributions of \(\widehat{CDF}\) values are reasonably uniform apart from the expected low-end and high-end spikes as long as the outlier probability is less than approximately 0.06, especially with 150–200 trials. The frequency distributions are clearly distorted, however, when the proportions of outliers are larger. Both the low-end and high-end spikes seem to be shifted slightly inwards from the most extreme boundaries of zero and 1. For example, the largest spike associated with the high-end outliers moves down into the 0.97–0.99 bins of \(\widehat{CDF}\) values rather than being concentrated in the top 0.99–1 bin. The pattern is accompanied by under-representation of \(\widehat{CDF}\) values in the bins from approximately 0.50 to 0.97. Such a distinctive pattern of departures from the expected uniform \(\widehat{CDF}\) distribution may thus be suggestive of a relatively large proportions of outliers in an RT dataset.

Histogram of pooled estimated cumulative distribution function ( \(\widehat{{\text{CDF}}}\) ) values of RTs for random samples of different numbers of ex-Gaussian RTs using parameters estimated from the word, repetition condition of the English Lexicon Project dataset, with different probabilities (p o ) of fast and slow outliers

The pooled \(\widehat{CDF}\) values that emerged from the simulations with just one type of outlier—either fast or slow—were quite similar to the ones shown in Fig. 12 at the end with the outliers, and they were reasonably flat otherwise. Thus, it appears that fast and slow outliers have relatively independent effects on the pooled \(\widehat{CDF}\) values, at least with proportions and latencies of outliers comparable to those used in these simulations.

Figure 13 shows how the presence of outliers affected the average ex-Gaussian parameter estimates used in computing the pooled \(\widehat{CDF}\) values for these simulations. Clearly, the strongest effect of fast outliers was to bias the estimates of µ toward smaller values, whereas the strongest effect of slow outliers was to bias the estimates of τ toward larger values. Each of these biases would make the outliers appear to have less extreme \(\widehat{CDF}\) values, which may account for the inward shifts of the fast and slow outlier spikes away from the boundaries of zero and 1.

Average estimated values of ex-Gaussian distribution parameters obtained from samples of N trials with different probabilities (p o ) of fast and slow outliers. The true average parameter values were µ = 494.9, σ = 59.0, and τ = 280.6

General discussion

Cdf pooling method.

The CDF pooling method presented here provides a bottom-up tool for obtaining an empirical description of outliers in real RT distributions, thereby enabling the study of the characteristics of these outliers. In contrast, most previous attempts to describe outliers have been top-down in the sense of assuming a particular classification criterion a priori (e.g., > 2 . 5 SDs above the mean) and then comparing trials that were versus were not classified as outliers (e.g., Berger & Kiefer, 2023 ; Miller, 2023 ). Although such comparisons can reveal differences between relatively fast and slow RTs, they are not necessarily indicative of the properties of outliers per se, because valid RTs may themselves be slow.

An important strength of the new method is that it allows researchers to summarize overall RT distributions using virtually unlimited numbers of trials, because there is no limit on the number of participants whose \(\widehat{CDF}\) values can be pooled together within a distribution. Thus, this method can provide arbitrarily high resolution, which is essential when trying to get a picture of rare events such as RT outliers. In fact, because \(\widehat{CDF}\) distributions should in principle be uniform for all conditions as well as for all participants, it might also be possible to pool them across conditions and even across experiments in order to get a picture of the outliers within a wider domain. In contrast, the most common previous method of RT distributional analysis—known as “Vincentizing”—is to compute the quantiles of the observed RTs for each participant and then to average these quantiles (Ratcliff, 1979 ; Vincent, 1912 ) or some transformation of them (Cousineau et al., 2016 ) across participants. Such quantile averaging has lower resolution than CDF pooling, because the number of quantiles cannot exceed the number of RTs per participant. With Vincentizing, researchers would need thousands of trials per participant in each condition to study rare outlier RTs, and even with that many trials there would be distortions due to averaging the outlier RTs of some participants with the valid RTs of other participants at a given high quantile (e.g., participants with 1% versus 2% outliers).

In principle, the new CDF pooling method can be used with any distribution—not just the ex-Gaussian. The major limitation of the pooling method is that it requires commitment to a specific RT distribution for computing the \(\widehat{CDF}\) values of the individual RTs of a given participant in a given condition [e.g., ex-Gaussian ( µ = 305, σ = 28, and τ = 174)]. Thus, researchers must select a particular distribution family and have enough trials per participant in each condition to get reasonably accurate estimates of its parameters. Fortunately, the observed RT distributions themselves can be checked when selecting the appropriate distribution family (see Appendix 2 ), which provides empirical guidance. Moreover, the method provides an internal check: the final distribution of pooled \(\widehat{CDF}\) values should be approximately flat, apart from outliers at one or both ends, if an appropriate distribution family has been chosen and its parameter estimates are sufficiently accurate. Likewise, unreasonable values of the estimated mixture model parameters (e.g., ω < 0 . 95) would indicate either that there were no outliers or that the model was not appropriate for a given dataset, as was the case with the FLP dataset examined here.

It is not possible to be completely certain about the accuracy of the new method’s performance in real datasets without having a pre-existing gold standard indicator of which RTs are outliers, but unfortunately no such standard is available. In the absence of such a standard, the results of simulations using the previously suggested outlier models (Fig. 1 ) and using distributions with known percentages of outliers (Fig. 12 ) provide encouragement that the method can reveal fundamental information about outliers when there are enough trials per condition and participant.

Outliers in lexical decision tasks

Because the new method was used here with a restricted set of real datasets (i.e., lexical decision tasks with limited response intervals and normal adult participant populations), the information about outliers obtained from the current analyses is—strictly speaking—limited to this task and population type. Nonetheless, because these tasks and populations are representative of many published RT experiments, it seems appropriate to regard the present conclusions about outliers in these tasks as a plausible tentative description of outliers in a variety of RT studies. Further use of the method within a wide variety of studies will be extremely useful for increasing knowledge about the frequency and characteristics of RT outliers more generally.

With respect to the real lexical decision task datasets examined in the present case study, the present analyses using the CDF pooling method suggest several conclusions about the shapes of RT distributions and the characteristics of the outliers they contain. First, outliers appear to be quite rare in these tasks. Apart from the obviously anomalous RTs less than 150 ms, there was no evidence of any fast outliers. Slow outliers were present in all conditions for three of the four datasets and in two conditions for the fourth, but these were also quite rare—probably less than 0.5%—supporting the conclusions reached through different types of analyses of the same datasets by Miller ( 2023 ).

Second, the slow outliers seemed to be in the range of approximately 4–6 standard deviations of RT above the mean of the valid RT distribution. Although the slow outlier RTs were necessarily at the upper ends of the distributions of valid RTs, there did seem to be some overlap between the valid RTs and outliers, presumably because of the long positive tail of the valid RT distribution. Based on the values of ω from the mixture model summarized in Table 2 , the distribution of outlier RTs often overlapped with the top 1–1.5% of the valid RT distribution, except for the FLP dataset for which this parameter has little meaning due to the absence of outliers in the first place.

Third, the ex-Gaussian distribution provided a good description of individual RT distributions in all of these datasets. This finding is consistent with many previous reports of good fits for the ex-Gaussian (e.g., Balota & Yap, 2011 ; Hohle, 1965 ; Luce, 1986 ), and it extends those reports because the ex-Gaussian was pitted against an unusually large number of competing distributions. The near uniformity of the \(\widehat{CDF}\) values obtained from fitted ex-Gaussians and pooled across huge numbers of trials (e.g., Fig. 5 ) also provides further evidence that this distributional model is appropriate. The presence of high-end spikes in these pooled \(\widehat{CDF}\) distributions also suggests that the ex-Gaussian parameter estimation is robust enough to tolerate occasional outliers, although the simulations with added outliers show that the pooled CDF distributions may take on a different shape when the proportion of outliers exceeds 5–6% (Fig. 12 ).

Implications for outlier exclusion procedures

The overlap between the distributions of valid and outlier RTs suggested by the present analyses makes it very difficult to come up with a method of classifying valid versus outlier trials when RT is the only variable used for classification. Given the overlap of valid RTs and outliers, any given cut-off will either reject some valid RTs as outliers, accept some outliers as valid RTs, or both. On the other hand, the overlap suggests that outliers in RT tasks may not generally be atypical enough to seriously distort the summary results (e.g., sample mean), especially considering that they seem to be rare.

The present results provide some guidance for researchers wanting to use existing outlier exclusion procedures. As discussed by Miller ( 2023 ), almost all existing outlier exclusion procedures involve an experimenter-determined cut-off defining how extreme an RT must be in order to be classified as an outlier (e.g., > 2 . 5 SDs above the sample mean). The fact that there seem to be few slow outliers in the present datasets suggest that relatively extreme cut-offs should be chosen. With the ex-Gaussian ( µ = 300, σ = 30, τ = 100) distribution, for example, the commonly-used 2.5 SD cut-off would classify approximately 3% of RTs as outliers; a rarely used 3.5 SD cut-off would be needed to reduce that percentage to the evidently more appropriate value of 1%. The cost of identifying valid RTs as outliers, of course, is loss of power, not only because of the smaller number of trials included in the analysis but also because slow valid RTs may be particularly informative when the effects of interest are largest for the slowest responses.

Naturally, information about the empirical characteristics of outlier RTs can also inform the search for better outlier exclusion procedures. In particular, when simulations are conducted to compare different procedures, the simulation conditions should reflect empirically realistic conditions. The present results strongly suggest limits on those conditions. For example, in hindsight, the values of 5–10% slow outliers used in most simulations (e.g., Berger & Kiefer, 2021 ; Ratcliff, 1993 ; Ulrich & Miller, 1994 ; Van Selst & Jolicœur, 1994 ) seem unrealistically large, and the value of 1% slow outliers used by Bush et al. ( 1993 ) seems more appropriate. Likewise, the present results also suggest that the simulated slow outlier RTs should not be made too extreme relative to the valid RTs, with an outlier median only approximately 4–6 standard deviations of RT above the mean of the valid RT distribution.

More speculatively, the method of CDF pooling could be extended to form the basis of a new RT outlier exclusion procedure. Researchers could first examine the observed RTs from all participants, select an appropriate distribution family, and then use the mixture model to estimate the minimum \(\widehat{CDF}\) values of outliers, ω . The \(\widehat{CDF}\) of each RT from each participant could then be computed, and any RT with \(\widehat{CDF}\) ( RT ) > ω could be classified as an outlier. While this procedure has some intuitive plausibility based on the current analyses, empirical examination would be required to evaluate its performance with actual datasets (cf. Miller, 2023 ).

Conclusions

The novel CDF pooling method appears to be a promising approach for obtaining the high-resolution information about RT distributions needed to study the distributions’ tails. It performs well with several previously suggested models of what real outliers might look like, producing useful approximations to the known probabilities and latencies of outliers within those models. When applied to real data, the method produces reasonable results that pass a number of internal consistency checks. Thus, the method provides a plausible procedure for learning about outliers in real datasets. The information gained would be useful for assessing the likely effects of outliers on different methods of RT analyses, developing new classification criteria for outlier exclusion, and identifying factors that contribute to the occurrence of outliers in the first place.

Software availability

R routines for estimating parameters of the ex-Gaussian distribution and computing the to-be-pooled CDF values are available at https://github.com/milleratotago/CDFpooling .

Bush et al. ( 1993 ) were only interested in having a distribution with an appropriate skew, so they simulated valid RTs as coming from chi-square distributions with means of 4–12 ms. To produce valid RTs in a realistic range for this illustration, I instead used a gamma distribution for the valid RTs with parameters adjusted to give realistic RTs.

For both the overlap and tails methods, the probabilities of outliers and the parameters of the distributions of valid RTs, fast outliers, and slow outliers varied randomly from participant to participant according to higher-level distributions (e.g., the ex-Gaussian µ parameter varied across participants according to a uniform distribution from 250–500 ms). Figure 1 e and f depict distributions for a single participant with parameter values at the medians within each of their assumed higher-level distributions.

As is standard, “hats” are used in this article to indicate values that are estimates derived from sample data rather than true values.

In fact, most of the excluded RTs were less than 50 ms, and these are well outside the range of potentially valid observations in a choice RT task. Including the approximately 0.03% of trials in the 50–150 ms range had only miniscule effects on the analyses reported here. There were even smaller percentages of trials in the 50–150 ms range for two of the three additional datasets analyzed subsequently, but 0.34% of trials were in this range for the English Language Project dataset. Even for this dataset, however, there was virtually no effect of including these trials in the analyses.

Additional simulations were conducted with versions of the RAT model in which the outlier RTs were convolutions of the valid ex-Gaussian RT with a uniform delay ranging from 500–2000 ms or 1000–2000 ms, thus reducing the overlap of the valid RT and outlier distributions. The CDF pooling method was much more effective in detecting the proportions and minimum \(\widehat{CDF}\)  values of outliers in these simulations than in the original simulations using the 0–2000 ms delays used by Ratcliff ( 1993 ), highlighting the importance of reduced valid/outlier overlap for obtaining accurate results with the CDF pooling method and mixture model.

I thank Denis Cousineau for raising the issues addressed in this section.

For the Weibull distribution, parameter estimates obtained by maximum likelihood can be seriously biased under some conditions (e.g., Smith, 1985 ). With parameter values that are plausible for RT distributions, however, the bias is generally small when estimates are based on more than 80 observations (Cousineau, 2009 ). Separate simulations confirmed that the maximum-likelihood procedure estimates Weibull parameter and sample likelihood values quite accurately for the numbers of trials and ranges of parameters values found in the present datasets.

Average ranks were used in preference to average likelihood scores because the latter can be highly skewed in cases where a distribution does not provide a reasonable fit to an observed set of RTs.

Anders, R., Alario, F.-X., & Van Maanen, L. (2016). The shifted Wald distribution for response time data analysis. Psychological Methods, 21 (3), 309–327. https://doi.org/10.1037/met0000066

Article   PubMed   Google Scholar  

Balota, D. A., Yap, M. J., Hutchison, K. A., Cortese, M. J., Kessler, B., Loftis, B., ..., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods , 39 (3), 445–459. https://doi.org/10.3758/bf03193014

Balota, D. A., & Yap, M. J. (2011). Moving beyond the mean in studies of mental chronometry: The power of response time distributional analyses. Current Directions in Psychological Science, 20 (3), 160–166. https://doi.org/10.1177/0963721411408885

Article   Google Scholar  

Berger, A., & Kiefer, M. (2023). Electrophysiological correlates of response time outliers: Outlier related potentials. Psychophysiology , 60 (e14305). https://doi.org/10.1111/psyp.14305

Berger, A., & Kiefer, M. (2021). Comparison of different response time outlier exclusion methods: A simulation study. Frontiers in Psychology, 12 (675558), 1–13. https://doi.org/10.3389/fpsyg.2021.675558

Boring, E. G. (1929). A history of experimental psychology . Appleton-Century.

Google Scholar  

Brebner, J. M. T., & Welford, A. T. (1980). Introduction: An historical background sketch. In A. T. Welford (Ed.), Reaction times (pp. 1–23). Academic Press.

Burbeck, S. L., & Luce, R. D. (1982). Evidence from auditory simple reaction times for both change and level detectors. Perception & Psychophysics, 32 , 117–133. https://doi.org/10.3758/BF03204271

Bush, L. K., Hess, U., & Wolford, G. (1993). Transformations for within-subject designs: A Monte Carlo investigation. Psychological Bulletin, 113 (3), 566–579. https://doi.org/10.1037/0033-2909.113.3.566

Carpenter, R. H. S., & Williams, M. L. L. (1995). Neuronal computation of log likelihood in control of saccadic eye movements. Nature, 377 , 59–62. https://doi.org/10.1038/377059a0

Castro, S. C., Strayer, D. L., Matzke, D., & Heathcote, A. (2019). Cognitive workload measurement and modeling under divided attention. Journal of Experimental Psychology: Human Perception & Performance, 45 (6), 826–839. https://doi.org/10.1037/xhp0000638

Cousineau, D. (2009). Nearly unbiased estimators for the three-parameter Weibull distribution with greater efficiency than the iterative likelihood method. British Journal of Mathematical & Statistical Psychology, 62 , 167–191. https://doi.org/10.1348/000711007X270843

Cousineau, D., & Chartier, S. (2010). Outliers detection and treatment: A review. International Journal of Psychological Research, 3 (1), 58–67.

Cousineau, D., Brown, S., & Heathcote, A. (2004). Fitting distributions using maximum likelihood: Methods and packages. Behavior Research Methods, Instruments & Computers, 36 , 742–756. https://doi.org/10.3758/BF03206555

Cousineau, D., Thivierge, J.-P., Harding, B., & Lacouture, Y. (2016). Constructing a group distribution from individual distributions. Canadian Journal of Experimental Psychology, 70 (3), 253–277. https://doi.org/10.1037/cep0000069

Cousineau, D., Harding, B., Walker, J. A., Durand, G., Groulx, J. T., Lauzon, S., & Goulet, M.-A. (2023). Analyses of response time data in the same-different task. Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale, 77 (2), 115–129. https://doi.org/10.1037/cep0000301

Dolan, C. V., Van der Maas, H. L. J., & Molenaar, P. C. M. (2002). A framework for ML estimation of parameters of (mixtures of) common reaction time distributions given optional truncation or censoring. Behavior Research Methods, Instruments & Computers, 34 , 304–323. https://doi.org/10.3758/BF03195458

Ferrand, L., New, B., Brysbaert, M., Keuleers, E., Bonin, P., Méot, A., ..., & Pallier, C. (2010). The French Lexicon Project: Lexical decision data for 38,840 French words and 38,840 pseudowords. Behavior Research Methods , 42 (2), 488–496. https://doi.org/10.3758/BRM.42.2.488

Heathcote, A. (2004). Fitting Wald and Ex-Wald distributions to response time data: An example using functions for the S-PLUS package. Behavior Research Methods, Instruments & Computers, 36 , 678–694. https://doi.org/10.3758/BF03206550

Heathcote, A., Popiel, S. J., & Mewhort, D. J. K. (1991). Analysis of response-time distributions: An example using the Stroop task. Psychological Bulletin, 109 , 340–347. https://doi.org/10.1037/0033-2909.109.2.340

Hohle, R. H. (1965). Inferred components of reaction times as functions of foreperiod duration. Journal of Experimental Psychology, 69 , 382–386. https://doi.org/10.1037/h0021740

Holden, J. G., & Rajaraman, S. (2012). The self-organization of a spoken word. Frontiers in Psychology, 3 , 209. https://doi.org/10.3389/fpsyg.2012.00209

Article   PubMed   PubMed Central   Google Scholar  

Hutchison, K. A., Balota, D. A., Neely, J. H., Cortese, M. J., Cohen-Shikora, E. R., Tse, C.-S., ..., & Buchanan, E. (2013). The Semantic Priming Project. Behavior Research Methods , 45 (4), 1099–1114. https://doi.org/10.3758/s13428-012-0304-z

Keuleers, E., Diependaele, K., & Brysbaert, M. (2010). Practice effects in large-scale visual word recognition studies: A lexical decision study on 14,000 Dutch mono- and disyllabic words and nonwords. Frontiers in Psychology , 1 (174). https://doi.org/10.3389/fpsyg.2010.00174

Lacouture, Y., & Cousineau, D. (2008). How to use MATLAB to fit the ex-Gaussian and other probability functions to a distribution of response times. Tutorials in Quantitative Methods for Psychology, 4 (1), 35–45. https://doi.org/10.20982/tqmp.04.1.p035

Lo, S., & Andrews, S. (2015). To transform or not to transform: Using generalized linear mixed models to analyse reaction time data. Frontiers in Psychology , 6 (1171). https://doi.org/10.3389/fpsyg.2015.01171

Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95 , 492–527. https://doi.org/10.1037/0033-295X.95.4.492

Luce, R. D. (1986). Response times: Their role in inferring elementary mental organization . Oxford University Press.

McGill, W. J., & Gibbon, J. (1965). The general-gamma distribution and reaction times. Journal of Mathematical Psychology, 2 , 1–18. https://doi.org/10.1016/0022-2496(65)90014-3

Miller, J. O. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14 (2), 247–279. https://doi.org/10.1016/0010-0285(82)90010-X

Miller, J. O. (1998). Cupid: A program for computations with probability distributions. Behavior Research Methods, Instruments & Computers, 30 (3), 544–545. https://doi.org/10.3758/BF03200688

Miller, J. O. (2023). Outlier exclusion procedures for reaction time analysis: The cures are generally worse than the disease. Journal of Experimental Psychology: General, 152 (11), 3189–3217. https://doi.org/10.1037/xge0001450

Ollman, R. T. (1966). Fast guesses in choice reaction time. Psychonomic Science, 6 , 155–156. https://doi.org/10.3758/bf03328004

Palmer, E. M., Horowitz, T. S., Torralba, A., & Wolfe, J. M. (2011). What are the shapes of response time distributions in visual search? Journal of Experimental Psychology: Human Perception & Performance, 37 (1), 58–71. https://doi.org/10.1037/a0020747

Ratcliff, R. (1979). Group reaction time distributions and an analysis of distribution statistics. Psychological Bulletin, 86 , 446–461. https://doi.org/10.1037/0033-2909.86.3.446

Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114 , 510–532. https://doi.org/10.1037/0033-2909.114.3.510

Ratcliff, R., & Murdock, B. B., Jr. (1976). Retrieval processes in recognition memory. Psychological Review, 83 , 190–214. https://doi.org/10.1037/0033-295X.83.3.190

Rieger, T. C., & Miller, J. O. (2020). Are model parameters linked to processing stages? An empirical investigation for the ex-Gaussian, ex-Wald, and EZ diffusion models. Psychological Research, 84 (6), 1683–1699. https://doi.org/10.1007/s00426-019-01176-4

Ruthruff, E. D. (1996). A test of the deadline model for speed-accuracy tradeoffs. Perception & Psychophysics, 58 (1), 56–64. https://doi.org/10.3758/BF03205475

Schlosberg, H., & Heineman, C. (1950). The relationship between two measures of response strength. Journal of Experimental Psychology, 40 , 235–247. https://doi.org/10.1037/h0062267

Schwarz, W. (2001). The ex-Wald distribution as a descriptive model of response times. Behavior Research Methods, Instruments & Computers, 33 , 457–469. https://doi.org/10.3758/BF03195403

Schwarz, W. (2002). On the convolution of inverse Gaussian and exponential random variables. Communications in Statistics: Theory & Methods, 31 , 2113–2121. https://doi.org/10.1081/STA-120017215

Smith, A. P. (1985). Noise, biased probability and serial reaction. British Journal of Psychology, 76 , 89–95. https://doi.org/10.1111/j.2044-8295.1985.tb01933.x

Snodgrass, J. G., Luce, R. D., & Galanter, E. (1967). Some experiments on simple and choice reaction time. Journal of Experimental Psychology, 75 (1), 1–17. https://doi.org/10.1037/h0021280

Storms, G., & Delbeke, L. (1992). The irrelevance of distributional assumptions on reaction times in multidimensional scaling of same/different judgment tasks. Psychometrika, 57 (4), 599–614. https://doi.org/10.1007/bf02294422

Ulrich, R., & Miller, J. O. (1993). Information processing models generating lognormally distributed reaction times. Journal of Mathematical Psychology, 37 (4), 513–525. https://doi.org/10.1006/jmps.1993.1032

Ulrich, R., & Miller, J. O. (1994). Effects of truncation on reaction time analysis. Journal of Experimental Psychology: General, 123 (1), 34–80. https://doi.org/10.1037/0096-3445.123.1.34

Ulrich, R., Pope, H. G., Cléret, L., Petróczi, A., Nepusz, T., Schaffer, J., ..., & Simon, P. (2018). Doping in two elite athletics competitions assessed by randomized-response surveys. Sports Medicine, 48, 211–219. https://doi.org/10.1007/s40279-017-0765-4

Van Selst, M., & Jolicœur, P. (1994). A solution to the effect of sample size on outlier elimination. Quarterly Journal of Experimental Psychology, Section A: Human Experimental Psychology, 47 (3), 631–650. https://doi.org/10.1080/14640749408401131

Vankov, I. I. (2023). The hazards of dealing with response time outliers. Frontiers in Psychology, 14 (1220281), 1–11. https://doi.org/10.3389/fpsyg.2023.1220281

Vincent, S. B. (1912). The function of the vibrissae in the behaviour of the white rat. Behavioral Monographs, 1 (5), 1–82.

Download references

Acknowledgements

I thank Melvin Yap and Ludovic Ferrand for their assistance in obtaining and coding the raw data from the English Lexicon Project and French Lexicon Project, respectively. I am also grateful to Alexander Berger, Denis Cousineau, Patricia Haden, Wolf Schwarz, Rolf Ulrich, and Anne Voormann for helpful comments on earlier versions of the article.

Open Access funding enabled and organized by CAUL and its Member Institutions

Author information

Authors and affiliations.

Department of Psychology, University of Otago, PO Box 56, Dunedin, 9054, New Zealand

Jeff Miller

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jeff Miller .

Ethics declarations

Conflicting interests.

The author declares that he has no conflicts of interest with respect to the authorship or publication of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Practices Statement

The data sets for the Semantic Priming Project, Dutch Lexicon Project, English Lexicon Project, and French Lexicon Project, were retrieved from the on-line repositories indicated in the original publications of Hutchison et al. ( 2013 ), Keuleers et al. ( 2010 ), Balota et al. ( 2007 ), and Ferrand et al. ( 2010 ), respectively.

Simulations of outlier models

This appendix presents further information about the simulation methods and results obtained with the outlier models shown in Fig. 1 .

Simulation method

For all models, valid RTs were randomly sampled from ex-Gaussian distributions with parameters of µ , σ , and τ . Following Berger and Kiefer ( 2021 ), random participant-to-participant variation was implemented by sampling each participant’s values of these three parameters from uniform distributions ( µ : 250–500 ms; σ : 20–50 ms; τ : 150–200 ms). The proportion of true outliers also varied randomly across participants. In simulations with a low, medium, or high proportion of outliers, each participant’s probability of a slow outlier was randomly selected from a uniform distribution ranging between 0–0.02, 0–0.05, or 0–0.1, respectively.

For the models depicted in Fig. 1 a and b, the distribution of slow outlier latencies was determined as specified in the original theoretical model (e.g., a valid RT plus a uniform increment of 0–2000 ms for the model of Ratcliff, 1993 ). For the outlier model of Ulrich and Miller ( 1994 ), the distribution of slow outliers was an ex-Gaussian whose parameters depended on the parameters of the participant’s valid RT distribution in the same way as in the overlap model of Berger and Kiefer ( 2021 ), as described in the main text. For the model of Van Selst and Jolicœur ( 1994 ), the distribution of slow outliers was an ex-Gaussian whose µ was 9.3 valid distribution σ s larger than the valid distribution’s µ , as in the original simulations carried out with that model by Van Selst and Jolicœur ( 1994 ). For the BK overlap method depicted in Fig. 1 e, each participant’s outliers came from specific ex-Gaussian distributions whose parameters depended on the randomly selected parameters of that participant’s valid RT distribution as specified by Berger and Kiefer ( 2021 ). For the BK tails method shown in Fig. 1 f, again following Berger and Kiefer ( 2021 ), the outliers were defined relative to the observed minimum and maximum RTs in any given sample of valid trials, so an outlier in one sample was always more extreme than any valid RT in that sample, though it could be less extreme than a valid RT in some other sample. Using these procedures, a sample of 200 RTs was generated for each participant.

Analysis and results

For all models, the CDF value of each randomly generated RT—valid or outlier—was determined relative to the true ex-Gaussian distribution of valid RTs for the simulated participant (i.e., with that participant’s true values of µ , σ , and τ ). These CDF values were pooled across simulated participants, and the dots in Fig. 4 show the relative frequencies of these different CDF values.

In addition, each sample of 200 RTs for each simulated participant was used to obtain maximum likelihood estimates of the best-fitting ex-Gaussian parameters (i.e., \(\widehat{\mu }\) , \(\widehat{\sigma }\) , and \(\widehat{\tau }\) ). Then, the estimated \(\widehat{CDF}\) value of each RT in the sample was determined relative to the ex-Gaussian distribution with these estimated parameters. These \(\widehat{CDF}\) values were also pooled across simulated participants, and the histogram bars in Fig. 4 show the relative frequencies of these different \(\widehat{CDF}\) values.

As noted in the main text, the differences between the frequencies of CDF and \(\widehat{CDF}\) values are caused in large part by biases in the parameter estimates \(\widehat{\mu }\) , \(\widehat{\sigma }\) , and \(\widehat{\tau }\) resulting from the presence of the outliers. Table 5 summarizes these biases. Each bias was computed as the mean across simulated participants of the estimated parameter value minus the true parameter value, so positive biases indicate that the parameters estimated from the RT samples were larger than the true parameters of the underlying ex-Gaussian distributions of valid RTs.

The results in Table 5 mainly show that the estimated values \(\widehat{\tau }\) are larger than the true τ values and that this overestimation increases with larger proportions of outliers. This makes sense, since slow outliers make it look as if the RT distribution has a longer tail at the high end, and the τ parameter increases with the length of this tail. There are also smaller biases in the estimates of µ and σ , and these values tend to be underestimated relative to the true values. These biases can be understood as a secondary consequence of the overestimation of τ . When too much of the observed RT mean and variance are allocated to τ , there is correspondingly not enough left over to be allocated to µ and σ .

Selecting the reaction time distribution family

As was illustrated in the section “Pooling CDF values,” the proposed method of constructing a pooled distribution of \(\widehat{CDF}\) values requires the specification of a distribution family that is a good approximation of the true underlying RT distributions. This appendix describes the different RT distribution families that were considered for the present analyses and shows the results supporting the choice of the ex-Gaussian family for the present datasets.

The choice of an RT distribution family is complicated, partly because of the large number of different distribution families that have been suggested as possible candidate models of RTs (e.g., Luce, 1986 ). There have been few previous attempts to compare the fits of alternative distribution families to empirical datasets, and these have compared only a few of the suggested families at a time (e.g., Palmer et al., 2011 ). Furthermore, the proposed distribution families are all somewhat similar, including long tails at the high end of the RT distribution, so it is difficult to tell them apart. Fortunately, for the present purposes, it is probably not crucial that the chosen distribution be exactly correct; it only needs to be a good enough approximation to produce nearly correct \(\widehat{CDF}\) values. Thus, the goal of the present analysis was to select the best available distribution family as a model for RTs, without making any claims that this is necessarily the “true” distribution.

Candidate distribution families

In the RT literature, the ex-Gaussian is by far the most-often considered descriptive distributional family (e.g., Burbeck & Luce, 1982 ; Hohle, 1965 ). An informal survey of the literature shows than many other distribution families have also been considered, however, including the lognormal (e.g., Ratcliff & Murdock, 1976 ; Schlosberg & Heineman, 1950 ; Storms & Delbeke, 1992 ; Ulrich & Miller, 1993 ), gamma (e.g., Lo & Andrews, 2015 ; McGill & Gibbon, 1965 ; Palmer et al., 2011 ), Weibull (e.g., Logan, 1988 ; Palmer et al., 2011 ), Wald (e.g., Cousineau et al., 2004 ; Dolan et al., 2002 ; Heathcote, 2004 ), shifted Wald (e.g., Anders et al., 2016 ; Castro et al., 2019 ), ex-Wald (e.g., Heathcote, 2004 ; Holden & Rajaraman, 2012 ; Palmer et al., 2011 ; Rieger & Miller, 2020 ; Schwarz, 2001 ), recinormal (e.g., Carpenter & Williams, 1995 ; Lo & Andrews, 2015 ), loglogistic (e.g., Cousineau et al., 2016 ), Gumbel (e.g., Cousineau et al., 2016 ), Frechet (e.g., Luce, 1986 ), and double monomial (e.g., Luce, 1986 ; Snodgrass et al., 1967 ). All of these distributions were considered here. To search even more thoroughly for the best-fitting distribution, I also considered a number of other distributions that could be derived from these distributions by adding a constant shift or an exponentially-distributed shift.

Table 6 lists all 28 of the distribution families considered in the present work. Nine families have two parameters, and all are skewed except for the Gaussian, which was only included for completeness. An additional version of each of these nine families was constructed by including a third parameter reflecting a positive shift added to scores in the distribution, except for the Gaussian for which a shift is redundant with a change in µ . Furthermore, for each of these families, a second additional version of the family was formed as the convolution of the family with an exponential distribution—the mean of the exponential thus being a third parameter. Two of these are already included in the list of previously considered distributions, of course—namely, the ex-Gaussian and ex-Wald. Finally, there were two inherently three-parameter families as shown at the bottom of the table. For each of the 28 families in Table 6 , MATLAB routines were developed to estimate distribution parameters using maximum likelihood (cf., Cousineau et al., 2004 ), and these routines are freely available as part of an updated MATLAB Cupid package (Miller, 1998 ) available at https://github.com/milleratotago/Cupid .

Separately for RTs of each of the participants in each dataset and each of the four experimental conditions being distinguished (i.e., word/nonword x response repetition/alternation), parameters were estimated by maximum likelihood for each of the distributions in Table 6 Footnote 7 . Then, for each combination of participant and condition, the different distributions were ranked from best to worst (i.e., 1–28) in accordance with the maximum likelihood value estimated for that distribution. Finally, for each dataset, the average rank was computed for each of the distributions in Table 6 , averaging across the participants and the four experimental conditions Footnote 8 .

Figure 14 is a heat map showing the average ranking of each distribution’s fits to the individual participant and condition RT distributions for each of the datasets, with lower numbers indicating better fits. The ex-Gaussian distribution provides the lowest average rank for three of the four datasets and the second-lowest rank for the fourth (i.e., French Lexicon Project), so it seemed appropriate to choose this as the most plausible descriptive family for the observed RT distributions. Notably, the similar ex-Wald distribution (Schwarz, 2001 , 2002 ) also does reasonably well, whereas several other distributions that have been suggested as RT models fare quite badly in this comparison (e.g., recinormal, loglogistic, shifted Weibull). It is particularly interesting that the ex-Gaussian fits best in these lexical decision task datasets, because part of the RT variance was no doubt due to differences among items (i.e., different words or different nonwords). Thus, the theoretical distributions of RTs to individual words might not be ex-Gaussian, with the ex-Gaussian shape only emerging after pooling RTs across different words.

Heat map showing the average rank of each of the candidate reaction time (RT) distributions shown in Table 6 , with lower average ranks indicating better fits

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Miller, J. Estimating the proportions and latencies of reaction time outliers: A pooling method and case study of lexical decision tasks. Behav Res (2024). https://doi.org/10.3758/s13428-024-02419-y

Download citation

Accepted : 29 March 2024

Published : 29 May 2024

DOI : https://doi.org/10.3758/s13428-024-02419-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Reaction time distributions
• Lexical decision task
• Find a journal
• Publish with us
• Track your research