case study vs mixed methods research

Revisiting the difference between mixed methods and multimethods: Is it all in the name?

• Published: 09 February 2018
• Volume 52 , pages 2757–2770, ( 2018 )

Cite this article

• M. Teresa Anguera   ORCID: orcid.org/0000-0001-7147-2927 1 ,
• Angel Blanco-Villaseñor 1 ,
• José Luis Losada 1 ,
• Pedro Sánchez-Algarra 1 &
• Anthony J. Onwuegbuzie 2 , 3  

11k Accesses

134 Citations

6 Altmetric

Explore all metrics

The literature on mixed methods and multimethods has burgeoned over the last 20 years, and researchers from a growing number and diversity of fields have progressively embraced these approaches. However, rapid growth in any movement inevitably gives rise to gaps or shortcomings, such as “identity crises” or divergent conceptual views. Although some authors draw a clear and sometimes opinionated distinction between mixed methods and multimethods , for others, they are synonymous. The concepts underlying both terms therefore have become blurred and generated much confusion. The aim of this article is to explore the origins of the confusion, describe our view of mixed methods and multimethod studies, and by doing so, help to clearly delineate the two concepts. The authors have presented their opinion of how these terms and concepts should be distinguished and call for a constructive debate of the issues involved in the mixed methods and multimethod literature. This is a way truly to propel the field forward.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

A worked example of Braun and Clarke’s approach to reflexive thematic analysis

David Byrne

What is Qualitative in Qualitative Research

Patrik Aspers & Ugo Corte

Qualitative Research: Ethical Considerations

Anguera, M.T.: Observational typology. Qual. Quant. 13 (6), 449–484 (1979)

Google Scholar  

Anguera, M.T.: Observational methods (general). In: Fernández-Ballesteros, R. (ed.) Encyclopedia of Psychological Assessment, vol. 2, pp. 632–637. Sage, London (2003)

Anguera, M.T., Hernández-Mendo, A.: Avances en estudios observacionales de Ciencias del Deporte desde los mixed methods (advances in mixed methods observational studies in sports sciences). Cuad. Psicol. Dep. 16 (1), 17–30 (2016)

Anguera, M.T., Izquierdo, C.: Methodological approaches in human communication: from complexity of perceived situation to data analysis. In: Riva, G., Anguera, M.T., Wiederhold, B.K., Mantovani, F. (eds.) From Communication to Presence. Cognition, Emotions and Culture Towards the Ultimate Communicative Experience, pp. 203–222. IOS Press, Amsterdam (2006)

Anguera, M.T., Blanco-Villaseñor, A., Losada, J.L.: Diseños observacionales, cuestiones clave en el proceso de la metodología observacional [observational designs, key issues in the process of observational methodology]. Metod. Cienc. Comport. 3 (2), 135–160 (2001)

Anguera, M.T., Camerino, O., Castañer, M., Sánchez-Algarra, P.: Mixed methods en actividad física y deporte (mixed methods in physical activity and sport). Rev. Psicol. Dep. 23 (1), 123–130 (2014)

Anguera, M.T., Camerino, O., Castañer, M., Sánchez-Algarra, P., Onwuegbuzie, A.J.: The specificity of observational studies in physical activity and sports sciences: moving forward in mixed methods research and proposals for achieving quantitative and qualitative symmetry. Front. Psychol. 8 , 2196 (2017). https://doi.org/10.3389/fpsyg.2017.02196

Article   Google Scholar  

Anguera, M.T., Portell, M., Chacón-Moscoso, S., Sanduvete-Chaves, S.: Indirect observation in everyday contexts: concepts and methodological guidelines within a mixed methods framework. Front. Psychol. 9 , 13 (2018). https://doi.org/10.3389/fpsyg.2018.00013

Archibald, M.M.: Investigator triangulation: a collaborative strategy with potential for mixed methods research? J. Mix. Methods Res. 10 (3), 228–250 (2015). https://doi.org/10.1177/1558689815570092

Archibald, M.M., Radil, A.I., Zhang, X., Hanson, W.E.: Current mixed methods practices in qualitative research: a content analysis of leading journals. Int. J. Qual. Methods 14 (2), 5–33 (2015)

Bakeman, R., Gottman, J.M.: Applying observational methods: a systematic view. In: Osofsky, J.D. (ed.) Handbook of Infant Development, 2nd edn, pp. 818–853. Wiley, New York (1987)

Bakeman, R., Gottman, J.M.: Observing Interaction. Introduction to Sequential Analysis, 2nd edn. Cambridge University Press, Cambridge (1997)

Book   Google Scholar  

Bazeley, P.: Editorial: integrating data analyses in mixed method research. J. Mix. Methods Res. 3 (3), 203–207 (2009). https://doi.org/10.1177/1558689809334443

Bazeley, P.: Integrative analysis strategies for mixed data sources. Am. Behav. Sci. 56 (6), 814–828 (2012). https://doi.org/10.1177/0002764211426330

Bazeley, P.: Writing up multimethod and mixed methods research for diverse audiences. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 296–313. Oxford University Press, Oxford (2015)

Bazeley, P., Kemp, L.: Mosaics, triangles, and DNA: metaphors for integrated analysis in mixed methods research. J. Mix. Methods Res. 6 (1), 55–72 (2012). https://doi.org/10.1177/1558689811419514

Bergman, M.M.: Multimethod research and mixed methods research: old wine in new bottles? J. Mix. Methods Res. 1 (1), 101–104 (2007). https://doi.org/10.1177/2345678906291429

Blanco-Villaseñor, A., Losada, J.L., Anguera, M.T.: Analytic techniques in observational designs in environment–behavior relation. Medio Ambient Comport. Hum. 4 (2), 111–126 (2003)

Borkan, J.M.: Mixed methods studies: a foundation for primary care research. Ann. Fam. Med. 2 , 4–6 (2004)

Brewer, J., Hunter, A.: Multimethod Research: A Synthesis of Styles. Sage, Newbury Park (1989)

Brewer, J., Hunter, A.: Foundations of Multimethod Research: Synthesizing Styles, 2nd edn. Sage, Thousand Oaks (2006)

Burns, K.K., Bellows, M., Eigenseher, C., Jackson, K., Gallivan, J., Rees, J.: Exploring patient engagement practices and resources within a health care system: applying a multi-phased mixed methods knowledge mobilization approach. Int. J. Mult. Res. Approach. 8 (2), 233–247 (2014). https://doi.org/10.1080/18340806.2014.11082063

Campbell, D.T., Fiske, D.W.: Convergent and discriminant validation by the multitrait–multimethod matrix. Psychol. Bull. 56 , 81–105 (1959)

Christ, T.W.: Book review: the oxford handbook of multimethod and mixed methods research inquiry. J. Mix. Methods Res. 10 (2), 207–209 (2016). https://doi.org/10.1177/1558689815622704

Cook, T.D.: Postpositivism critical multiplism. In: Shortlan, R.L., Mark, M.M. (eds.) Social Science and Social Policy, pp. 21–62. Sage, Beverly Hills (1985)

Cook, T.D., Campbell, D.T.: Quasi Experimentation: Design and Analysis Issues for Field Settings. Rand Mc Nally, Chicago (1979)

Creswell, J.W.: A Concise Introduction to Mixed Methods Research. Sage, Thousand Oaks (2015)

Creswell, J.W., Plano Clark, V.L.: Designing and Conducting Mixed Methods Research. Sage, Thousand Oaks (2007)

Creswell, J.W., Tashakkori, A.: Developing publishable mixed methods manuscripts. J. Mix. Meth. Res. 1 (2), 107–111 (2007). https://doi.org/10.1177/1558689806298644

Creswell, J.W., Tashakkori, A.: How do research manuscripts contribute to the literature on mixed methods? J. Mix. Meth. Res. 2 (2), 115–120 (2008). https://doi.org/10.1177/1558689808315361

Creswell, J.W., Plano Clark, V.L., Gutmann, M.L., Hanson, W.E.: Advanced mixed methods research designs. In: Tashakkori, A., Teddie, C. (eds.) Handbook of Mixed Methods in Social & Behavioral Research, pp. 209–240. Sage, Thousand Oaks (2003)

Denzin, N.K.: The Research Act: A Theoretical Introduction to Sociological Method. McGraw-Hill, New York (1978)

Fassnacht, G.: Theory and Practice of Observing Behaviour. Academic Press, New York (1982)

Fetters, M.D., Freshwater, D.: Publishing a methodological mixed methods research article. J. Mix. Methods Res. 9 (3), 203–213 (2015). https://doi.org/10.1177/1558689815594687

Fetters, M.D., Curry, L.A., Creswell, J.W.: Achieving integration in mixed methods designs: principles and practices. Health Serv. Res. 48 (6 Pt. 2), 2134–2156 (2013). https://doi.org/10.1111/1475-6773.12117

Fowler, F.J.: Survey Research Methods. Sage, Newbury Park (1993)

Galt, K.A.: SPSS text analysis for surveys 2.1 and qualitative and mixed methods analysis. J. Mix. Methods Res. 2 (3), 284–286 (2008). https://doi.org/10.1177/1558689808317830

Gibbs, G.R.: Atlas.ti software to assist with the qualitative analysis of data. J. Mix. Methods Res. 1 (1), 103–104 (2007). https://doi.org/10.1177/2345678906291490

Greene, J.C.: Is mixed methods social inquiry a distinctive methodology? J. Mix. Methods Res. 2 (1), 7–22 (2008). https://doi.org/10.1177/1558689807309969

Greene, J.C.: Preserving distinctions within the multimethod and mixed methods research merger. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 606–615. Oxford University Press, Oxford (2015)

Guetterman, T.C.: What distinguishes a novice from an expert mixed methods researcher? Qual. Quant. 51 , 377–398 (2017)

Hesse-Biber, S.N.: Mixed Methods Research: Merging Theory with Practice. Guilford Press, New York (2010)

Hesse-Biber, S.N.: Introduction: navigating a turbulent research landscape: working the boundaries, tensions, diversity, and contradictions of multimethod and mixed methods inquiry. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. xxxiii–liii. Oxford University Press, Oxford (2015)

Chapter   Google Scholar  

Hesse-Biber, S.N., Rodriguez, D., Frost, N.A.: Qualitatively driven approach to multimethod and mixed methods research. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 3–20. Oxford University Press, Oxford (2015)

Hillyard, S.: New Frontiers in Ethnography. Bingley, Emerald (2010)

Holstein, J.A., Gubrium, J.F.: Phenomenology, etnomethodology, and interpretative practice. In: Denzin, N.K., Lincoln, Y.S. (eds.) Handbook of Qualitative Research, pp. 262–272. Sage, Thousand Oaks (1994)

Hunter, A., Brewer, J.: Designing multimethod research. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 185–205. Oxford University Press, Oxford (2015a)

Hunter, A., Brewer, J.: Conundrums of multimethod resarch. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 616–623. Oxford University Press, Oxford (2015b)

Johnson, R.B.: Dialectical pluralism and mixed research. Am. Behav. Sci. 56 (6), 751–754 (2012). https://doi.org/10.1177/0002764212442494

Johnson, R.B., Onwuegbuzie, A.J., Turner, L.A.: Toward a definition of mixed methods research. J. Mix. Methods Res. 1 (2), 112–133 (2007). https://doi.org/10.1177/1558689806298224

Kerlinger, F.N.: Foundations of Behavioral Research: Educational, Psychological and Sociological Inquiry. Holt Rinehart & Winston, New York (1973)

Lazarsfeld, P.F.: Foreward to Ernest Greenwood, Experimental Sociology. King’s Crown Press, New York (1945)

Lee, R.M.: Unobtrusive Methods in Social Research. Open University Press, Buckingham (2000)

Leeman, J., Voils, C.I., Sandelowski, M.: Conducting mixed methods literature reviews: synthesizing the evidence needed to develop and implement complex social and health interventions. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 167–184. Oxford University Press, Oxford (2015)

Levitt, H.M., Bamberg, M., Creswell, J.W., Frost, D.M., Josselson, R., Suárez-Orozco, C.: Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: the APA Publications and Communications Board Task Force Report. Am. Psychol. 73 (1), 26–46 (2018)

Mark, M.M.: Mixed and multimethods in predominantly quantitative studies, especially experiments and quasi-experiments. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 21–41. Oxford University Press, Oxford (2015)

Maxwell, J., Chmiel, M., Rogers, S.E.: Designing integration in multimethod and mixed methods research. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 223–239. Oxford University Press, Oxford (2015)

Mertens, D.: Publishing mixed methods research. J. Mix. Methods Res. 5 (1), 3–6 (2011). https://doi.org/10.1177/1558689810390217

Molina-Azorín, J.F., Fetters, M.D.: Mixed methods research prevalence studies: field-specific studies on the state of the art of mixed methods research. J. Mix. Methods Res. 10 (2), 123–128 (2016). https://doi.org/10.1177/1558689816636707

Morales-Sánchez, V., Pérez-López, V., Anguera, M.T.: Tratamiento metodológico de la observación indirecta en la gestión de organizaciones deportivas (indirect observational methodology in managing sports services). Rev. Psicol. Dep. 23 (1), 201–207 (2014)

Morse, J.M.: Principles of mixed methods and multimethod research design. In: Tashakkori, A., Teddie, C. (eds.) Handbook of Mixed Methods in Social & Behavioral Research, pp. 189–208. Sage, Thousand Oaks (2003)

Morse, J.M.: Procedures and practice of mixed method design. Maintaining control, rigor, and complexity. In: Taskakkori, A., Teddie, C. (eds.) Sage Handbook of Mixed Methods in Social & Behavioral Research, 2nd edn, pp. 339–352. Sage, Thousand Oaks (2010)

Morse, J.M.: Issues in qualitatively-driven mixed-method designs: walking through a mixed-method project. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 206–222. Oxford University Press, Oxford (2015)

Morse, J., Cheek, J.: Introducing qualitatively-driven mixed-method designs. Qual. Health Res. 25 (6), 731–733 (2015). https://doi.org/10.1177/1049732315583299

Morse, J.M., Niehaus, L.: Mixed Method Design: Principles and Procedures. Left Coast Press, Walnut Creek (2009)

Newman, I., Onwuegbuzie, A.J., Hitchcock, J.H.: Using the general lineal model to facilitate the full integration of qualitative and quantitative analysis: the potential to improve prediction ad theory building and testing. Gen. Linear Model J. 4 (1), 12–28 (2015)

Nigras, K.: Spreadsheet software can facilitate mixed methods research—using old tools in a new context! J. Mix. Methods Res. 1 (3), 297–299 (2007). https://doi.org/10.1177/1558689807301250

Onwuegbuzie, A.J., Hitchcock, J.H.: Advanced mixed analysis approaches. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 275–295. Oxford University Press, Oxford (2015)

Onwuegbuzie, A.J., Slate, J.R., Leech, N.L., Collins, K.M.T.: Conducting mixed analyses: a general typology. Int. J. Mult. Res. Approach. 1 , 4–17 (2007). https://doi.org/10.5172/mra.455.1.1.4

Onwuegbuzie, A.J., Johnson, R.B., Collins, K.M.T.: Assessing legitimation in mixed research: a new framework. Qual. Quant. 45 , 1253–1271 (2011)

Onwuegbuzie, A.J., Frels, R.K., Collins, K.M.T., Leech, N.L.: Conclusion: a four phase model for teaching and learning mixed research. Int. J. Mult. Res. Approach. 7 (1), 133–156 (2013)

Plano Clark, V.L., Badiee, M.: Research questions in mixed methods research. In: Tashakkori, A., Teddie, C. (eds.) Sage Handbook of Mixed Methods in Social & Behavioral Research, 2nd edn, pp. 275–304. Sage, Thousand Oaks (2010)

Plano Clark, V.L., Ivankova, N.V.: Mixed Methods Research: A Practical Guide to the Field. Sage, Thousand Oaks (2016)

Portell, M., Anguera, M.T., Chacón-Moscoso, S., Sanduvete, S.: Guidelines for reporting evaluations based on observational methodology. Psicothema 27 , 283–289 (2015a). https://doi.org/10.7334/psicothema2014.276

Portell, M., Anguera, M.T., Hernández-Mendo, A., Jonsson, G.K.: Quantifying biopsychosocial aspects in everyday contexts: an integrative methodological approach from the behavioral sciences. Psychol. Res. Behav. Manag. 8 , 153–160 (2015b). https://doi.org/10.2147/PRBM.S82417

Salmons, J.E.: Conducting multimethod and mixed methods research online. In: Hesse-Biber, S.N., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 522–547. Oxford University Press, Oxford (2015)

Sánchez-Algarra, P., Anguera, M.T.: Qualitative/quantitative integration in the inductive observational study of interactive behaviour: impact of recording and coding among predominating perspectives. Qual. Quant. 47 (2), 1237–1257 (2013). https://doi.org/10.1007/s11135-012-9764-6

Sandelowski, M.: Tables or tableaux? Writing and reading mixed methods studies. In: Tashakkori, A., Teddie, C. (eds.) Handbook of Mixed Methods in Social & Behavioral Research, pp. 321–350. Sage, Thousand Oaks (2003)

Sandelowski, M.: Metasynthesis of qualitative research. In: Cooper, H. (ed.) APA Handbook of Research Methods in Psychology: Vol. 2. Research Designs: Quantitative, Qualitative, Neuropsychological, and Biological, pp. 19–36. American Psychological Association, Washington, DC (2012)

Sandelowski, M.: Unmixing mixed-methods research. Res. Nurs. Health 37 , 3–8 (2014). https://doi.org/10.1002/nur.21570

Sandelowski, M., Barroso, J.: Handbook for Synthesizing Qualitative Research. Springer, New York (2007)

Sandelowski, M., Voils, C.I., Knafl, G.: On quantitizing. J. Mix. Methods Res. 3 , 208–222 (2009). https://doi.org/10.1177/1558689809334210

Sandelowski, M., Voils, C., Leeman, J., Crandell, J.: Mapping the mixed methods research synthesis terrain. J. Mix. Methods Res. 6 , 317–331 (2012). https://doi.org/10.1177/1558689811427913

Sandelowski, M., Leeman, J., Knafl, K., Crandell, J.: Text-in-context: a method for extracting findings in mixed-methods mixed research synthesis studies. J. Adv. Nurs. 69 , 1428–1437 (2013a). https://doi.org/10.1111/jan.12000

Sandelowski, M., Voils, C.I., Crandell, J., Leeman, J.: Synthesizing qualitative and quantitative research findings. In: Beck, C.T. (ed.) Routledge International Handbook of Qualitative Nursing Research, pp. 347–356. Routledge, New York (2013b)

Sorensen, A.: Use of QSR NVivo 7 qualitative analysis software for mixed methods research. J. Mix. Methods Res. 2 (1), 106–110 (2008). https://doi.org/10.1177/1558689807309082

Stange, K.C., Crabtree, B.F., Miller, W.L.: Publishing multimethod research. Ann. Fam. Med. 4 , 292–294 (2006). https://doi.org/10.1370/afm.615

Strauss, A., Corbin, J.: Grounded theory methodology. In: Denzin, N.K., Lincoln, Y.S. (eds.) Handbook of Qualitative Research, pp. 273–285. Sage, Thousand Oaks (1994)

Tashakkori, A., Teddie, C.: Mixed Methodology: Combining Qualitative and Quantitative Approaches. Sage, Thousand Oaks (1998)

Tashakkori, A., Teddie, C.: Handbook of Mixed Methods in the Behavioral and Social Sciences. Sage, Thousand Oaks (2003)

Tashakkori, A., Teddie, C.: Putting the human back in “Human Research Methodology”: the researcher in mixed methods research. J. Mix. Methods Res. 4 (4), 271–277 (2010a). https://doi.org/10.1177/1558689810382532

Tashakkori, A., Teddie, C. (eds.): The Sage Handbook of Mixed Methods in Social & Behavioral Research, 2nd edn. Sage, Thousand Oaks (2010b)

Teddie, C., Tashakkori, A.: A general typology of research designs featuring mixed methods. Res. Sch. 13 (1), 12–28 (2006)

Teddie, C., Tashakkori, A.: Foundations of Mixed Methods: Integrating Quantitative and Qualitative Approaches in the Social & Behavioral Sciences. Sage, Thousand Oaks (2009)

Teddie, C., Tashakkori, A.: Overview of contemporary issues in mixed methods research. In: Tashakkori, A., Teddie, C. (eds.) The Sage Handbook of Mixed Methods in Social & Behavioral Research, 2nd edn, pp. 1–41. Sage, Thousand Oaks (2010)

Webb, E.T., Campbell, D.T., Schwartz, R.D., Sechrest, L., Grove, J.B.: Nonreactive Measures in the Social Sciences. Houghton Mifflin, Boston (1966)

Download references

Author information

Authors and affiliations.

University of Barcelona, Barcelona, Spain

M. Teresa Anguera, Angel Blanco-Villaseñor, José Luis Losada & Pedro Sánchez-Algarra

Sam Houston State University, Huntsville, TX, USA

Anthony J. Onwuegbuzie

University of Johannesburg, Johannesburg, South Africa

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to M. Teresa Anguera .

Rights and permissions

Reprints and permissions

About this article

Anguera, M.T., Blanco-Villaseñor, A., Losada, J.L. et al. Revisiting the difference between mixed methods and multimethods: Is it all in the name?. Qual Quant 52 , 2757–2770 (2018). https://doi.org/10.1007/s11135-018-0700-2

Download citation

Published : 09 February 2018

Issue Date : November 2018

DOI : https://doi.org/10.1007/s11135-018-0700-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mixed methods
• Multimethods
• Methodology
• Integration
• Find a journal
• Publish with us
• Track your research

Join thousands of product people at Insight Out Conf on April 11. Register free.

Insights hub solutions

Analyze data

Uncover deep customer insights with fast, powerful features, store insights, curate and manage insights in one searchable platform, scale research, unlock the potential of customer insights at enterprise scale.

Featured reads

Inspiration

Three things to look forward to at Insight Out

Tips and tricks

Make magic with your customer data in Dovetail

Four ways Dovetail helps Product Managers master continuous product discovery

Events and videos

© Dovetail Research Pty. Ltd.

• What is mixed methods research?

Last updated

20 February 2023

Reviewed by

Miroslav Damyanov

By blending both quantitative and qualitative data, mixed methods research allows for a more thorough exploration of a research question. It can answer complex research queries that cannot be solved with either qualitative or quantitative research .

Analyze your mixed methods research

Dovetail streamlines analysis to help you uncover and share actionable insights

Mixed methods research combines the elements of two types of research: quantitative and qualitative.

Quantitative data is collected through the use of surveys and experiments, for example, containing numerical measures such as ages, scores, and percentages. 

Qualitative data involves non-numerical measures like beliefs, motivations, attitudes, and experiences, often derived through interviews and focus group research to gain a deeper understanding of a research question or phenomenon.

Mixed methods research is often used in the behavioral, health, and social sciences, as it allows for the collection of numerical and non-numerical data.

• When to use mixed methods research

Mixed methods research is a great choice when quantitative or qualitative data alone will not sufficiently answer a research question. By collecting and analyzing both quantitative and qualitative data in the same study, you can draw more meaningful conclusions. 

There are several reasons why mixed methods research can be beneficial, including generalizability, contextualization, and credibility. 

For example, let's say you are conducting a survey about consumer preferences for a certain product. You could collect only quantitative data, such as how many people prefer each product and their demographics. Or you could supplement your quantitative data with qualitative data, such as interviews and focus groups , to get a better sense of why people prefer one product over another.

It is important to note that mixed methods research does not only mean collecting both types of data. Rather, it also requires carefully considering the relationship between the two and method flexibility.

You may find differing or even conflicting results by combining quantitative and qualitative data . It is up to the researcher to then carefully analyze the results and consider them in the context of the research question to draw meaningful conclusions.

When designing a mixed methods study, it is important to consider your research approach, research questions, and available data. Think about how you can use different techniques to integrate the data to provide an answer to your research question.

• Mixed methods research design

A mixed methods research design  is   an approach to collecting and analyzing both qualitative and quantitative data in a single study.

Mixed methods designs allow for method flexibility and can provide differing and even conflicting results. Examples of mixed methods research designs include convergent parallel, explanatory sequential, and exploratory sequential.

By integrating data from both quantitative and qualitative sources, researchers can gain valuable insights into their research topic . For example, a study looking into the impact of technology on learning could use surveys to measure quantitative data on students' use of technology in the classroom. At the same time, interviews or focus groups can provide qualitative data on students' experiences and opinions.

• Types of mixed method research designs

Researchers often struggle to put mixed methods research into practice, as it is challenging and can lead to research bias. Although mixed methods research can reveal differences or conflicting results between studies, it can also offer method flexibility.

Designing a mixed methods study can be broken down into four types: convergent parallel, embedded, explanatory sequential, and exploratory sequential.

Convergent parallel

The convergent parallel design is when data collection and analysis of both quantitative and qualitative data occur simultaneously and are analyzed separately. This design aims to create mutually exclusive sets of data that inform each other. 

For example, you might interview people who live in a certain neighborhood while also conducting a survey of the same people to determine their satisfaction with the area.

Embedded design

The embedded design is when the quantitative and qualitative data are collected simultaneously, but the qualitative data is embedded within the quantitative data. This design is best used when you want to focus on the quantitative data but still need to understand how the qualitative data further explains it.

For instance, you may survey students about their opinions of an online learning platform and conduct individual interviews to gain further insight into their responses.

Explanatory sequential design

In an explanatory sequential design, quantitative data is collected first, followed by qualitative data. This design is used when you want to further explain a set of quantitative data with additional qualitative information.

An example of this would be if you surveyed employees at a company about their satisfaction with their job and then conducted interviews to gain more information about why they responded the way they did.

Exploratory sequential design

The exploratory sequential design collects qualitative data first, followed by quantitative data. This type of mixed methods research is used when the goal is to explore a topic before collecting any quantitative data.

An example of this could be studying how parents interact with their children by conducting interviews and then using a survey to further explore and measure these interactions.

Integrating data in mixed methods studies can be challenging, but it can be done successfully with careful planning.

No matter which type of design you choose, understanding and applying these principles can help you draw meaningful conclusions from your research.

• Strengths of mixed methods research

Mixed methods research designs combine the strengths of qualitative and quantitative data, deepening and enriching qualitative results with quantitative data and validating quantitative findings with qualitative data. This method offers more flexibility in designing research, combining theory generation and hypothesis testing, and being less tied to disciplines and established research paradigms.

Take the example of a study examining the impact of exercise on mental health. Mixed methods research would allow for a comprehensive look at the issue from different angles. 

Researchers could begin by collecting quantitative data through surveys to get an overall view of the participants' levels of physical activity and mental health. Qualitative interviews would follow this to explore the underlying dynamics of participants' experiences of exercise, physical activity, and mental health in greater detail.

Through a mixed methods approach, researchers could more easily compare and contrast their results to better understand the phenomenon as a whole.  

Additionally, mixed methods research is useful when there are conflicting or differing results in different studies. By combining both quantitative and qualitative data, mixed methods research can offer insights into why those differences exist.

For example, if a quantitative survey yields one result while a qualitative interview yields another, mixed methods research can help identify what factors influence these differences by integrating data from both sources.

Overall, mixed methods research designs offer a range of advantages for studying complex phenomena. They can provide insight into different elements of a phenomenon in ways that are not possible with either qualitative or quantitative data alone. Additionally, they allow researchers to integrate data from multiple sources to gain a deeper understanding of the phenomenon in question.  

• Challenges of mixed methods research

Mixed methods research is labor-intensive and often requires interdisciplinary teams of researchers to collaborate. It also has the potential to cost more than conducting a stand alone qualitative or quantitative study . 

Interpreting the results of mixed methods research can be tricky, as it can involve conflicting or differing results. Researchers must find ways to systematically compare the results from different sources and methods to avoid bias.

For example, imagine a situation where a team of researchers has employed an explanatory sequential design for their mixed methods study. After collecting data from both the quantitative and qualitative stages, the team finds that the two sets of data provide differing results. This could be challenging for the team, as they must now decide how to effectively integrate the two types of data in order to reach meaningful conclusions. The team would need to identify method flexibility and be strategic when integrating data in order to draw meaningful conclusions from the conflicting results.

• Advanced frameworks in mixed methods research

Mixed methods research offers powerful tools for investigating complex processes and systems, such as in health and healthcare.

Besides the three basic mixed method designs—exploratory sequential, explanatory sequential, and convergent parallel—you can use one of the four advanced frameworks to extend mixed methods research designs. These include multistage, intervention, case study , and participatory. 

This framework mixes qualitative and quantitative data collection methods in stages to gather a more nuanced view of the research question. An example of this is a study that first has an online survey to collect initial data and is followed by in-depth interviews to gain further insights.

Intervention

This design involves collecting quantitative data and then taking action, usually in the form of an intervention or intervention program. An example of this could be a research team who collects data from a group of participants, evaluates it, and then implements an intervention program based on their findings .

This utilizes both qualitative and quantitative research methods to analyze a single case. The researcher will examine the specific case in detail to understand the factors influencing it. An example of this could be a study of a specific business organization to understand the organizational dynamics and culture within the organization.

Participatory

This type of research focuses on the involvement of participants in the research process. It involves the active participation of participants in formulating and developing research questions, data collection, and analysis.

An example of this could be a study that involves forming focus groups with participants who actively develop the research questions and then provide feedback during the data collection and analysis stages.

The flexibility of mixed methods research designs means that researchers can choose any combination of the four frameworks outlined above and other methodologies , such as convergent parallel, explanatory sequential, and exploratory sequential, to suit their particular needs.

Through this method's flexibility, researchers can gain multiple perspectives and uncover differing or even conflicting results when integrating data.

When it comes to integration at the methods level, there are four approaches.

Connecting involves collecting both qualitative and quantitative data during different phases of the research.

Building involves the collection of both quantitative and qualitative data within a single phase.

Merging involves the concurrent collection of both qualitative and quantitative data.

Embedding involves including qualitative data within a quantitative study or vice versa.

• Techniques for integrating data in mixed method studies

Integrating data is an important step in mixed methods research designs. It allows researchers to gain further understanding from their research and gives credibility to the integration process. There are three main techniques for integrating data in mixed methods studies: triangulation protocol, following a thread, and the mixed methods matrix.

Triangulation protocol

This integration method combines different methods with differing or conflicting results to generate one unified answer.

For example, if a researcher wanted to know what type of music teenagers enjoy listening to, they might employ a survey of 1,000 teenagers as well as five focus group interviews to investigate this. The results might differ; the survey may find that rap is the most popular genre, whereas the focus groups may suggest rock music is more widely listened to. 

The researcher can then use the triangulation protocol to come up with a unified answer—such as that both rap and rock music are popular genres for teenage listeners. 

Following a thread

This is another method of integration where the researcher follows the same theme or idea from one method of data collection to the next. 

A research design that follows a thread starts by collecting quantitative data on a specific issue, followed by collecting qualitative data to explain the results. This allows whoever is conducting the research to detect any conflicting information and further look into the conflicting information to understand what is really going on.

For example, a researcher who used this research method might collect quantitative data about how satisfied employees are with their jobs at a certain company, followed by qualitative interviews to investigate why job satisfaction levels are low. They could then use the results to explore any conflicting or differing results, allowing them to gain a deeper understanding of job satisfaction at the company. 

By following a thread, the researcher can explore various research topics related to the original issue and gain a more comprehensive view of the issue.

Mixed methods matrix

This technique is a visual representation of the different types of mixed methods research designs and the order in which they should be implemented. It enables researchers to quickly assess their research design and adjust it as needed. 

The matrix consists of four boxes with four different types of mixed methods research designs: convergent parallel, explanatory sequential, exploratory sequential, and method flexibility. 

For example, imagine a researcher who wanted to understand why people don't exercise regularly. To answer this question, they could use a convergent parallel design, collecting both quantitative (e.g., survey responses) and qualitative (e.g., interviews) data simultaneously.

If the researcher found conflicting results, they could switch to an explanatory sequential design and collect quantitative data first, then follow up with qualitative data if needed. This way, the researcher can make adjustments based on their findings and integrate their data more effectively.

Mixed methods research is a powerful tool for understanding complex research topics. Using qualitative and quantitative data in one study allows researchers to understand their subject more deeply. 

Mixed methods research designs such as convergent parallel, explanatory sequential, and exploratory sequential provide method flexibility, enabling researchers to collect both types of data while avoiding the limitations of either approach alone.

However, it's important to remember that mixed methods research can produce differing or even conflicting results, so it's important to be aware of the potential pitfalls and take steps to ensure that data is being correctly integrated. If used effectively, mixed methods research can offer valuable insight into topics that would otherwise remain largely unexplored.

What is an example of mixed methods research?

An example of mixed methods research is a study that combines quantitative and qualitative data. This type of research uses surveys, interviews, and observations to collect data from multiple sources.

Which sampling method is best for mixed methods?

It depends on the research objectives, but a few methods are often used in mixed methods research designs. These include snowball sampling, convenience sampling, and purposive sampling. Each method has its own advantages and disadvantages.

What is the difference between mixed methods and multiple methods?

Mixed methods research combines quantitative and qualitative data in a single study. Multiple methods involve collecting data from different sources, such as surveys and interviews, but not necessarily combining them into one analysis. Mixed methods offer greater flexibility but can lead to differing or conflicting results when integrating data.

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 17 February 2024

Last updated: 19 November 2023

Last updated: 5 March 2024

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

• Research article
• Open access
• Published: 24 September 2018

A mixed methods case study exploring the impact of membership of a multi-activity, multicentre community group on social wellbeing of older adults

• Gabrielle Lindsay-Smith   ORCID: orcid.org/0000-0003-3864-1412 1 ,
• Grant O’Sullivan 1 ,
• Rochelle Eime 1 , 2 ,
• Jack Harvey 1 , 2 &
• Jannique G. Z. van Uffelen 1 , 3  

BMC Geriatrics volume  18 , Article number:  226 ( 2018 ) Cite this article

69k Accesses

30 Citations

12 Altmetric

Metrics details

Social wellbeing factors such as loneliness and social support have a major impact on the health of older adults and can contribute to physical and mental wellbeing. However, with increasing age, social contacts and social support typically decrease and levels of loneliness increase. Group social engagement appears to have additional benefits for the health of older adults compared to socialising individually with friends and family, but further research is required to confirm whether group activities can be beneficial for the social wellbeing of older adults.

This one-year longitudinal mixed methods study investigated the effect of joining a community group, offering a range of social and physical activities, on social wellbeing of adults with a mean age of 70. The study combined a quantitative survey assessing loneliness and social support ( n  = 28; three time-points, analysed using linear mixed models) and a qualitative focus group study ( n  = 11, analysed using thematic analysis) of members from Life Activities Clubs Victoria, Australia.

There was a significant reduction in loneliness ( p  = 0.023) and a trend toward an increase in social support ( p  = 0.056) in the first year after joining. The focus group confirmed these observations and suggested that social support may take longer than 1 year to develop. Focus groups also identified that group membership provided important opportunities for developing new and diverse social connections through shared interest and experience. These connections were key in improving the social wellbeing of members, especially in their sense of feeling supported or connected and less lonely. Participants agreed that increasing connections was especially beneficial following significant life events such as retirement, moving to a new house or partners becoming unwell.

Conclusions

Becoming a member of a community group offering social and physical activities may improve social wellbeing in older adults, especially following significant life events such as retirement or moving-house, where social network changes. These results indicate that ageing policy and strategies would benefit from encouraging long-term participation in social groups to assist in adapting to changes that occur in later life and optimise healthy ageing.

Peer Review reports

Ageing population and the need to age well

Between 2015 and 2050 it is predicted that globally the number of adults over the age of 60 will more than double [ 1 ]. Increasing age is associated with a greater risk of chronic illnesses such as cardio vascular disease and cancer [ 2 ] and reduced functional capacity [ 3 , 4 ]. Consequently, an ageing population will continue to place considerable pressure on the health care systems.

However, it is also important to consider the individuals themselves and self-perceived good health is very important for the individual wellbeing and life-satisfaction of older adults [ 5 ]. The terms “successful ageing” [ 6 ] and “healthy ageing” [ 5 ] have been used to define a broader concept of ageing well, which not only includes factors relating to medically defined health but also wellbeing. Unfortunately, there is no agreed definition for what exactly constitutes healthy or successful ageing, with studies using a range of definitions. A review of 28 quantitative studies found that successful ageing was defined differently in each, with the majority only considering measures of disability or physical functioning. Social and wellbeing factors were included in only a few of the studies [ 7 ].

In contrast, qualitative studies of older adults’ opinions on successful ageing have found that while good physical and mental health and maintaining physical activity levels are agreed to assist successful ageing, being independent or doing something of value, acceptance of ageing, life satisfaction, social connectedness or keeping socially active were of greater importance [ 8 , 9 , 10 ].

In light of these findings, the definition that is most inclusive is “healthy ageing” defined by the World Health Organisation as “the process of developing and maintaining the functional ability (defined as a combination of intrinsic capacity and physical and social environmental characteristics), that enables well-being in older age” (p28) [ 5 ].This definition, and those provided in the research of older adults’ perceptions of successful ageing, highlight social engagement and social support as important factors contributing to successful ageing, in addition to being important social determinants of health [ 11 , 12 ].

Social determinants of health, including loneliness and social support, are important predictors of physical, cognitive and mental health and wellbeing in adults [ 12 ] and older adults [ 13 , 14 , 15 ]. Loneliness is defined as a perception of an inadequacy in the quality or quantity of one’s social relationships [ 16 ]. Social support, has various definitions but generally it relates to social relationships that are reciprocal, accessible and reliable and provide any or a combination of supportive resources (e.g. emotional, information, practical) and can be measured as perceived or received support [ 17 ]. These types of social determinants differ from those related to inequality (health gap social determinants) and are sometimes referred to as ‘social cure’ social determinants [ 11 ]. They will be referred to as ‘social wellbeing’ outcome measures in this study.

Unfortunately, with advancing age, there is often diminishing social support, leading to social isolation and loneliness [ 18 , 19 ]. Large nationally representative studies of adults and older adults reported that social activity predicted maintenance or improvement of life satisfaction as well as physical activity levels [ 20 ], however older adults spent less time in social activity than middle age adults.

Social wellbeing and health

A number of longitudinal studies have found that social isolation for older adults is a significant predictor of mortality and institutionalisation [ 21 , 22 , 23 ]. A meta-analysis by Holt-Lunstadt [ 12 ] reported that social determinants of health, including social integration and social support (including loneliness and lack of perceived social support) to be equal to, or a greater risk to mortality as common behavioural risk factors such as smoking, physical inactivity and obesity. Loneliness is independently associated with poor physical and mental health in the general population, and especially in older adults [ 13 , 14 , 15 ]. Adequate perceived social support has also been consistently associated with improved mental and physical health in both general and older adults [ 20 , 24 , 25 , 26 , 27 , 28 , 29 ]. The mechanism suggested for this association is that social support buffers the negative impacts of stressful situations and life events [ 30 ]. The above research demonstrates the benefit of social engagement for older adults; in turn this highlights the importance of strategies that reduce loneliness and improve social support and social connectedness for older adults.

Socialising in groups seems to be especially important for the health and wellbeing of older adults who may be adjusting to significant life events [ 26 , 31 , 32 , 33 ]. This is sometimes referred to as social engagement or social companionship [ 26 , 30 , 31 ]. It seems that the mechanism enabling such health benefits with group participation is through strengthening of social identification, which in turn increases social support [ 31 , 34 , 35 ]. Furthermore, involvement in community groups can be a sustainable strategy to reduce loneliness and increase social support in older adults, as they are generally low cost and run by volunteers [ 36 , 37 , 38 , 39 ].

Despite the demonstrated importance of social factors for successful ageing and the established risk associated with reduced social engagement as people age, few in-depth studies have longitudinally investigated the impact of community groups on social wellbeing. For example, a non-significant increase in social support and reduction in depression was found in a year-long randomised controlled trial conducted in senior centres in Norway with lonely older adults in poor physical and mental health [ 37 ]. Some qualitative studies have reported that community groups and senior centres can contribute to fun and socialisation for older adults, however social wellbeing was not the primary focus of the studies [ 38 , 40 , 41 ]. Given that social wellbeing is a broad and important area for the health and quality of life in older adults, an in-depth study is warranted to understand how it can be maximised in older adults. This mixed methods case study of an existing community aims to: i) examine whether loneliness and social support of new members of Life Activities Clubs (LACs) changes in the year after joining and ii) conduct an in-depth exploration of how social wellbeing changes in new and longer-term members of LACs.

A mixed methods study was chosen as the design for this research to enable an in-depth exploration of how loneliness and social support may change as a result of joining a community group. A case study was conducted using a concurrent mixed-methods design, with a qualitative component giving context to the quantitative results. Where the survey focused on the impact of group membership on social support and loneliness, the focus groups were an open discussion of the benefits in the lived context of LAC membership. The synthesis of the two sections of the study was undertaken at the time of interpretation of the results [ 42 ].

The two parts of our study were as follows:

a longitudinal survey (three time points over 1 year: baseline, 6 and 12 months). This part of the study formed the quantitative results;

a focus group study of members of the same organisation (qualitative).

Ethics approval to conduct this study was obtained from the Victoria University Human Research Ethics Committee (HRE14–071 [survey] and HRE15–291 [focus groups]) All participants provided informed consent to partake in the study prior to undertaking the first survey or focus group.

Setting and participants

Life activities clubs victoria.

Life Activities Clubs Victoria (LACVI) is a large not-for-profit group with 23 independently run Life Activities Clubs (LACs) based in both rural and metropolitan Victoria. It has approximately 4000 members. The organisation was established to assist in providing physical, social and recreational activities as well as education and motivational support to older adults managing significant change in their lives, especially retirement.

Eighteen out of 23 LAC clubs agreed to take part in the survey study. During the sampling period from May 2014 to December 2016, new members from the participating clubs were given information about the study and invited to take part. Invitations took place in the form of flyers distributed with new membership material.

Inclusion/ exclusion criteria

Community-dwelling older adults who self-reported that they could walk at least 100 m and who were new members to LACVI and able to complete a survey in English were eligible to participate. New members were defined as people who had never been members of LACVI or who had not been members in the last 2 years.

To ensure that the cohort of participants were of a similar functional level, people with significant health problems limiting them from being able to walk 100 m were excluded from participating in the study.

Once informed consent was received, the participants were invited to complete a self-report survey in either paper or online format (depending on preference). This first survey comprised the baseline data and the same survey was completed 6 months and 12 months after this initial time point. Participants were sent reminders if they had not completed each survey more than 2 weeks after each was delivered and then again 1 week later.

Focus groups

Two focus groups (FGs) were conducted with new and longer-term members of LACs. The first FG ( n  = 6) consisted of members who undertook physical activity in their LAC (e.g. walking groups, tennis, cycling). The second FG ( n  = 5) consisted of members who took part in activities with a non-physical activity (PA) focus (e.g. book groups, social groups, craft or cultural groups). LACs offer both social and physical activities and it was important to the study to capture both types of groups, but they were kept separate to assist participants in feeling a sense of commonality with other members and improving group dynamic and participation in the discussions [ 43 ]. Of the people who participated in the longitudinal survey study, seven also participated in the FGs.

The FG interviews were facilitated by one researcher (GLS) and notes around non-verbal communication, moments of divergence and convergence amongst group members, and other notable items were taken by a second researcher (GOS). Both researchers wrote additional notes after the focus groups and these were used in the analysis of themes. Focus groups were recorded and later transcribed verbatim by a professional transcriptionist, including identification of each participant speaking. One researcher (GLS) reviewed each transcription to check for any errors and made any required modifications before importing the transcriptions into NVivo for analysis. The transcriber identified each focus group participant so themes for individuals or other age or gender specific trends could be identified.

Dependent variables

• Social support

Social support was assessed using the Duke–UNC Functional Social support questionnaire [ 44 ]. This scale specifically measures participant perceived functional social support in two areas; i) confidant support (5 questions; e.g. chances to talk to others) and ii) affective support (3 questions; e.g. people who care about them). Participants rated each component of support on a 5-item likert scale between ‘much less than I would like’ (1 point) to ‘as much as I would like’ (5 points). The total score used for analysis was the mean of the eight scores (low social support = 1, maximum social support = 5). Construct validity, concurrent validity and discriminant validity are acceptable for confidant and affective support items in the survey in the general population [ 44 ].

Loneliness was measured using the de Jong Gierveld and UCLA-3 item loneliness scales developed for use in many populations including older adults [ 45 ]. The 11-item de Jong Gierveld loneliness scale (DJG loneliness) [ 46 ] is a multi-dimensional measure of loneliness and contains five positively worded and six negatively worded items. The items fall into four subscales; feelings of severe loneliness, feelings connected with specific problem situations, missing companionship, feelings of belongingness. The total score is the sum of the items scores (i.e. 11–55): 11 is low loneliness and 55 is severe loneliness. Self-administered versions of this scale have good internal consistency (> = 0.8) and inter-item homogeneity and person scalability that is as good or better than when conducted as face-to face interviews. The validity and reliability for the scale is adequate [ 47 ]. The UCLA 3-item loneliness scale consists of three questions about how often participants feel they lack companionship, feel left out and feel isolated. The responses are given on a three-point scale ranging from hardly ever (1) to often (3). The final score is the sum of these three items with the range being from lowest loneliness (3) to highest loneliness (9). Reliability of the scale is good, (alpha = 0.72) as are discriminant validity and internal consistency [ 48 ]. The scale is commonly used to measure loneliness with older adults ([ 49 ] – review), [ 50 , 51 ].

Sociodemographic variables

The following sociodemographic characteristics were collected in both the survey and the focus groups: age, sex, highest level of education, main life occupation [ 52 ], current employment, ability to manage on income available, present marital status, country of birth, area of residence [ 53 ]. They are categorised as indicated in Table  2 .

Health variables

The following health variables were collected: Self-rated general health (from SF-12) [ 54 ] and Functional health (ability to walk 100 m- formed part of the inclusion criteria) [ 55 ]. See Table 2 for details about the categories of these variables.

The effects of becoming a member on quantitative outcome variables (i.e. Social support, DJG loneliness and UCLA loneliness) were analysed using linear mixed models (LMM). LMM enabled testing for the presence of intra-subject random effects, or equivalently, correlation of subjects’ measures over time (baseline, 6-months and 12 months). Three correlation structures were examined: independence (no correlation), compound symmetry (constant correlation of each subjects’ measures over the three time points) and autoregressive (correlation diminishing with increase in spacing in time). The best fitting correlation structure was compound symmetry; this is equivalent to a random intercept component for each subject. The LMM incorporated longitudinal trends over time, with adjustment for age as a potential confounder. Statistical analyses were conducted using SPSS for windows (v24).

UCLA loneliness and social support residuals were not normally distributed and these scales were Log10 transformed for statistical analysis.

Analyses were all adjusted for age, group attendance (calculated as average attendance at 6 and 12 months) and employment status at baseline (Full-time, Part-time, not working).

Focus group transcripts were analysed using thematic analysis [ 56 , 57 ], a flexible qualitative methodology that can be used with a variety of epistemologies, approaches and analysis methods [ 56 ]. The transcribed data were analysed using a combination of theoretical and inductive thematic analysis [ 56 ]. It was theorised that membership in a LAC would assist with social factors relating to healthy ageing [ 5 ], possibly through a social identity pathway [ 58 ], although we wanted to explore this. Semantic themes were drawn from these codes in order to conduct a pragmatic evaluation of the LACVI programs [ 56 ]. Analytic rigour in the qualitative analysis was ensured through source and analyst triangulation. Transcriptions were compared to notes taken during the focus groups by the researchers (GOS and GLS). In addition, Initial coding and themes (by GLS) were checked by a second researcher (GOS) and any disagreements regarding coding and themes were discussed prior to finalisation of codes and themes [ 57 ].

Sociodemographic and health characteristics of the 28 participants who completed the survey study are reported in Table  1 . The mean age of the participants was 66.9 and 75% were female. These demographics are representative of the entire LACVI membership. Education levels varied, with 21% being university educated, and the remainder completing high school or technical certificates. Two thirds of participants were not married. Some sociodemographic characteristics changed slightly at 6 and 12 months, mainly employment (18% in paid employment at baseline and 11% at 12-months) and ability to manage on income (36% reporting trouble managing on their income at baseline and 46% at 12 months). Almost 90% of the participants described themselves as being in good-excellent health.

Types of activities

There were a variety of types of activities that participants took part in: physical activities such as walking groups ( n  = 7), table tennis ( n  = 5), dancing class ( n  = 2), exercise class ( n  = 1), bowls ( n  = 2), golf ( n  = 3), cycling groups ( n  = 1) and non-physical leisure activities such as art and literature groups ( n  = 5), craft groups ( n  = 5), entertainment groups ( n  = 12), food/dine out groups ( n  = 18) and other sedentary leisure activities (e.g. mah jong, cards),( n  = 4). A number of people took part in more than one activity.

Frequency of attendance at LACVI and changes in social wellbeing

At six and 12 months, participants indicated how many times in the last month they attended different types of activities at their LAC. Most participants maintained the same frequency of participation over both time points. Only four people participated more frequently at 12 than at 6 months and nine reduced participation levels. The latter group included predominantly those who reduced from more than two times per week at 6 months to 2×/week at 6 months to one to two times per week ( n  = 5) or less than one time per week ( n  = 2) at 12 months. Average weekly club attendance at six and 12 months was included as a covariate in the statistical model.

Outcome measures

Overall, participants reported moderate social support and loneliness levels at baseline (See Table 2 ). Loneliness, as measured by both scales, reduced significantly over time. There was a significant effect of time on the DJG loneliness scores (F (2, 52) = 3.83, p  = 0.028), with Post-Hoc analysis indicating a reduction in DJG loneliness between baseline and 12 months ( p  = 0.008). UCLA loneliness scores (transformed variable) also changed significantly over time (F (2, 52) = 4.08, p  = 0.023). Post hoc tests indicated a reduction in UCLA loneliness between baseline and 6 months ( p  = 0.007). There was a small non-significant increase in social support (F (2, 53) =2.88, p  = 0.065) during the first year of membership (see Table 2 and Figs. 1 and 2 ).

DJG loneliness for all participants over first year of membership at LAC club ( n  = 28).

*Represents significant difference compared to baseline ( p  < 0.01)

UCLA loneliness score for all participants over first year of membership at LAC club ( n  = 28).

*Indicates log values of the variable at 6-months were significantly different from baseline ( p  < 0.01)

In total, 11 participants attended the two focus groups, six people who participated in PA clubs (four women) and five who participated in social clubs (all women). All focus group participants were either retired ( n  = 9) or semi-retired ( n  = 2). The mean age of participants was 67 years (see Table 2 for further details). Most of the participants (82%) had been members of a LAC for less than 2 years and two females in the social group had been members of LAC clubs for 5 and 10 years respectively.

Analysis of the focus group transcripts identified two themes relating to social benefits of group participation; i) Social resources and ii) Social wellbeing (see Fig. 3 ). Group discussion suggested that membership of a LAC provides access to more social resources through greater and diverse social contact and opportunity. It is through this improvement in social resources that social wellbeing may improve.

Themes arising from focus group discussion around the benefits of LAC membership

Social resources

The social resources theme referred to an increase in the availability and variety of social connections that resulted from becoming a member of a LAC. The social nature of the groups enabled an expansion and diversification of members’ social network and improved their sense of social connectedness. There was widespread agreement in both the focus groups that significant life events, especially retirement, illness or death of spouse and moving house changes one’s social resources. Membership of the LAC had benefits especially at these times and these events were often motivators to join such a club. Most participants found that their social resources declined after retirement and even felt that they were grieving for the loss of their work.

“ I just saw work as a collection of, um, colleagues as opposed to friends. I had a few good friends there. Most were simply colleagues or acquaintances …. [interviewer- Mmm.] ..Okay, you’d talk to them every day. You’d chatter in the kitchen, oh, pass banter back and forth when things are busy or quiet, but... Um, in terms of a friendship with those people, like going to their home, getting to know them, doing other things with them, very few. But what I did miss was the interaction with other people. It had simply gone….. But, yeah, look, that, the, yeah, that intervening period was, oh, a couple of months. That was a bit tough…. But in that time the people in LAC and the people in U3A…. And the other dance group just drew me into more things. Got to know more people. So once again, yeah, reasonable group of acquaintances.” (Male, PAFG)

Group members indicated general agreement with these two responses, however one female found she had a greater social life following retirement due to the busy nature of her job.

Within the social resources theme, three subthemes were identified, i) Opportunity for social connectedness, ii) Opportunity for friendships, and iii) Opportunity for social responsibility/leadership . Interestingly, these subthemes were additional to the information gathered in the survey. This emphasises the power of the inductive nature of the qualitative exploration employed in the focus groups to broaden the knowledge in this area.

The most discussed and expanded subtheme in both focus groups was Opportunity for social connectedness , which arose through developing new connections, diversifying social connections, sharing interests and experiences with others and peer learning. Participants in both focus groups stated that being a member of LAC facilitated their socialising and connecting with others to share ideas, skills and to do activities with, which was especially important through times of significant life events. Furthermore, participants in each of the focus groups valued developing diverse connections:

“ Yeah, I think, as I said, I finished up work and I, and I had more time for wa-, walking. So I think a, in meeting, in going to this group which, I saw this group of women but then someone introduced me to them. They were just meeting, just meeting a new different set of people, you know? As I said, my work people and these were just a whole different group of women, mainly women. There’s not many men. [Interviewer: Yes.]….. Although our leader is a man, which is ironic and is about, this man out in front and there’s about 20 women behind him, but, um, so yeah, and people from different walks of life and different nationalities there which I never knew in my work life, so yeah. That’s been great. So from that goes on other things, you know, you might, uh, other activities and, yeah, people for coffee and go to the pictures or something, yeah. That’s great.” (Female, PAFG)

Simply making new connections was the most widely discussed aspect related to the opportunity for social connectedness subtheme, with all participants agreeing that this was an important benefit of participation in LAC groups.

“Well, my experience is very similar to everybody else’s…….: I, I went from having no social life to a social life once I joined a group.” (Female, PAFG)

There was agreement in both focus groups that these initial new connections made at a LAC are strengthened through development of deeper personal connections with others who have similar demographics and who are interested in the same activities. This concurs with the Social Identity Theory [ 58 ] discussed previously.

“and I was walking around the lake in Ballarat, like wandering on my own. I thought, This is ridiculous. I mean, you’ve met all those groups of women coming the opposite way, so I found out what it was all about, so I joined, yeah. So that’s how I got into that.[ Interviewer: Yeah.] Basically sick of walking round the lake on my own. [Interviewer: Yeah, yeah.] So that’s great. It’s very social and they have coffee afterwards which is good.” (female, PAFG)

The subtheme Opportunity for development of friendships describes how, for some people, a number of LAC members have progressed from being just initial social connections to an established friendship. This signifies the strength of the connections that may potentially develop through LAC membership. Some participants from each group mentioned friendships developing, with slightly more discussion of this seen in the social group.

“we all have a good old chat, you know, and, and it’s all about friendship as well.” (female, SocialFG)

The subtheme Opportunity for social responsibility or leadership was mentioned by two people in the active group, however it was not brought up in the social group. This opportunity for leadership is linked with the development of a group identity and desiring to contribute meaningfully to a valued group.

“with our riding group, um, you, a leader for probably two rides a year so you’ve gotta prepare for it, so some of them do reccie rides themselves, so, um, and also every, uh, so that’s something that’s, uh, a responsibility.” (male, PAFG)

Social wellbeing

The social resources described above seem to contribute to a number of social, wellbeing outcomes for participants. The sub themes identified for Social wellbeing were , i) Increased social support, ii) Reduced loneliness, iii) Improved home relationships and iv) Improved social skills.

Increased social support

Social support was measured quantitatively in the survey (no significant change over time for new members) and identified as a benefit of LAC membership during the focus group discussions. However, only one of the members of the active group mentioned social support directly.

‘it’s nice to be able to pick up the phone and share your problem with somebody else, and that’s come about through LAC. ……‘Cos before that it was through, with my family (female, PAFG)

There was some agreement amongst participants of the PA group that they felt this kind of support may develop in time but most of them had been members for less than 2 years.

“[Interviewer: Yeah. Does anyone else have that experience? (relating to above quote)]” There is one lady but she’s actually the one that I joined with anyway. [Interviewer: Okay.] But I, I feel there are others that are definitely getting towards that stage. It’s still going quite early days. (female1, PAFG) [Interviewer: I guess it’s quite early for some of you, yeah.] “yeah” (female 2, PAFG)

Social support through sharing of skills was mentioned by one participant in the social group also, with agreement indicated by most of the others in the social focus group.

Discussion in the focus groups also touched on the subthemes Reduced loneliness and Improved home relationships, which were each mentioned by one person. And focus groups also felt that group membership Improved social skills through opening up and becoming more approachable (male, PAFG) or enabling them to become more accepting of others’ who are different (general agreement in Social FG).

This case study integrated results from a one-year longitudinal survey study and focus group discussions to gather rich information regarding the potential changes in social wellbeing that older adults may experience when joining community organisations offering group activities. The findings from this study indicate that becoming a member of such a community organisation can be associated with a range of social benefits for older adults, particularly related to reducing loneliness and maintaining social connections.

Joining a LAC was associated with a reduction in loneliness over 1 year. This finding is in line with past group-intervention studies where social activity groups were found to assist in reducing loneliness and social isolation [ 49 ]. This systematic review highlighted that the majority of the literature explored the effectiveness of group activity interventions for reducing severe loneliness or loneliness in clinical populations [ 49 ]. The present study extends this research to the general older adult population who are not specifically lonely and reported to be of good general health, rather than a clinical focus. Our findings are in contrast to results from an evaluation of a community capacity-building program aimed at reducing social isolation in older adults in rural Australia [ 59 ]. That program did not successfully reduce loneliness or improve social support. The lack of change from pre- to post-program in that study was reasoned to be due to sampling error, unstandardised data collection, and changes in sample characteristics across the programs [ 59 ]. Qualitative assessment of the same program [ 59 ] did however suggest that participants felt it was successful in reducing social isolation, which does support our findings.

Changes in loneliness were not a main discussion point of the qualitative component of the current study, however some participants did express that they felt less lonely since joining LACVI and all felt they had become more connected with others. This is not so much of a contrast in results as a potential situational issue. The lack of discussion of loneliness may have been linked to the common social stigma around experiencing loneliness outside certain accepted circumstances (e.g. widowhood), which may lead to underreporting in front of others [ 45 ].

Overall, both components of the study suggest that becoming a member of an activity group may be associated with reductions in loneliness, or at least a greater sense of social connectedness. In addition to the social nature of the groups and increased opportunity for social connections, another possible link between group activity and reduced loneliness is an increased opportunity for time out of home. Previous research has found that more time away from home in an average day is associated with lower loneliness in older adults [ 60 ]. Given the significant health and social problems that are related to loneliness and social isolation [ 13 , 14 , 15 ], the importance of group involvement for newly retired adults to prevent loneliness should be advocated.

In line with a significant reduction in loneliness, there was also a trend ( p  = 0.056) toward an increase in social support from baseline to 12 months in the survey study. Whilst suggestive of a change, it is far less conclusive than the findings for loneliness. There are a number of possible explanations for the lack of statistically significant change in this variable over the course of the study. The first is the small sample size, which would reduce the statistical power of the study. It may be that larger studies are required to observe changes in social support, which are possibly only subtle over the course of 1 year. This idea is supported by a year-long randomised controlled trial with 90 mildly-depressed older adults who attended senior citizen’s club in Norway [ 37 ]. The study failed to see any change in general social support in the intervention group compared to the control over 1 year. Additional analysis in that study suggested that people who attended the intervention groups more often, tended to have greater increases in SS ( p  = 0.08). The researchers stated that the study suffered from significant drop-out rates and low power as a result. In this way, it was similar to our findings and suggests that social support studies require larger numbers than we were able to gain in this early exploratory study. Another possible reason for small changes in SS in the current study may be the type of SS measured. The scale used gathered information around functional support or support given to individuals in times of need. Maybe it is not this type of support that changes in such groups but more specific support such as task-specific support. It has been observed in other studies and reviews that task-specific support changes as a result of behavioural interventions (e.g. PA interventions) but general support does not seem to change in the time frames often studied [ 61 , 62 , 63 ].

There were many social wellbeing benefits such as increased social connectivity identified in focus group discussion, but the specific theme of social support was rarely mentioned. It may be that general social support through such community groups may take longer than 1 year to develop. There is evidence that strong group ties are sequentially positively associated between social identification and social support [ 34 ], suggesting that the connections formed through the groups may lead increased to social support from group members in the future. This is supported by results from the focus group discussions, where one new member felt she could call on colleagues she met in her new group. Other new members thought it was too soon for this support to be available, but they could see the bonds developing.

Other social wellbeing changes

In addition to social support and loneliness that were the focus of the quantitative study, the focus group discussions uncovered a number of other benefits of group membership that were related to social wellbeing (see Fig. 3 ). The social resources theme was of particular interest because it reflected some of the mechanisms that appeared enable social wellbeing changes as a result of being a member of a LAC but were not measured in the survey. The main social resources relating to group membership that were mentioned in the focus groups were social connectedness, development of friendships and opportunity for social responsibility or leadership. As mentioned above, there was wide-spread discussion within the focus groups of the development of social connections through the clubs. Social connectedness is defined as “the sense of belonging and subjective psychological bond that people feel in relation to individuals and groups of others.” ([ 25 ], pp1). As well as being an important predecessor of social support, greater social connectedness has been found to be highly important for the health of older adults, especially cognitive and mental health [ 26 , 32 , 34 , 35 , 64 ]. One suggested theory for this health benefit is that connections developed through groups that we strongly identify with are likely to be important for the development of social identity [ 34 ], defined by Taifel as: “knowledge that [we] belong to certain social groups together with some emotional and value significance to [us] of this group membership” (Tajfel, 1972, p. 31 in [ 58 ] p 2). These types of groups to which we identify may be a source of “personal security, social companionship, emotional bonding, intellectual stimulation, and collaborative learning and……allow us to achieve goals.” ([ 58 ] p2) and an overall sense of self-worth and wellbeing. There was a great deal of discussion relating to the opportunity for social connectedness derived through group membership being particularly pertinent following a significant life event such as moving to a new house or partners becoming unwell or dying and especially retirement. This change in their social circumstance is likely to have triggered the need to renew their social identity by joining a community group. Research with university students has shown that new group identification can assist in transition for university students who have lost their old groups of friends because of starting university [ 65 ]. In an example relevant to older adults, maintenance or increase in number of group memberships at the time of retirement reduced mortality risk 8 years later compared to people who reduce their number of group activities in a longitudinal cohort study [ 66 ]. This would fit with the original Activity Theory of ageing; whereby better ageing experience is achieved when levels of social participation are maintained, and role replacement occurs when old roles (such as working roles) must be relinquished [ 67 ]. These connections therefore appear to assist in maintaining resilience in older adults defined as “the ability to maintain or improve a level of functional ability (a combination of intrinsic physical and mental capacity and environment) in the face of adversity” (p29, [ 5 ]). Factors that were mentioned in the focus groups as assisting participants in forming connections with others were shared interest, learning from others, and a fun and accepting environment. It was not possible to assess all life events in the survey study. However, since the discussion from the focus groups suggested this to be an important motivator for joining clubs and potentially a beneficial time for joining them, it would be worth exploring in future studies.

Focus group discussion suggested that an especially valuable time for joining such clubs was around retirement, to assist with maintaining social connectivity. The social groups seem to provide social activity and new roles for these older adults at times of change. It is not necessarily important for all older adults but maybe these ones identify themselves as social beings and therefore this maintenance of social connection helps to continue their social role. Given the suggested importance of social connectivity gained through this organisation, especially at times of significant life events, it would valuable to investigate this further in future and consider encouragement of such through government policy and funding. The majority of these types of clubs exist for older adults in general, but this study emphasises the need for groups such as these to target newly retired individuals specifically and to ensure that they are not seen as ‘only for old people’.

Strengths and limitations

The use of mixed –methodologies, combining longitudinal survey study analysed quantitatively, with a qualitative exploration through focus group discussions and thematic analysis, was a strength of the current study. It allowed the researchers to not only examine the association between becoming a member of a community group on social support and loneliness over an extended period, but also obtain a deeper understanding of the underlying reasons behind any associations. Given the variability of social support definitions in research [ 17 ] and the broad area of social wellbeing, it allowed for open exploration of the topic, to understand associations that may exist but would have otherwise been missed. Embedding the research in an existing community organisation was a strength, although with this also came some difficulties with recruitment. Voluntary coordination of the community groups meant that informing new members about the study was not always feasible or a priority for the volunteers. In addition, calling for new members was innately challenging because they were not yet committed to the club fully. This meant that so some people did not want to commit to a year-long study if they were not sure how long they would be a member of the club. This resulted in slow recruitment and a resulting relatively low sample size and decreased power to show significant statistical differences, which is a limitation of the present study. However, the use of Linear Mixed Models for analysis of the survey data was a strength because it was able to include all data in the analyses and not remove participants if one time point of data was missing, as repeated measures ANOVAs would do. The length of the study (1 year) is another strength, especially compared to previous randomised controlled studies that are typically only 6–16 weeks in length. Drop-out rate in the current study is very low and probably attributable to the benefits of working with long-standing organisations.

The purpose of this study was to explore in detail whether there are any relationships between joining existing community groups for older adults and social wellbeing. The lack of existing evidence in the field meant that a small feasibility-type case study was a good sounding-board for future larger scale research on the topic, despite not being able to answer questions of causality. Owing to the particularistic nature of case studies, it can also be difficult to generalise to other types of organisations or groups unless there is a great deal of similarity between them [ 68 ]. There are however, other types of community organisations in existence that have a similar structure to LACVI (Seniors centres [ 36 , 40 ], Men’s Sheds [ 38 ], University of the Third Age [ 34 , 69 ], Japanese salons [ 70 , 71 ]) and it may be that the results from this study are transferable to these also. This study adds to the literature around the benefits of joining community organisations that offer social and physical activities for older adults and suggests that this engagement may assist with reducing loneliness and maintaining social connection, especially around the time of retirement.

Directions for future research

Given that social support trended toward a significant increase, it would be useful to repeat the study on a larger scale in future to confirm this. Either a case study on a similar but larger community group or combining a number of community organisations would enable recruitment of more participants. Such an approach would also assist in assessing the generalisability of our findings to other community groups. Given that discussions around social benefits of group membership in the focus groups was often raised in conjunction with the occurrence of significant life events, it would be beneficial to include a significant life event scale in any future studies in this area. The qualitative results also suggest that it would be useful to investigate whether people who join community groups in early years post retirement gain the same social benefits as those in later stages of retirement. Studies investigating additional health benefits of these community groups such as physical activity, depression and general wellbeing would also be warranted.

With an ageing population, it is important to investigate ways to enable older adults to age successfully to ensure optimal quality of life and minimisation of health care costs. Social determinants of health such as social support, loneliness and social contact are important contributors to successful ageing through improvements in cognitive health, quality of life, reduction in depression and reduction in mortality. Unfortunately, older adults are at risk of these social factors declining in older age and there is little research investigating how best to tackle this. Community groups offering a range of activities may assist by improving social connectedness and social support and reducing loneliness for older adults. Some factors that may assist with this are activities that encourage sharing interests, learning from others, and are conducted in a fun and accepting environment. Such groups may be particularly important in developing social contacts for newly retired individuals or around other significant life events such as moving or illness of loved ones. In conclusion, ageing policy and strategies should emphasise participation in community groups especially for those recently retired, as they may assist in reducing loneliness and increasing social connections for older adults.

Abbreviations

Focus group

Life Activities Club

Life Activities Clubs Victoria

Linear mixed model

Physical activity

World Health Organisation

United Nations. In: P.D. Department of Economic and Social Affairs, editor. World Population Ageing 2015. New York: United Nations; 2015.

Google Scholar  

World Health Organisation. Global Health and Ageing. 2011 [cited 2014 March 25]; Available from: http://www.who.int/ageing/publications/global_health/en/ .

Balogun JA, et al. Age-related changes in balance performance. Disabil Rehabil. 1994;16(2):58–62.

Article   CAS   Google Scholar  

Singh MAF. Exercise comes of age: rationale and recommendations for a geriatric exercise prescription. J Gerontol Ser A Biol Med Sci. 2002;57(5):M262–82.

Article   Google Scholar  

World Health Organisation. World report on ageing and health. Geneva: World Health Organisation; 2015.

Rowe JW, Kahn RL. Successful Aging1. The Gerontologist. 1997;37(4):433–40.

Depp CA, Jeste DV. Definitions and predictors of successful aging: a comprehensive review of larger quantitative studies. Am J Geriatr Psychiatry. 2006;14(1):6–20.

Song M, Kong E-H. Older adults’ definitions of health: a metasynthesis. Int J Nurs Stud. 2015;52(6):1097–106.

Tate RB, Lah L, Cuddy TE. Definition of successful aging by elderly Canadian males: the Manitoba follow-up study. The Gerontologist. 2003;43(5):735–44.

Phelan EA, et al. Older Adults’ views of “successful aging”—how do they compare with Researchers’ definitions? J Am Geriatr Soc. 2004;52(2):211–6.

Haslam SA, et al. Social cure, what social cure? The propensity to underestimate the importance of social factors for health. Soc Sci Med. 2018;198:14–21.

Holt-Lunstad J, Smith TB, Layton JB. Social relationships and mortality risk: a meta-analytic review. PLoS Med. 2010;7(7):e1000316.

Richard A, et al. Loneliness is adversely associated with physical and mental health and lifestyle factors: results from a Swiss national survey. PLoS One. 2017;12(7):1–18.

Luo Y, et al. Loneliness, health, and mortality in old age: a national longitudinal study. Soc Sci Med. 2012;74(6):907–14.

Luo Y, Waite LJ. Loneliness and mortality among older adults in China. J Gerontol B Psychol Sci Soc Sci. 2014;69(4):633–45.

de Jong Gierveld J, Van Tilburg T, Dykstra PA. Loneliness and social isolation. In: Vangelisti A, Perlman D, editors. Cambridge handbook of personal relationships. Cambridge: Cambridge University Press; 2006. p. 485-500.

Williams P, Barclay L, Schmied V. Defining social support in context: a necessary step in improving research, intervention, and practice. Qual Health Res. 2004;14(7):942–60.

Valtorta N, Hanratty B. Loneliness, isolation and the health of older adults: do we need a new research agenda? J R Soc Med. 2012;105(12):518–22.

Jylhä M. Old age and loneliness: cross-sectional and longitudinal analyses in the Tampere longitudinal study on aging. Can J Aging La Revue can du vieil. 2010;23(2):157–68.

Huxhold O, Miche M, Schüz B. Benefits of having friends in older ages: differential effects of informal social activities on well-being in middle-aged and older adults. J Gerontol B Psychol Sci Soc Sci. 2014;69(3):366–75.

Berkman L, Syme S. Social networks, host resistance and mortality: a nine year follow-up study of alameda county residents. Am J Epidemiol. 1979;185(11):1070–88.

House JS, Landis KR, Umberson D. Social relationships and health. Science. 1988;241(4865):540–5.

Pynnonen K, et al. Does social activity decrease risk for institutionalization and mortality in older people? J Gerontol B Psychol Sci Soc Sci. 2012;67(6):765–74.

Broadhead WE, et al. The epidemiologic evidence for a relationship between social support and health. Am J Epidemiol. 1983;117(5):521–37.

Haslam C, et al. Social connectedness and health. Encyclopedia of geropsychology. 2017:2174–82. https://doi.org/10.1007/978-981-287-080-3_46-1 .

Haslam C, Cruwys T, Haslam SA. “The we’s have it”: evidence for the distinctive benefits of group engagement in enhancing cognitive health in aging. Soc Sci Med. 2014;120:57–66.

Uebelacker LA, et al. Social support and physical activity as moderators of life stress in predicting baseline depression and change in depression over time in the Women’s Health Initiative. Soc Psychiatry Psychiatr Epidemiol. 2013;48(12):1971–82.

Tajvar M, et al. Social support and health of older people in middle eastern countries: a systematic review. Australas J Ageing. 2013;32(2):71–8.

Dalgard OS, Bjork S, Tambs K. Social support, negative life events and mental health. Br J Psychiatry. 1995;166(1):29–34.

Cohen S, Wills TA. Stress, social support, and the buffering hypothesis. Psychol Bull. 1985;98(2):310–57.

Gilmour H. Social participation and the health and well-being of Canadian seniors. Health Rep. 2012;23(4):1B.

Glei DA, et al. Participating in social activities helps preserve cognitive function: an analysis of a longitudinal, population-based study of the elderly. Int J Epidemiol. 2005;34(4):864–71.

Lee SH, Kim YB. Which type of social activities may reduce cognitive decline in the elderly?: a longitudinal population-based study. BMC Geriatr. 2016;16(1):165.

Haslam C, et al. Group ties protect cognitive health by promoting social identification and social support. J Aging Health. 2016;28(2):244–66.

Haslam C, et al. Groups 4 health: evidence that a social-identity intervention that builds and strengthens social group membership improves mental health. J Affect Disord. 2016;194:188–95.

Bøen H. Characteristics of senior Centre users -- and the impact of a group programme on social support and late-life depression. Norsk Epidemiologi. 2012;22(2):261–9.

Bøen H, et al. A randomized controlled trial of a senior Centre group programme for increasing social support and preventing depression in elderly people living at home in Norway. BMC Geriatr. 2012;12:20.

Golding BG. Social, local, and situated: recent findings about the effectiveness of older Men’s informal learning in community contexts. Adult Educ Q. 2011;61(2):103.

Life Activities Clubs. Life Activities Clubs. About Us. . 2014 [cited 2014 January 13, 2014]; Available from: http://www.life.org.au/aboutus .

Hutchinson SL, Gallant KA. Can senior Centres be contexts for aging in third places? J Leis Res. 2016;48(1):50–68.

Millard J. The health of older adults in community activities. Work Older People. 2017;21(2):90–9.

Creswell JW, Plano-Clark VL. Designing and conducting mixed methods research. Thousand Oaks, Calif: SAGE Publications; 2007. p. c2007.

Loeb S, Penrod J, Hupcey J. Focus groups and older adults: tactics for success. J Gerontol Nurs. 2006;32(3):32–8.

PubMed   Google Scholar  

Broadhead W, et al. The Duke-UNC functional social support questionnaire: measurement of social support in family medicine patients. Med Care. 1988;26(7):709–23.

De Jong Gierveld J, Van Tilburg T. Living arrangements of older adults in the Netherlands and Italy: Coresidence values and behaviour and their consequences for loneliness. J Cross Cult Gerontol. 1999;14(1):1–24.

de Jong-Gierveld J, Kamphuls F. The development of a Rasch-type loneliness scale. Appl Psychol Meas. 1985;9(3):289–99.

Tilburg Tv, Leeuw Ed. Stability of scale quality under various data collection procedures: a mode comparison on the ‘De Jong-Gierveld loneliness scale. Int J Public Opin Res. 1991;3(1):69–85.

Hughes ME, et al. A short scale for measuring loneliness in large surveys - results from two population-based studies. Res Aging. 2004;26(6):655–72.

Dickens AP, et al. Interventions targeting social isolation in older people: a systematic review. BMC Public Health. 2011;11:647.

Hawkley LC, et al. Loneliness predicts increased blood pressure: 5-year cross-lagged analyses in middle-aged and older adults. Psychol Aging. 2010;25(1):132.

Netz Y, et al. Loneliness is associated with an increased risk of sedentary life in older Israelis. Aging Ment Health. 2013;17(1):40–7.

Australian Bureau of Statistics. Australian and New Zealand Standard Classification of Occupations, 2013, Version 1.2. Canberra: Australian Bureau of Statistics; 2013.

Australian Bureau of Statistics, Australian Standard Geographical Classification (ASGC). Canberra: Australian Bureau of Statistics; 2011.

Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey - construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220–33.

Sanson-Fisher RW, Perkins JJ. Adaptation and validation of the SF-36 health survey for use in Australia. J Clin Epidemiol. 51(11):961–7.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Patton MQ. Qualitative research & evaluation methods. 4 ed. Thousand Oaks, California: Sage Publications; 2015.

Haslam SA, et al. Social identity, health and well-being: an emerging agenda for applied psychology. Appl Psychol. 2009;58(1):1–23.

Bartlett H, et al. Preventing social isolation in later life: findings and insights from a pilot Queensland intervention study. Ageing Soc. 2013;33(07):1167–89.

Petersen J, et al. Time out-of-home and cognitive, physical, and emotional wellbeing of older adults: a longitudinal mixed effects model. PLoS One. 2015;10(10):e0139643.

Lindsay Smith G, et al. The association between social support and physical activity in older adults: a systematic review. Int J Behav Nutr Phys Act. 2017;14(1):56.

Sallis JF, et al. The development of scales to measure social support for diet and exercise behaviors. Prev Med. 1987;16(6):825–36.

Oka R, King A, Young DR. Sources of social support as predictors of exercise adherence in women and men ages 50 to 65 years. Womens Health Res Gender Behav Policy. 1995;1:161–75.

CAS   Google Scholar  

Greenaway KH, et al. From “we” to “me”: group identification enhances perceived personal control with consequences for health and well-being. J Pers Soc Psychol. 2015;109(1):53–74.

Iyer A, et al. The more (and the more compatible) the merrier: multiple group memberships and identity compatibility as predictors of adjustment after life transitions. Br J Soc Psychol. 2009;48(4):707–33.

Steffens NK, et al. Social group memberships in retirement are associated with reduced risk of premature death: evidence from a longitudinal cohort study. BMJ Open. 2016;6(2):e010164.

Havighurst RJ. Successful aging. Gerontol. 1961;1:8–13.

Yin R. Case study research: design and methods. Beverly Hills, CA: Sage publishing; 1994.

Merriam SB, Kee Y. Promoting community wellbeing: the case for lifelong learning for older adults. Adult Educ Q. 2014;64(2):128–44.

Hikichi H, et al. Social interaction and cognitive decline: results of a 7-year community intervention. Alzheimers Dement: Translat Res Clin Interv. 2017;3(1):23–32.

Hikichi H, et al. Effect of a community intervention programme promoting social interactions on functional disability prevention for older adults: propensity score matching and instrumental variable analyses, JAGES Taketoyo study. J Epidemiol Community Health. 2015;69(9):905–10.

Download references

The primary author contributing to this study (GLS) receives PhD scholarship funding from Victoria University. The other authors were funded through salaries at Victoria University.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due the ethics approval for this study not allowing open access to the individual participant data but are available from the corresponding author on reasonable request.

Author information

Authors and affiliations.

Institute for Health and Sport, Victoria University, Melbourne, Australia

Gabrielle Lindsay-Smith, Grant O’Sullivan, Rochelle Eime, Jack Harvey & Jannique G. Z. van Uffelen

School of Health and Life Sciences, Federation University, Ballarat, Australia

Rochelle Eime & Jack Harvey

Department of Movement Sciences, Physical Activity, Sports and Health Research Group, KU Leuven - University of Leuven, Leuven, Belgium

Jannique G. Z. van Uffelen

You can also search for this author in PubMed   Google Scholar

Contributions

GLS, RE and JVU made substantial contributions to the conception and design of the study. GLS and GOS supervised data collection for the surveys (GLS) and focus groups (GOS and GLS). GLS, GOS, RE, JH and JVU were involved in data analysis and interpretation. All authors were involved in drafting, the manuscript and approved the final version.

Corresponding author

Correspondence to Gabrielle Lindsay-Smith .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval to conduct this study was obtained from the Victoria University Human Research Ethics Committee (HRE14–071 [survey] and HRE15–291 [focus groups]). All survey participants provided informed consent to partake in the study prior to undertaking the first survey by clicking ‘agree’ in the survey software (online participants) or signing a written consent form (paper participants) after they had read all participant information relating to the study. Focus group participants also completed a written consent form after reading and understanding all participant information regarding the study.

Consent for publication

Focus group participants have given permission for their quotes to be used in de-identified form prior to undertaking the research.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Lindsay-Smith, G., O’Sullivan, G., Eime, R. et al. A mixed methods case study exploring the impact of membership of a multi-activity, multicentre community group on social wellbeing of older adults. BMC Geriatr 18 , 226 (2018). https://doi.org/10.1186/s12877-018-0913-1

Download citation

Received : 19 April 2018

Accepted : 10 September 2018

Published : 24 September 2018

DOI : https://doi.org/10.1186/s12877-018-0913-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social engagement
• Group activity

BMC Geriatrics

ISSN: 1471-2318

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Privacy Policy

Buy Me a Coffee

Home » Case Study – Methods, Examples and Guide

Case Study – Methods, Examples and Guide

Table of Contents

A case study is a research method that involves an in-depth examination and analysis of a particular phenomenon or case, such as an individual, organization, community, event, or situation.

It is a qualitative research approach that aims to provide a detailed and comprehensive understanding of the case being studied. Case studies typically involve multiple sources of data, including interviews, observations, documents, and artifacts, which are analyzed using various techniques, such as content analysis, thematic analysis, and grounded theory. The findings of a case study are often used to develop theories, inform policy or practice, or generate new research questions.

Types of Case Study

Types and Methods of Case Study are as follows:

Single-Case Study

A single-case study is an in-depth analysis of a single case. This type of case study is useful when the researcher wants to understand a specific phenomenon in detail.

For Example , A researcher might conduct a single-case study on a particular individual to understand their experiences with a particular health condition or a specific organization to explore their management practices. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a single-case study are often used to generate new research questions, develop theories, or inform policy or practice.

Multiple-Case Study

A multiple-case study involves the analysis of several cases that are similar in nature. This type of case study is useful when the researcher wants to identify similarities and differences between the cases.

For Example, a researcher might conduct a multiple-case study on several companies to explore the factors that contribute to their success or failure. The researcher collects data from each case, compares and contrasts the findings, and uses various techniques to analyze the data, such as comparative analysis or pattern-matching. The findings of a multiple-case study can be used to develop theories, inform policy or practice, or generate new research questions.

Exploratory Case Study

An exploratory case study is used to explore a new or understudied phenomenon. This type of case study is useful when the researcher wants to generate hypotheses or theories about the phenomenon.

For Example, a researcher might conduct an exploratory case study on a new technology to understand its potential impact on society. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as grounded theory or content analysis. The findings of an exploratory case study can be used to generate new research questions, develop theories, or inform policy or practice.

Descriptive Case Study

A descriptive case study is used to describe a particular phenomenon in detail. This type of case study is useful when the researcher wants to provide a comprehensive account of the phenomenon.

For Example, a researcher might conduct a descriptive case study on a particular community to understand its social and economic characteristics. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a descriptive case study can be used to inform policy or practice or generate new research questions.

Instrumental Case Study

An instrumental case study is used to understand a particular phenomenon that is instrumental in achieving a particular goal. This type of case study is useful when the researcher wants to understand the role of the phenomenon in achieving the goal.

For Example, a researcher might conduct an instrumental case study on a particular policy to understand its impact on achieving a particular goal, such as reducing poverty. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of an instrumental case study can be used to inform policy or practice or generate new research questions.

Case Study Data Collection Methods

Here are some common data collection methods for case studies:

Interviews involve asking questions to individuals who have knowledge or experience relevant to the case study. Interviews can be structured (where the same questions are asked to all participants) or unstructured (where the interviewer follows up on the responses with further questions). Interviews can be conducted in person, over the phone, or through video conferencing.

Observations

Observations involve watching and recording the behavior and activities of individuals or groups relevant to the case study. Observations can be participant (where the researcher actively participates in the activities) or non-participant (where the researcher observes from a distance). Observations can be recorded using notes, audio or video recordings, or photographs.

Documents can be used as a source of information for case studies. Documents can include reports, memos, emails, letters, and other written materials related to the case study. Documents can be collected from the case study participants or from public sources.

Surveys involve asking a set of questions to a sample of individuals relevant to the case study. Surveys can be administered in person, over the phone, through mail or email, or online. Surveys can be used to gather information on attitudes, opinions, or behaviors related to the case study.

Artifacts are physical objects relevant to the case study. Artifacts can include tools, equipment, products, or other objects that provide insights into the case study phenomenon.

How to conduct Case Study Research

Conducting a case study research involves several steps that need to be followed to ensure the quality and rigor of the study. Here are the steps to conduct case study research:

• Define the research questions: The first step in conducting a case study research is to define the research questions. The research questions should be specific, measurable, and relevant to the case study phenomenon under investigation.
• Select the case: The next step is to select the case or cases to be studied. The case should be relevant to the research questions and should provide rich and diverse data that can be used to answer the research questions.
• Collect data: Data can be collected using various methods, such as interviews, observations, documents, surveys, and artifacts. The data collection method should be selected based on the research questions and the nature of the case study phenomenon.
• Analyze the data: The data collected from the case study should be analyzed using various techniques, such as content analysis, thematic analysis, or grounded theory. The analysis should be guided by the research questions and should aim to provide insights and conclusions relevant to the research questions.
• Draw conclusions: The conclusions drawn from the case study should be based on the data analysis and should be relevant to the research questions. The conclusions should be supported by evidence and should be clearly stated.
• Validate the findings: The findings of the case study should be validated by reviewing the data and the analysis with participants or other experts in the field. This helps to ensure the validity and reliability of the findings.
• Write the report: The final step is to write the report of the case study research. The report should provide a clear description of the case study phenomenon, the research questions, the data collection methods, the data analysis, the findings, and the conclusions. The report should be written in a clear and concise manner and should follow the guidelines for academic writing.

Examples of Case Study

Here are some examples of case study research:

• The Hawthorne Studies : Conducted between 1924 and 1932, the Hawthorne Studies were a series of case studies conducted by Elton Mayo and his colleagues to examine the impact of work environment on employee productivity. The studies were conducted at the Hawthorne Works plant of the Western Electric Company in Chicago and included interviews, observations, and experiments.
• The Stanford Prison Experiment: Conducted in 1971, the Stanford Prison Experiment was a case study conducted by Philip Zimbardo to examine the psychological effects of power and authority. The study involved simulating a prison environment and assigning participants to the role of guards or prisoners. The study was controversial due to the ethical issues it raised.
• The Challenger Disaster: The Challenger Disaster was a case study conducted to examine the causes of the Space Shuttle Challenger explosion in 1986. The study included interviews, observations, and analysis of data to identify the technical, organizational, and cultural factors that contributed to the disaster.
• The Enron Scandal: The Enron Scandal was a case study conducted to examine the causes of the Enron Corporation’s bankruptcy in 2001. The study included interviews, analysis of financial data, and review of documents to identify the accounting practices, corporate culture, and ethical issues that led to the company’s downfall.
• The Fukushima Nuclear Disaster : The Fukushima Nuclear Disaster was a case study conducted to examine the causes of the nuclear accident that occurred at the Fukushima Daiichi Nuclear Power Plant in Japan in 2011. The study included interviews, analysis of data, and review of documents to identify the technical, organizational, and cultural factors that contributed to the disaster.

Application of Case Study

Case studies have a wide range of applications across various fields and industries. Here are some examples:

Business and Management

Case studies are widely used in business and management to examine real-life situations and develop problem-solving skills. Case studies can help students and professionals to develop a deep understanding of business concepts, theories, and best practices.

Case studies are used in healthcare to examine patient care, treatment options, and outcomes. Case studies can help healthcare professionals to develop critical thinking skills, diagnose complex medical conditions, and develop effective treatment plans.

Case studies are used in education to examine teaching and learning practices. Case studies can help educators to develop effective teaching strategies, evaluate student progress, and identify areas for improvement.

Social Sciences

Case studies are widely used in social sciences to examine human behavior, social phenomena, and cultural practices. Case studies can help researchers to develop theories, test hypotheses, and gain insights into complex social issues.

Law and Ethics

Case studies are used in law and ethics to examine legal and ethical dilemmas. Case studies can help lawyers, policymakers, and ethical professionals to develop critical thinking skills, analyze complex cases, and make informed decisions.

Purpose of Case Study

The purpose of a case study is to provide a detailed analysis of a specific phenomenon, issue, or problem in its real-life context. A case study is a qualitative research method that involves the in-depth exploration and analysis of a particular case, which can be an individual, group, organization, event, or community.

The primary purpose of a case study is to generate a comprehensive and nuanced understanding of the case, including its history, context, and dynamics. Case studies can help researchers to identify and examine the underlying factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and detailed understanding of the case, which can inform future research, practice, or policy.

Case studies can also serve other purposes, including:

• Illustrating a theory or concept: Case studies can be used to illustrate and explain theoretical concepts and frameworks, providing concrete examples of how they can be applied in real-life situations.
• Developing hypotheses: Case studies can help to generate hypotheses about the causal relationships between different factors and outcomes, which can be tested through further research.
• Providing insight into complex issues: Case studies can provide insights into complex and multifaceted issues, which may be difficult to understand through other research methods.
• Informing practice or policy: Case studies can be used to inform practice or policy by identifying best practices, lessons learned, or areas for improvement.

Advantages of Case Study Research

There are several advantages of case study research, including:

• In-depth exploration: Case study research allows for a detailed exploration and analysis of a specific phenomenon, issue, or problem in its real-life context. This can provide a comprehensive understanding of the case and its dynamics, which may not be possible through other research methods.
• Rich data: Case study research can generate rich and detailed data, including qualitative data such as interviews, observations, and documents. This can provide a nuanced understanding of the case and its complexity.
• Holistic perspective: Case study research allows for a holistic perspective of the case, taking into account the various factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and comprehensive understanding of the case.
• Theory development: Case study research can help to develop and refine theories and concepts by providing empirical evidence and concrete examples of how they can be applied in real-life situations.
• Practical application: Case study research can inform practice or policy by identifying best practices, lessons learned, or areas for improvement.
• Contextualization: Case study research takes into account the specific context in which the case is situated, which can help to understand how the case is influenced by the social, cultural, and historical factors of its environment.

Limitations of Case Study Research

There are several limitations of case study research, including:

• Limited generalizability : Case studies are typically focused on a single case or a small number of cases, which limits the generalizability of the findings. The unique characteristics of the case may not be applicable to other contexts or populations, which may limit the external validity of the research.
• Biased sampling: Case studies may rely on purposive or convenience sampling, which can introduce bias into the sample selection process. This may limit the representativeness of the sample and the generalizability of the findings.
• Subjectivity: Case studies rely on the interpretation of the researcher, which can introduce subjectivity into the analysis. The researcher’s own biases, assumptions, and perspectives may influence the findings, which may limit the objectivity of the research.
• Limited control: Case studies are typically conducted in naturalistic settings, which limits the control that the researcher has over the environment and the variables being studied. This may limit the ability to establish causal relationships between variables.
• Time-consuming: Case studies can be time-consuming to conduct, as they typically involve a detailed exploration and analysis of a specific case. This may limit the feasibility of conducting multiple case studies or conducting case studies in a timely manner.
• Resource-intensive: Case studies may require significant resources, including time, funding, and expertise. This may limit the ability of researchers to conduct case studies in resource-constrained settings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Survey Research – Types, Methods, Examples

No internet connection.

All search filters on the page have been cleared., your search has been saved..

• All content
• Dictionaries
• Encyclopedias
• Expert Insights
• Foundations
• How-to Guides
• Journal Articles
• Little Blue Books
• Little Green Books
• Project Planner
• Tools Directory
• Sign in to my profile My Profile

• Sign in Signed in
• My profile My Profile

The Application of Concurrent or Sequential Mixed-Methods Research Designs and Their Methodological Implications: Investigating Tacit Knowledge, Its Use, and Application in Automotive Development

• By: Robin Bell , Vessela Warren & René Schmidt
• Product: Sage Research Methods Cases Part 1
• Publisher: SAGE Publications, Ltd.
• Publication year: 2022
• Online pub date: March 22, 2022
• Discipline: Business and Management
• Methods: Mixed methods , Research design , Research questions
• DOI: https:// doi. org/10.4135/9781529604474
• Keywords: product development , tacit knowledge Show all Show less
• Online ISBN: 9781529604474 Copyright: © SAGE Publications Ltd 2022 More information Less information

Mixed-methods research designs are a popular approach to research that can offer the advantages of both quantitative and qualitative data. However, using two types of data within one piece of research also presents challenges. To minimize these challenges and maximize the potential benefits, it is important for mixed-methods research projects to be well planned and thought out. This case study explores and discusses the use of concurrent and sequential mixed-methods research designs and the implications on the data collection and analysis processes, as well as the underpinning research philosophy. Specifically, this case study examines the use of a mixed-method research design in a study exploring the forms of tacit knowledge requested of employees in the automotive industry and how and why these forms of tacit knowledge are used. Understanding available options for data collection and their influence on the research process is essential to making an informed decision about whether to follow a concurrent or sequential mixed-methods research design.

Learning Outcomes

By the end of this case study, students should be able to:

• Understand the benefits and challenges of undertaking mixed-methods research
• Explain the difference between concurrent and sequential mixed-method research designs
• Assess the methodological implications of applying a concurrent or sequential mixed-method research design
• Critically evaluate whether a concurrent or sequential mixed-method research design is most appropriate for a given research problem
• Critically justify the suitability of a concurrent or sequential mixed-method research design for achieving a research aim

Project Overview and Context

The research sought to identify what forms of tacit knowledge are requested of employees working in product development within a multinational automotive company, and how and why these forms of tacit knowledge are used in the product development process, by using a sequential mixed-methods approach. A greater focus and emphasis has been placed on researching explicit knowledge within the knowledge literature compared to tacit knowledge ( Leonard & Sensiper, 1998 ). This is despite the fact that tacit knowledge has been widely accepted and acknowledged to be essential for sustainable business success ( Jisr & Maamari, 2017 ). This in part stems from tacit knowledge being based on an individual’s intuition, beliefs, and insight, which come from their own experience ( Polanyi, 1966 ), making the verbalization and identification of tacit knowledge challenging ( Fleck, 2014 ; Seidler‐de Alwis & Hartmann, 2008 ). The research was considered important as the codification of tacit knowledge has been suggested as a potential way for Western automotive manufacturers to remain competitive in the face of growing global competition ( Machacek & Hess, 2019 ).

Section Summary

• Tacit knowledge is an under-researched area.
• Eliciting and verbalizing tacit knowledge in research can be tricky.
• The codification of tacit knowledge is important for automotive manufacturers to remain globally competitive.

Mixed Methods Research Designs

To achieve the aim of the research in determining what types of tacit knowledge were most requested and how and why these were used in the automotive product development process, we determined that a mixed-methods approach was appropriate for the research. We determined that quantitative or qualitative research methods alone could not achieve the quantification of the types of tacit knowledge requested by the organization, while also achieving the depth of understanding as to how and why these forms of tacit knowledge were utilized and applied in the product development process ( Schmidt et al., 2021 ). Quantitative data are an effective tool for seeking to measure a phenomenon and provide objectivity, which aligns well with the first part of the research determining the “what types of tacit knowledge” were most commonly requested but lacks the ability to explore and provide a detailed understanding as to “how and why” these types of tacit knowledge are used in the context being researched. Qualitative research, however, can provide detail, richness, and sensitivity to context ( Tharenou et al., 2007 ), providing a more effective tool to understand the use and application of the tacit knowledge in practice. These considerations led to a mixed-methods research design being employed, where both quantitative and qualitative research were combined in a single research design ( Saunders et al., 2019 ).

Debate exists in the research-methods literature as to the value of mixed-methods research designs, with proponents of mixed-methods arguing that using both quantitative and qualitative data can potentially increase the validity, generalizability, and depth of the results. While sceptics suggest contradictions between the underpinning philosophy and methods regularly arise with mixed-methods research designs ( Easterby-Smith et al., 2018 ). This can lead to a lack of clarity and focus within research, with good intentions of including more data becoming detrimental, and as such, projects employing mixed-methods becoming “as clear as mud” ( Lowe, 2010 ). However, proponents of mixed-methods research posit bringing different types of data together can strengthen research by offering multiple perspectives that can increase validity and generalizability.

When proceeding with a mixed-methods research design, there are two important considerations to ensure the quality and fit of the research design proposed in achieving the research aim. These are the dominance of the methods within the research design and the sequencing of the methods ( Easterby-Smith et al., 2018 ). The dominance of the methods relates to whether objectivity or subjectivity is more important in the research, whereas the sequencing relates to whether one data collection method precedes the other, and if so, which one. A myriad of different mixed-methods research designs exist in the literature, for example, Tashakkori and Teddlie (2003) outline and discuss around 35 different potential mixed-methods designs with varying levels of complexity. This case study does not seek to explore all the different designs, but rather focus on the sequencing of the methods within mixed-methods research and the implications for the underpinning methodology and philosophy.

• Mixing methods in a research design can add additional depth to a research project.
• Clarity is needed as to how and why the data collected with mixed methods will be applied and analysed to achieve the research aim.

Research Practicalities

Tacit knowledge is a challenging phenomenon to research given its tacit nature, which means that it cannot be easily communicated as it is implicit or implied. To overcome this challenge, a previously tested and validated framework was employed in this work. The framework adopted was by Haldin-Herrgard (2003) , and it identifies 92 epitomes of tacit knowledge. These epitomes are vocabulary that are used to refer to tacit knowledge and, therefore, allow the verbalization of tacit knowledge in conversation or the written form. In order to identify what forms of tacit knowledge were requested in the automotive organization being researched, a sample of job role descriptions included within job adverts were screened to identify how frequently each epitome was used. This allowed for the measuring and quantification of the forms of tacit knowledge being requested of the workforce. This was a time-consuming process, and conceptual equivalency had to be ensured as the language of the original epitomes and the job adverts were different. However, it would likely have been challenging for interviewees to identify the forms of tacit knowledge they used given the implicit nature of tacit knowledge.

Having identified which of the epitomes and thus the forms of tacit knowledge that were requested most in the job adverts, the second stage of the research sought to understand in more detail how and why these forms of tacit knowledge were used in the specific context of the organization being researched. Interviews were chosen to collect subjective accounts of how and why forms of tacit knowledge were used by individuals in their job role. The interview questions were built around the most frequently requested form of tacit knowledge identified in the first stage of the research. This helped to bound and structure the interviews, as otherwise, the discussion of tacit knowledge could become very wide and hard to focus and analyse. The discussion of epitomes of tacit knowledge helped to guide the interview discussion and make discussing tacit knowledge less challenging. Structuring the interviewees around specific epitomes helped interviewees talk more explicitly.

The research was of a potentially sensitive nature as tacit knowledge used within organizations can help to achieve a business and competitive advantage. In addition, employees may feel reluctant to discuss their work and interactions with colleagues if the information was openly disclosed. As a result, the organization and interviewees all gave informed consent to be part of the research, were assured of confidentiality, and were briefed on the research and its purpose before the data collection process started. The data collected were saved in a secure format and location and the job descriptions and interview participants were all anonymized within the research.

• Job role descriptions were used to identify the forms of tacit knowledge most frequently requested.
• Interviews were chosen to allow participants to discuss how they used the most frequently requested forms of tacit knowledge in their job role.
• Epitomes of tacit knowledge were used to overcome the challenge of discussing tacit knowledge which is implicit rather than explicit.

Sequencing of Methods Within Mixed-Methods Research Design in Action

Mixed-methods research designs can either follow a concurrent or sequential data collection process. This is a decision that the researcher should make based on achieving their research aim, answering their research questions and the purpose of mixing methods. The decision made has far-reaching implications for the research project as it influences and affects the methodological and philosophical underpinnings of the research, the data collection and analysis process, and the presentation of the research. The difference between concurrent and sequential mixed-methods research designs will now be discussed, along with their benefits and suitability in practice.

Concurrent mixed-methods research design involves collecting both quantitative and qualitative data at roughly the same time. Therefore, the collection of quantitative and qualitative data are independent from each other, and one type of data are not informing the collection of the other type of data. This has obvious time benefits as both types of data are collected at the same time, potentially allowing the research to complete the data collection sooner. An example of such an approach would be to collect both quantitative data through Likert scales and qualitative data through open questions within a single questionnaire about a potential new product at the same time. Concurrent designs are effective when seeking to triangulate data to determine and demonstrate congruence between both quantitative and qualitative findings. Triangulation can be achieved by identifying whether the two types of data support the same results and conclusions. Triangulation can be used to reduce the risk of chance associations and bias, as studies using only one method are more vulnerable to errors ( Wilson, 2014 ). Research that utilizes different types of data can provide cross data validity checks and thus can help to demonstrate validity, and potentially extend generalizability. Concurrent mixed-methods designs can be particularly efficacious for research focused on evaluation, where multiple data points and streams can be reviewed to come to an overall result and conclusion. For example, Bell (2016) used a concurrent mixed-methods research design to evaluate the effectiveness of research methods training, by analyzing both quantitative grades and qualitative accounts of researcher’s preparedness to conduct research, to come to an overall assessment.

However, with all mixed-method research designs, it is important to consider how the data will be collected and how it will be integrated into the data process to inform the results to avoid a situation where data are collected and there is a lack of clarity regarding its value and purpose.

Sequential mixed-methods research designs have two distinct stages of data collection which occur one after the other. One of the benefits of such an approach is that it allows time for the first set of data to be analyzed and can, therefore, inform the later stage. The researcher can make the decision whether the first stage will involve quantitative or qualitative research, depending on the aim of the research project. If the project seeks to quantify or test a concept and then explore it further, it is likely that first stage will be quantitative, and the second stage will be qualitative. While, if the project seeks to explore a phenomenon to develop an understanding before testing or quantifying it, it is likely that the first stage will involve qualitative data followed by the collection of quantitative data. Such research designs are particularly useful if the research seeks to use the results of the first stage to inform the second stage of data collection. Therefore, if the purpose of adopting mixed methods is to add additional depth to support knowledge and theory development, sequential methods can be well suited to the task. Concurrent mixed-methods research designs do not allow for the evolution of the topic and understanding through the research process, which limits the potential development of knowledge and theory during the research process. In the context of researching what forms of tacit knowledge are requested of employees working in product development within a multinational automotive company and “how and why” these forms of tacit knowledge are used in the product development process, there is a clear need to first determine what forms of tacit knowledge are requested of employees before exploring the application of the tacit knowledge further. This led the research to be undertaken in two stages, thus adopting a sequential mixed-methods approach. The first stage sought to quantify the most frequently requested forms of tacit knowledge. This was achieved through employing manifest content analysis to internal job descriptions to highlight the epitomes of tacit knowledge to identify the most frequently requested forms of tacit knowledge. Then, the second stage of the research focused specifically on how and why the most frequently requested forms of tacit knowledge are applied in practice. This deeper exploration was conducted using semi-structured interviews to collect qualitative data to support the collection of richer and more context-specific data.

It is worth noting that in terms of triangulation, sequential designs are not as effective for triangulating data as the findings from the first stage might influence the second stage, or the phenomenon and its context might have changed between the two phases of data collection, thus influencing the findings ( Onwuegbuzie & Collins, 2007 ).

In the process of developing an appropriate mixed-methods research design, it is important that the researcher reviews the aim of their research to assess whether a concurrent or sequential mixed-methods research design is more appropriate. Both concurrent and sequential mixed-methods research designs have strengths and weaknesses, and it is important that the strengths are aligned with what the research seeks to achieve. The decision as to whether a concurrent or sequential mixed-methods research design is chosen has methodological and philosophical implications, which will now be explored in further detail.

• Consideration should be given to the sequencing of mixed methods in a mixed-methods research design.
• Concurrent mixed-methods research designs involve collecting both quantitative and qualitative data at roughly the same time.
• Sequential mixed-methods research designs collect one form of data first, which informs the following stage of data collection.
• Concurrent mixed-methods research designs can be particularly efficacious in evaluation research.

Methodological Implications of Concurrent and Sequential Mixed Methods Research Designs

Data collection decisions have implications for the research philosophy underpinning a research project. The decision as to whether to employ a concurrent or sequential mixed-methods research design also has methodological and philosophical implications, which will now be discussed in this section. Different approaches to undertaking research have implications for the underpinning research methodology and philosophy, as different approaches can be more effectively supported by different research philosophies and philosophical standpoints. Positivism and interpretivism are often considered to be the mainstream opposing philosophical standpoints, which align with the ends of the philosophical spectrum. Positivism applies principles of natural science and emphasizes highly structured data collection to facilitate replication to support law like generalizations ( Saunders et al., 2019 ). Positivism posits the use of objective and structured data to break the world around us into testable components, seeking to find generalizable casual relationships. This view of the world, or paradigm, appreciates the objectivity and structured nature of quantitative data, however, fails to readily recognize the value of qualitative data, as this is often unstructured and challenging to test to produce law link generalizations. Therefore, positivism does not readily support the use of mixed methods as it fails to recognize the value that qualitative data can bring to research. Interpretivism asserts that natural science assumptions and methods of research are not appropriate for social investigation, as the social world is not governed by regularities that hold law-like properties ( Ritchie & Lewis, 2003 ). Researchers need to understand the context and subjective reality of participants to understand the meaning behind their views and answers ( Saunders et al., 2019 ). Therefore, the interpretivist philosophy does not value such structured data, but rather seeks depth within datasets to elicit subjectivity in relation to context and specific situations. This means that the interpretivist view of the world does not value quantitative data, as it commonly fails to offer the situational context and depth to understand subjectivity. Interpretivism advocates qualitative data and does not readily acknowledge the value of including quantitative data in a mixed-methods research design. Rather than seeking to apply a research philosophy that does not value part of the data within the research design, other research philosophies are more sympathetic to the use of mixed-methods research. Two such research philosophies are pragmatism and realism, which will now be discussed in more detail.

The pragmatist research philosophy posits that it possible to work with variations in your epistemology, ontology, and axiology ( Saunders et al., 2019 ). Pragmatists seek to use data that fits the research aim and problem, and as such, it is a suitable philosophy for those who seek to promote the combining of methods ( Morgan, 2007 ). Pragmatism has three central tenets that are an emphasis on actionable knowledge, acknowledgement of the linkage to experience, and research being an experiential process ( Kelly & Cordeiro, 2020 ). A pragmatic standpoint encourages bringing data together from multiple data points to support researchers to come to actionable results and conclusions, through recognizing research participants experience. This makes pragmatism a particularly useful philosophy to underpin research that evaluates and assesses performance within an organization or process ( Kelly & Cordeiro, 2020 ), which aligns well with the potential application benefits of concurrent missed methods research. Pragmatism has commonly been employed within research seeking practical and actional benefits, while not considering its philosophical aspects ( Morgan, 2014 ). Therefore, it is important to link the use of pragmatism to more than solely seeking practical benefits and conclusions, which are a common goal and expectation within business and management research.

Realism is like pragmatism in its willingness to accept and advocate the use of mixed-methods research designs. However, pragmatism and realism have different starting and end points ( Allmark & Machaczek, 2018 ), as realism seeks to understand the world through an external and objective reality and how this influences people’s social interpretations and behaviors, which might not be perceptible to participants ( Saunders et al., 2019 ). This encourages realists to try and seek some degree of objectivity while acknowledging the needs and benefits of qualitative research to explore social interactions, interpretations, and behaviors, which need an element of subjective questioning and probing to elicit. Such an underpinning to research acknowledges the distinctive value and contribution of quantitative methods to provide objectivity and generalizability and qualitative methods to provide depth and context to the results. This makes realism particularly suitable to underpin sequential mixed-methods research where each stage of the research has a specific purpose within the overall research design.

When researching the forms of tacit knowledge requested of employees and “how and why” these forms of tacit knowledge are used, it was decided that the most effective underpinning research philosophy was realism. It was suitable as the research was seeking a degree of objectivity in the first stage to understand and quantify the tacit knowledge that was requested: While the second stage of the research employed the collection of qualitative data as the research recognized the need for subjective data collected through semi-structured interviews to explore participants behaviors and work interactions. Therefore, overall while the research sought some degree of objectivity, it still recognized the need to collect employee qualitative data for the component of the research where it was more effective.

• The decision as to whether to use concurrent or sequential mixed methods has potential methodological implications.
• Both the realist and pragmatic research philosophies value the use and application of mixed methods.

Practical Lessons Learned

While mixed-methods research designs can be attractive due to the collection of more data on a particular research phenomenon, they are not a panacea. More data are not always better, particularly if it does not have a clear purpose and adds little value to the overall project. Issues can surface around the quality of the data collection and analysis as the researcher must become an expert in both quantitative and qualitative methods, providing the reader thorough details of the data collection, analysis process, and the results in the same word limit, while achieving the level of rigor to satisfy both quantitative and qualitative researchers in the marking and peer-review process ( Lowe, 2010 ). It has been argued that the differentiation of quantitative and qualitative research methods is a crude dichotomy and omits many potentially useful possibilities ( Wood & Welch, 2010 ), and researchers should, therefore, not become too preoccupied with the methods used and how they are classified, but rather focus on what data are available to them and how it can be used to achieve a research aim and answer the research questions.

In the case of our research, time was given to determine what type of data would be most effectively able to answer each of the research questions. An existing framework has already been developed to identify tacit knowledge through 92 so-called epitomes of tacit knowledge, which are pieces of vocabulary that can be used to identify tacit knowledge ( Haldin-Herrgard, 2003 ). Adopting a sequential mixed-methods approach, a quantitative analysis of job descriptions enabled the research to first identify the main groupings of tacit knowledge in a specific department. This was a necessary first stage to identify the most important types of tacit knowledge requested in our specific context. The research then adopted a qualitative approach to elicit finer detail to answer why it was important through interviews and thematic analysis. The initial quantitative stage allowed the research to focus down on the main groupings and the subsequent qualitative stage to focus on the smaller individual details. This sequential mixed-method approach led to results that may be generalizable to similar businesses where essential tacit knowledge is under threat due to the streamlining of operations to maintain competitiveness in the marketplace. Deciding how the data will be used in advance of data collection saved a significant amount of time and provided operational clarity. Undertaking a sequential mixed-methods approach also allowed the data collection and analysis to be broken down into chunks, as the first stage involving the quantitative data collection and analysis was completed first, before moving onto the qualitative second stage. Having decided upon adopting a sequential mixed-methods research design, we needed to be reflexive to consider what the methodological and philosophical implications would be. We decided that a realist philosophy was most appropriate as the first stage sought to measure and identify objectively the forms of tacit knowledge request, but we still valued the depth and context that qualitative data could provide to understand how and why these forms of tacit knowledge are used in the automotive industry.

What becomes apparent from the experience of adopting mixed methods is the importance and need to think carefully about the value each method offers and how it will be integrated into the overall research design in order to achieve the research aim. This should be the initial focal point when considering developing a mixed-methods research project. Decisions regarding the value, purpose, and contribution of each method will help to inform decisions as to whether a concurrent or sequential mixed-methods research design is most suitable. In this research case, a sequential mixed-method design was most suitable with the quantitative stage being undertaken first and the second qualitative stage following on and building on the findings of the first stage. These decisions will inform the most suitable research philosophy to underpin the research.

• More data are not always better unless there is a clear understanding as to how it can be used to add value to a research project.
• How the different types of data collected will be used should be clearly thought out and planned before data collection to save time later.
• The value, purpose, and contribution of each method should help to inform the decision whether to follow a concurrent or sequential mixed methods research design.

Conclusions

Mixed-methods research designs have the potential to be a powerful research design. However, they might not be an easy option. The effectiveness of a piece of research is not solely assessed on the amount of data or the number of data types. A range of additional points is commonly considered when assessing the quality and effectiveness of a research project including, the quality of the data, the depth of analysis, the relevance of the sample, the validity and rigor checks put in place, and the suitability of the data analysis in achieving the research aim and answering the research questions. All these points need to be taken into consideration when developing a mixed-methods research project.

Before making a concrete decision to move forward with collecting mixed-methods data, careful consideration needs to be given as to how the data collection will be analyzed to achieve the research aim and research questions set. In the case of our research, we used the first stage of data collection to inform the second stage of data collection, so adopting a sequential mixed-methods design was most appropriate. This influenced the research process as the quantitative data had to be collected and analyzed before the second stage of data collection could be undertaken. While this might have been more time consuming, as the data were collected in a stop start nature, it helped to break the research project into manageable chunks.

When evaluating the suitability of a mixed-methods research design, it is important to carefully think about the data collection and analysis process and the implications for the underpinning research methodology and philosophy. This is a reflexive process where the researcher needs to ensure alignment between the most effective methods and research design and the most appropriate research philosophy to underpin this.

In summary, this case study explores and discusses the importance of deciding on whether a concurrent or sequential mixed-methods research design will be adopted. This decision should be made relatively early in the research process before data are collected as it will have implications on the data collection and analysis process, as well as the underpinning research philosophy. Understanding the options available and their influence on the research process is essential to make an informed decision.

• Mixed methods research designs are not an easy option.
• The quality and effectiveness of a piece of research is not commonly evaluated by the number of methods and data types used.
• When assessing the suitability of a mixed-methods research design, it is important to consider the data collection and analysis process and the implications for the underpinning research methodology and philosophy.

Classroom Discussion Questions

• 1. Having read this case study how have your views toward mixed methods research changed?
• 2. What are the methodological and research process implications for adopting a concurrent or a sequential mixed-methods research design?
• 3. Why is a concurrent mixed-methods research design commonly seen as advantageous for triangulation?
• 4. Identify a research aim which you think would be suitable to be achieved through the application of either concurrent or sequential mixed-methods research designs and justify why.

Multiple Choice Quiz Questions

1. Which of the following best describes a sequential mixed-methods research design?

Incorrect Answer

Feedback: This is not the correct answer. The correct answer is B.

Correct Answer

Feedback: Well done, correct answer.

2. What should drive the use of a mixed-methods research design?

Feedback: This is not the correct answer. The correct answer is A.

3. Which type of mixed methods is most suitable for triangulation?

Further Reading

Sign in to access this content, get a 30 day free trial, more like this, sage recommends.

We found other relevant content for you on other Sage platforms.

Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches

• Sign in/register

Navigating away from this page will delete your results

Please save your results to "My Self-Assessments" in your profile before navigating away from this page.

Sign in to my profile

Sign up for a free trial and experience all Sage Learning Resources have to offer.

You must have a valid academic email address to sign up.

Get off-campus access

• View or download all content my institution has access to.

Sign up for a free trial and experience all Sage Research Methods has to offer.

• view my profile
• view my lists

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Taking a critical stance towards mixed methods research: A cross-disciplinary qualitative secondary analysis of researchers’ views

Sergi Fàbregues

1 Department of Psychology and Education, Universitat Oberta de Catalunya, Barcelona, Spain

Elsa Lucia Escalante-Barrios

2 Department of Education, Universidad del Norte, Barranquilla, Colombia

José Francisco Molina-Azorin

3 Department of Management, Universidad de Alicante, Alicante, Spain

Quan Nha Hong

4 Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), University College London, London, United Kingdom

Joan Miquel Verd

5 Centre d’Estudis Sociologics sobre la Vida Quotidiana i el Treball (Sociological Research Centre on Everyday Life and Work—QUIT), Universitat Autonoma de Barcelona, Barcelona, Spain

Associated Data

Data cannot be shared publicly as participants did not give consent for their transcripts to be shared in this manner. Since the consent statement approved by the Institutional Review Board (IRB) of the Universitat Autònoma de Barcelona, and signed by the participants, did not include the provision that data would be made publicly available, we do not have participant consent to share this data. Also, the content is sensitive, and participants could be identified. Requests for anonymized data can be made to the Principal Investigator of the study, Sergi Fàbregues ( ude.cou@fseugerbafs ) or the Institutional Review Board (IRB) of the Universitat Autònoma de Barcelona ( tac.bau@haeec ).

Recent growth and institutionalization in the field of mixed methods research has provided fertile ground for a wide range of thoughtful criticism of how this research approach has been developed and conceptualized by some members of the mixed methods community. This criticism reflects the increasing maturity of the field as well as the different theoretical perspectives and methodological practices of researchers in different disciplines. While debates addressing these criticisms are likely to lead to valuable insights, no empirical studies have been carried out to date that have investigated researchers’ critical views on the development and conceptualization of mixed methods research. This study examines the criticisms of the mixed methods field raised by a cross-national sample of researchers in education, nursing, psychology, and sociology. We carried out a secondary analysis of semi-structured interviews with 42 researchers and identified 11 different criticisms, which we classified in four domains: essence of mixed methods, philosophy, procedures, and politics. The criticisms related to the procedures domain were equally distributed among the four disciplines, while those related to the essence, philosophy and politics domains were more common among sociologists. Based on our findings, we argue that the divergence of views on foundational issues in this field reflects researchers’ affiliation to different communities of practice, each having its own principles, values, and interests. We suggest that a greater awareness of this divergence of perspectives could help researchers establish effective collaboration and anticipate potential challenges when working with researchers having different methodological approaches.

Introduction

Since the 1990s, mixed methods research as a distinct methodology has seen vigorous institutionalization [ 1 , 2 ] with the launch of a specialized journal ( Journal of Mixed Methods Research ) in 2007, the establishment of the Mixed Methods International Research Association (MMIRA) in 2013, and the publication of a stream of handbooks, textbooks, and articles on the philosophy and practice of mixed methods. Mixed methods research is increasingly used in a wide range of disciplines, particularly in the social, educational, behavioral and health sciences [ 3 – 6 ]. Several scholars have argued that mixed methods inquiry can help researchers arrive at a more complete understanding of research problems, develop more robust quantitative instruments, and integrate several worldviews in a single research study [ 7 , 8 ].

A clear indication of the institutionalization of mixed methods research as a field is the expansion of the mixed methods community. Tashakkori, Johnson and Teddlie [ 9 ] have characterized this community as a group of scholars who share similar backgrounds, methodological orientations, philosophical assumptions, and views on research and practice. As distinct from researchers using only qualitative or quantitative approaches, these scholars often share similarities in training, research background, and professional affiliation. Those authors’ view of the mixed methods community is consistent with Thomas Kuhn’s preferred definition of paradigms, as cited by Morgan [ 10 ]. In disagreement with the view of paradigms as incompatible epistemological stances, Morgan [ 10 ], following Kuhn [ 11 ], advanced a more integrative notion of paradigms focused on shared beliefs and joint actions in a community of researchers. Denscombe [ 12 ] took this perspective one step further by proposing the notion of communities of practice , an idea originally developed by the educationalist Étienne Wenger. This notion brings us closer to a definition of paradigms as shared beliefs able to accommodate the diversity of perspectives and approaches that currently exists within the mixed methods community [ 12 , 13 ].

While the institutionalization of mixed methods as a field has helped to formalize and clarify research practices [ 14 ], this trend has also led to some criticism of the ways in which this approach has been developed and conceptualized by some members of the mixed methods community [ 15 , 16 ]. The critiques of the mixed methods field have already been summarized in two overviews published in the early 2010s. In the second edition of the Handbook , Tashakkori and Teddlie [ 17 ] outlined four frequent criticisms raised in the mixed methods literature, including overreliance on typologies and the higher status of quantitative versus qualitative research. One year later, Creswell [ 18 ] described some of the same criticisms in a summary of 11 key controversies. The emergence of this criticism testifies to the increasing maturity of the field and its progress towards what Creswell and Plano Clark [ 7 ] define as the period of reflection and refinement in mixed methods research. In their view, the mixed methods community should be honored that it has attracted critical attention and it should stimulate debate around the issues raised. Therefore, it is crucially important to address these criticisms in greater detail because such engagement will lead to valuable insights that could lay the basis for further discussion needed to ensure the healthy development of the field. Furthermore, addressing those criticisms is essential to enhance researchers’ understanding of the complexity of the mixed methods field and to provide them with the awareness needed to deal with tensions that might emerge when working in teams with researchers subscribing to different methodological viewpoints [ 19 , 20 ].

Most of the criticisms of mixed methods so far have been formulated in the literature by several scholars specialized in theoretical aspects of methodology. However, it would be useful to find out whether other researchers share these criticisms and whether they may have formulated others. Scholars with an interest in mixed methods research come from different academic disciplines that embody different theoretical and methodological perspectives. As Plano Clark and Ivankova [ 3 ] argue, these differences are highly likely to influence the ways in which those scholars view mixed methods as well as the questions they might raise regarding current ideas in the field. Therefore, it would be useful to examine the ways in which researchers’ criticisms differ according to discipline.

The aim of the present study is to examine the criticisms of the mixed methods field raised by a cross-national sample of 42 researchers working in the disciplines of education, nursing, psychology, and sociology. We report a secondary analysis of semi-structured interviews originally conducted to describe how researchers operationalize and conceptualize the quality of mixed methods research [ 14 ]. The contribution of this article is twofold: (a) it is, to our knowledge, the first study based on an empirical approach to examining researchers’ critical views on the development and conceptualization of mixed methods research, and (b) it enhances our understanding of the ways in which these critical views may be associated with different academic disciplines.

This article reports a secondary analysis of data originally collected in a multiple-case study of the quality of mixed methods based on semi-structured interviews with researchers in the disciplines of education, nursing, psychology, and sociology. In line with Heaton’s [ 21 ] definition of secondary analysis — which he calls supplementary analysis — as an “in-depth investigation of an emergent issue or aspect of the data which was not considered or fully addressed in the primary study” [ 21 ], in this article we re-analyze the original interview data in order to address the following two research questions (RQs): (RQ1) What criticisms of the mixed methods field are made by researchers in education, nursing, psychology, and sociology? and (RQ2) What differences and similarities can be identified in the criticisms reported by researchers working in different disciplines? In the following subsections, we provide a brief description of the sampling and data collection methods used in the original study, and of the procedures used in the secondary analysis of data. A more detailed explanation of procedures followed in the original study can be found in Fàbregues, Paré, and Meneses [ 22 ].

The original study

Sampling and recruitment of participants.

The disciplines of education, nursing, psychology, and sociology were selected for four main reasons: (1) professionals working in these disciplines contribute a relatively high proportion of mixed methods empirical articles and other methodological publications, (2) a considerable number of prevalence studies and methodological reviews on the use of mixed methods in subfields of these disciplines have been published, and (3) these disciplines are characterized by their clear disciplinary boundaries, and this characteristic offers the possibility of gaining useful comparative insights [ 6 ]. Criterion and maximum variation sampling were used to select the researchers who participated in the study [ 23 ]. In the criterion sampling, participants fulfilled two inclusion criteria: (1) they had carried out research primarily in one of the four disciplines mentioned above, and (2) they had contributed to at least one methodological publication on mixed methods research. Participant identification started with a systematic search for methodological publications on mixed methods published in English during or after 2003. Selected publications fulfilled the definitions of mixed methods suggested by either Creswell and Tashakkori [ 24 ] or Johnson, Onwuegbuzie, and Turner [ 25 ]. A number of characteristics of the first authors’ profiles were extracted, including the field of expertise, the country of affiliation, and the job title. To ensure heterogeneity of the sample, maximum variation sampling was applied to authors meeting the two inclusion criteria. An iterative approach was used to recruit 11 participants for each discipline. Sample size was based on recommendations found in the literature [ 26 ]. Potential participants were contacted using a prioritized list until a total of 44 participants had been recruited.

Data collection

Data collection involved semi-structured interviews. Questions from the interview guide were focused on the following topics: (a) participants’ research background and methodological expertise, (b) participants’ conceptualization of mixed methods research, and (c) how participants perceived the quality of the mixed methods approach in practice. Interviews were conducted using Skype, telephone and, in two cases, e-mail correspondence. Interviews were audio-recorded and transcribed, and average interview length was 49 minutes. We carried out a member-checking process by sending back to participants the transcriptions and summaries of key points of the interviews to confirm that the data accurately represented their views. At this stage, we also gave participants an opportunity to clarify or expand the statements they made during the interview.

Trustworthiness

Four strategies were used to enhance the trustworthiness of the original study. First, as explained above, all participants member-checked their transcribed interviews and summaries to confirm accuracy. Second, peer-debriefing was carried out during data collection by one researcher working together with another researcher familiar with mixed methods research who was not included in the sample. Third, an audit trail was used to record the decisions made during the study and to help researchers to reflect on the influence on the study findings of their own assumptions and disciplines. Fourth, the decisions taken during the analysis and interpretation of the data, as well as the disagreements arising during this stage, were discussed by the researchers until a consensus was reached.

The secondary data analysis

While the original study aimed to examine researchers’ views on the conceptualization and operationalization of the quality of mixed methods research, the aim of this secondary analysis of the same data is to examine researchers’ critical views of commonly accepted concepts and practices in the mixed methods field. Ethical approval for secondary data analysis was included in the ethics application for the original study, which was approved by the Institutional Review Board of the Universitat Autònoma de Barcelona. Participants signed an informed consent before the interview. Of the 44 researchers who participated in the original study, two did not consent to the subsequent use of their interview data for a secondary analysis. Therefore, the information provided by these two researchers was not used in the present study. Table 1 shows the characteristics of the 42 participants finally included in this secondary analysis.

Qualitative content analysis as described by Graneheim & Lundman [ 27 ] was used to carry out the secondary analysis of the interview data. This form of analysis is especially appropriate when, as in this study, researchers are interested in systematically describing only the topics of interest indicated by the research questions, and not in obtaining a holistic overview of all of the data [ 28 ]. The data analysis was carried out in three stages using NVivo 12 for Mac (QSR International Pty Ltd, Victoria, Australia). In the first stage, the interview transcripts were read thoroughly in order to extract the sections of text in which the participants raised criticisms of the mixed methods field. These sections of text constituted the unit of analysis. In the second stage, the extracted sections were divided into meaning units, which were subsequently condensed, abstracted and labelled with codes. Each code included a description of the meaning of the code, an indicator to identify its presence in the data, and an example of a passage coded as belonging to that code. In the third stage, the codes were compared for similarities and differences, and clustered into several categories. The underlying meaning of these categories was then examined and formulated into themes. These themes represented the study participants’ criticisms of mixed methods as a field, which were the focus of RQ1. Decisions made in this phase of the study, along with any disagreements, were discussed by the researchers until a consensus was reached.

In order to answer RQ2, a multiple correspondence analysis [ 29 – 31 ] was carried out. This technique is a non-inferential form of statistical analysis designed to analyze the multivariate association of categorical variables by generating a representation of the underlying structure of a dataset. Since the statistical requirements of multiple correspondence analysis (i.e., sampling, linearity, and normality) are highly flexible, this method is especially suited for examining qualitative interview data transformed into quantitative data [ 32 ]. The output of the multiple correspondence analysis is a scatterplot representing the spatial grouping of categories and participants. The distances between plotted points represent the degree of similarity in the patterns of participants’ responses. Multiple correspondence analysis was used to examine the relationship between the participants’ discipline and the themes relating to criticisms. To perform this analysis, we used the NVivo “matrix coding query” function to generate a matrix in which binary codes related to criticisms identified in the qualitative content analysis (the mention or failure to mention the criticism) were displayed in the columns, while the 42 participants were displayed in the rows. The matrix output was exported to XLSTAT Version 2018.1 (Addinsoft, Paris, France), which was used to perform the analysis, using the binary codes for the criticisms as active variables and the participants’ discipline as supplementary variables. Following the recommendations of Bazeley [ 32 ], after carrying out the multiple correspondence analysis, we checked the results against the qualitative data to verify the interpretation of the statistical analysis.

RQ1; What criticisms of the mixed methods field are made by researchers in education, nursing, psychology, and sociology?

Eleven criticisms of how some members of the mixed methods community have developed and conceptualized this research approach were identified in 27 of the 42 interviews included in this secondary analysis. These criticisms were then grouped into the four domains used by Creswell [ 33 ] to map the landscape of mixed methods research: (1) the essence of mixed methods research (definitions and nomenclature), (2) philosophy (philosophical assumptions and paradigmatic stances), (3) procedures ( methods and techniques for carrying out mixed methods research), and (4) politics (justification of the use of mixed methods research). Each of these four domains and the corresponding criticisms are discussed in the following sections with the support of verbatim quotes from the interviews. Table 2 shows the criticisms for each domain and the number of participants making each criticism.

Domain 1: The essence of mixed methods research

Criticism 1 : The accepted definition of mixed methods research takes into account only the mixing of both quantitative and qualitative methods . Some participants objected that the most common definition of mixed methods research that usually prevails in the literature conceives the approach as being limited to the use of quantitative and qualitative methods. These participants believe that the field should adopt a broader definition that would also include the mixing of methods within the same tradition in a single design, that is, the combination of two or more quantitative methods or qualitative methods:

“(…) for me mixed methods is not only mixing qualitative and quantitative methods , but it could also be qualitative + qualitative or quantitative + quantitative methods” (Sociologist #4).

These participants argued that the current definition of mixed methods implies that mixing the two distinct families of methods is often the only appropriate approach while in fact this definition obviates the contingent nature of research. Certain research questions might be better answered by using a combination of methods from a single tradition. Furthermore, combining two methods from the same tradition can be as valuable and as challenging as combining two methods from different traditions. One participant used the term “pressure” to describe the feeling that he was obliged to mix quantitative and qualitative methods even when this approach was not the most appropriate one:

“ … combining methods isn’t just a matter of combining quantitative and qualitative methods . You can combine different methods that are both qualitative or both quantitative and that’s , that’s valuable in it- itself , and I am worried about the kind of pressure to combine quantitative and qualitative as if that would always be appropriate” (Sociologist #2).

Criticism 2 : The terminology used in mixed methods reflects a lack of agreement among its proponents . Several participants noted the lack of clear agreement on the terminology generally used to describe the concepts and procedures that pertain to mixed methods research. They also cited a tendency to use multiple definitions for the same term and different terms to refer to similar notions. One participant cited as problematic the use of several different terms (e.g., legitimation, validity, rigor) to refer to the quality of mixed methods research:

“I would like to see a word that’s used by as many people as possible to describe that [quality]… . But , you know , I , I just think if we , everybody continues to use different terms , that could be problematic” (Educationalist #8).

According to this participant, while synonymous terms might add some precision when used to describe the complexities of implementing mixed methods research, their use can also generate confusion, especially among reviewers, editors and researchers who are trying to familiarize themselves with the field:

“…it just gets to the point where if everyone has a different definition , then how useful is that ? And that gets confusing for those who review manuscripts , or editors , when people are using in different ways that exact same term” (Educationalist #8).

Participants suggested two possible reasons for this lack of agreement. First, the tendency among some scholars to consider that mixed methods researchers should be able to use whatever terminology they may find convenient. Second, the desire of some authors to claim priority for the terminology that defines a particular method or typology. In order to resolve this lack of agreement, participants suggested that members of the mixed methods community should work towards building a greater consensus on terminology:

“There needs to be a common language” (Educationalist #9).

Criticism 3 : Mixed methods research is not a new type of methods practice . Some participants noted the tendency in the literature to present mixed methods as a new type of research practice that emerged during the past three decades. They pointed out that the use of mixed methods has a prior history that considerably predates the time when it became formalized as a research field. These participants cited examples of studies in sociology by Jahoda and Zeisel (Marienthal study of unemployment) and fieldwork in anthropology by Margaret Mead, both dating from the early 20th century. While these studies had an influence on methodology in the social sciences on account of the ways in which they creatively combined multiple quantitative and qualitative data sources, they have been generally overlooked in the mixed methods literature:

“I don’t particularly think that [mixed methods research has allowed us to answer research questions which were left unanswered in the past] but what I do think is that , you know , do remember as well that mixed methods research does actually have a long history in Sociology” (Sociologist #7).

Domain 2: Philosophy

Criticism 4 : Mixed methods research is not a third paradigm . A considerable number of participants argued against the idea of characterizing mixed methods research as a third paradigm. They found two major faults with this characterization. First, it relies on the idea of mixed methods research as an approach that is distinct from quantitative and qualitative methodologies. In the view of these participants, mixed methods approaches do not rely on singular elements that are distinct in their nature, philosophy, or procedures:

“So no , I think , ultimately , I’m probably , I’m not really convinced that is a distinct methodology (…) So I worry when , when the idea of something that’s very special about mixed methods is given a lot , is given too much primacy” (Sociologist #3). “I don’t think it’s helpful to see it as a separate approach in terms of actually conducting , you know , planning and conducting , the research…I certainly think it’s stretching it to see it as a different , as a separate paradigm…I think the whole idea of ‘paradigm’ is a little bit difficult” (Sociologist #5).

Second, the conceptualization of mixed methods as a paradigm presupposes a strong link between epistemology and method, that is, the identification of the use of mixed methods with a particular epistemological or ontological view, whereas, in fact, these are separate entities. Attaching epistemological and ontological assumptions to mixed methods research would weaken its functionality and creative potential:

“…if we restrict mixed methods to only one paradigm then we’re bottlenecking mixed methods into a certain area , and we restrict the functionality of it” (Nurse, #10).

Criticism 5 : Current discussions of mixed methods research conceive quantitative and qualitative research as separate paradigms . Related to the previous criticism, a number of participants noted that current conceptualizations of mixed methods take for granted the nature of the quantitative and qualitative approaches, conceiving them as separate paradigms based on particular philosophical assumptions, thus reinforcing the conventional divide between them and accentuating their differences:

“…the whole purpose , of course , of mixed methods is that it’s , that’s a paradigm , but I’m not convinced it is because it still draws on those conventional traditional paradigms…I find that’s likely less helpful because again it starts from the assumption that there is a strong division between qualitative and quantitative research” (Educationalist #2).

These participants stated that the mixed methods literature may have uncritically incorporated the methodological “rules” (conventions) that were dominant in the 1980s by associating qualitative research with the constructivist paradigm and quantitative research with the positivist paradigm. This linkage between philosophy and method may have been a result of the process of formalization of the methodology carried out by the “second generation of mixed methods researchers” (from 1980s to present), while the “first generation” (i.e., from the 1900s to 1980s) might not have had a philosophical problem:

“I think the biggest problem that mixed methods research is in right now is having adopted , without reflection the rules that were established in the mid 80’s on , on paradigms in quali and quanti…we have these pillars , these quali-quanti pillars and we’re working on these rooms…All these classical studies [from the first generation] had no problems in doing quali-quanti , it was only the attempt to formalize it which has actually created these , these problems” (Sociologist #6).

According to this participant, the association of the quantitative and qualitative approaches with particular epistemological stances contradicts the very nature of the mixed methods approach: if such philosophical and methodological differences between quantitative and qualitative research really existed, then the integration of the two approaches would not be possible:

“…the big problem with having adopted this [association] , on the one hand , it actually makes mixed methods impossible . So , it is not possible within one single design to argue that your da- , that there is a single and objectifiable reality out there , on the other hand , and there are multiple or no reality , there’s no reality” (Sociologist #6).

In the view of another participant, part of the mixed methods literature may have accentuated the differences between the two methodologies by representing their characteristics in different columns in a table, while ignoring the existence of methods that incorporate features of both approaches (e.g., survey containing both open and closed ended questions or qualitative studies that include descriptive statistics):

“I know what’s been recently suggested in the literature (…) I’m not even sure that I would say that we should have drawn a line between qualitative and quantitative as firmly as we have . A lot of the qualitative work that I do includes descriptive statistics” (Educationalist #7).

Criticism 6 : Superficiality of pragmatism . Some participants argued that authors in the mixed methods community sometimes characterize the notion of pragmatism in a superficial way by reducing it to merely eclecticism and confusing it with “practicalism”. In this way, these authors advocate a “what works” approach which may be useful when justifying the integration of the quantitative and qualitative methods, but this attitude distorts the nature of pragmatism by failing to consider its underlying theoretical and philosophical assumptions:

“…they tend to think that pragmatism is just the practicalities , and it’s just the technicalities…” (Educationalist #2).

One participant noted that the feebly argued debates on pragmatism to date may have led the mixed methods community to undervalue the important contribution this paradigm has made to the philosophical basis of empirical inquiry:

“I’ve probably never in my life seen such weak debates on pragmatism as I have in mix- , in the mixed methods debate . I mean if I think of this fabulous contribution that , that pragmatism as a philosophical discipline has made” (Sociologist #6).

Moreover, another participant observed that many researchers in the field have acquired their knowledge of pragmatism mainly from the descriptions of the mixed methods paradigm found in the literature, whereas a sound basis for pragmatism in mixed methods research practice would require consulting the seminal papers on pragmatism, such as those by John Dewey, Charles Sanders Pierce or William James:

“… from what I’ve read anything about pragmatism that’s in a mixed methods paper does tend to be superficial…you have to go right back to the original authors of pragmatism and I think sometimes when we speak about pragmatism in mixed methods research , students particularly ten- tend , maybe just read some articles in pragmatism and think they know about this , but I think it is important to go right back to Dewey and James and Pierce” (Nurse, #6).

Criticism 7 : Mixed methods research aligns with positivism . A few participants noted that some members of the mixed methods community tend to accord a higher status to the quantitative component because they consider that it is more objective and more closely embodies the scientific method. In their view, some researchers regard the qualitative component as mainly a supplement to the quantitative component. Consequently, researchers may fail to appreciate the added value that may be gained by using mixed methods research:

“qualitative research [is often used] to almost to kind of flesh out the , the , the quantitative aspects , so it’s a kind of embellishment rather than seeing it as something that might challenge some of the quantitative findings or might contribute to , to ultimately rephrasing the research question or to reanalyzing the , the quantitative data” (Sociologist #5).

Domain 3: Procedures

Criticism 8 : Limitations of typologies . A number of participants criticized the tendency of some authors to present mixed methods designs and procedures from a typological perspective. Typologies are used in the mixed methods literature as classifications of methodological features, such as the timing and priority of the quantitative and qualitative components and the stage at which integration is carried out [ 7 , 9 ]. In the view of those participants, typologies are presented in the literature in a way that is excessively mechanical and prescriptive, unnecessarily simplifying the process of carrying out a mixed methods study by suggesting that a successful implementation of a mixed methods design can be carried out only by following a predefined set of steps:

“I’m arguing against approaches that I think are too sort of mechanical in the sense of laying out : ‘Ok , here’s categories A , B , C , D and E , and here are the rules for applying them . And if you just follow the rules , then you’ll be ok’” (Educationalist #1).

Participants noted that this approach entails four problems. First, in order to adapt their approach to the research questions that they need to answer, researchers may need to modify the guidelines suggested in the literature. Therefore, guidelines for the use of mixed methods designs should be only “guiding principles” that are adaptable to varying circumstances and able to take into account the interactions between the different elements of the design. One of these participants stated the following:

“I think , I mean , I started off by using…the sort of prescription…and it’s only when you start to get delve more into mixed methods… So , it’s , it’s really , I think [they should be] just guiding principles” (Nurse, #5).

Second, the typologies may curtail the creativity of researchers by restricting them to a series of predefined models that are considered the “correct” ways of combining quantitative and qualitative methods. As expressed by the following participant:

“…researchers are using mixed methods in such creative ways , it’s like , it’s just , when you read these designs and they can be just so , so different and they just don’t fit into , you know , the typologies” (Nurse, #3).

Third, rather than being empirically generated by examining how mixed methods research is actually carried out in practice, these typologies are the highly formalized result of a list of ideal designs formulated by mixed methods theorists, as noted by this participant:

“There were basically two different approaches [to the development of typologies] and the one that was most common was the sort of develop very formal systems…The opposite of that was Bryman who went out and interviewed qualitative researchers about what they did…he talked to people about what they really did rather than coming up with formal systems” (Sociologist #1).

Fourth, the existing typologies are too extensive, which makes them difficult for inexperienced researchers to apply, as we can see in the following quote:

“…there must be forty or fifty different designs associated with mixed methods and I think it’s , you know , I think that’s confusing to people and it’s … in some way I think it becomes irrelevant” (Psychologist #9).

Criticism 9 : Procedures described in the literature are not aligned with mixed methods practice . Some participants mentioned occasional discrepancies between the procedures explained in textbooks and articles and the implementation and reporting of mixed methods in practice, which may not always conform to published guidelines and typologies.

“you , you open any textbook…and the rules that are proposed there are broken every day very successfully by researchers who , who actually conduct the research…the practice and the debate need to run parallel and they probably , right now I think they are a bit too separate from each other” (Sociologist #3).

Participants attributed this disjuncture to the fact that a few influential authors probably lacked sufficient practical experience in using mixed methods. These authors may have tended to suggest methodological guidelines “from their desk” without testing them in practice or reviewing the empirical work of other researchers:

“…there might be a gap that , that a lot of researchers talk about using mixed methods , but I’m not sure if they actually do it in practice” (Psychologist #2).

Furthermore, participants also noted that in some cases those authors may have placed greater emphasis on the philosophical and theoretical basis of mixed methods than on describing the techniques involved in implementing mixed methods research:

“…people spend far too much time talking about epistemology , most of those discussions are actually very simple , but people make them very complicate…I think…technical questions about how you work with the data and what it means [are more important]” (Educationalist #6).

Domain 4: Politics

Criticism 10 : Mixed methods research is not better than monomethod research . Some participants pointed out a tendency among some members of the mixed methods community to consider this type of research to be inherently superior to monomethod research:

“…thinking about the , the papers that I’ve reviewed have been for the kind of applied end journals…I think the main issue for me has been in terms of , you know , the purpose of using mixed methods; that I think there’s a tendency to slip into thinking that more is necessarily better” (Sociologist #9).

Participants noted that to attribute a higher status to mixed methods research is wrong because this view could lead to the oversimplification of other approaches, which would undermine their prestige. Furthermore, participants argued that a mixed methods approach is not always the best research option and a fully integrated design may not be the most appropriate. What really determines the suitability of an approach or a design is the research question of a study, so that a monomethod design is sometimes the most appropriate.

Criticism 11 : Homogenization of mixed methods research . A few participants criticized a tendency in the mixed methods field to homogenize terminology and procedures. In their view, some members of the community have tried to develop a “mixed methods way of doing things” which would be acceptable to all researchers and would require them to write in a particular way using particular terminologies and strategies:

“They’re , they’re trying to develop a language , they’re trying to develop an approach , a strategy that , that is going to be acceptable by all mixed methods researchers , which really is , is unacceptable” (Educationalist #2).

This attitude towards homogenization of mixed methods research could hinder the advancement of the field since it promotes a uniform approach, suppresses intellectual disputes and ignores the diversity of approaches and attitudes regarding mixed methods found in the literature. As one participant argued, to find space for legitimate difference in the field is very difficult due to the protectionist attitude of some prominent authors who are interested in propagating their own ideas rather than incorporating the ideas of other authors:

“I fear that there’s , among , among those who have some prominence , there are some who would be very eager to protect their own turf and not wanting to come together for some kind of joint effort” (Educationalist #3).

RQ2: What differences and similarities can be identified in the criticisms reported by researchers working in different disciplines?

Of the 27 participants who raised criticisms, ten were sociologists, eight were educationalists, five were nurses, and four were psychologists. Multiple correspondence analysis was used to analyze the differences, depending on their discipline, in the types of criticisms the participants raised. Fig 1 shows the multiple correspondence analysis map for the first two axes. Highly associated categories are plotted near to one another on the basis of their loading to the corresponding axes, while the least associated categories are plotted far from one another. Therefore, the axes should be interpreted based on the grouping seen on the map of the relative positions of the categories, which are expressed by the magnitude of the coordinates. The measures known as eigenvalues indicate how much of the categorical information is explained by each dimension. Higher eigenvalues indicate a greater amount of variance of the variables in that dimension [ 29 , 30 ].

Note: The Yes label indicates that the criticism was mentioned, while the No label indicates that the criticism was not mentioned.

On Axis 1, which accounts for most of the variation in the data (eigenvalue of 92.4%), the yes categories of the criticisms associated with the domains of philosophy, politics and the essence of mixed methods are plotted on the right-hand side of the map, while the no categories of the same domains are on the left-hand side of the map. As shown in the figure, sociologists were more likely to formulate criticisms associated with the three abovementioned domains while psychologists and nurses were less likely to formulate criticisms associated with any of these three domains. Educationalists were less inclined than sociologists to formulate criticisms associated with those domains, as indicated by the proximity of the education category to the centroid (the center of the axis). Finally, the fact that the yes and no categories of the procedures domain are plotted on Axis 2, which has very low explanatory power (eigenvalue of 7.6%), reveals that participants in the four disciplines showed no relevant differences in formulating this criticism.

Summary of findings

The aim of this study, based on a secondary analysis of interview data, was to describe the criticisms of mixed methods as a field raised by a sample of researchers in the disciplines of education, nursing, psychology, and sociology. Overall, the findings revealed a number of criticisms related to several different issues. These criticisms were initially identified by the first author in the data analysis phase of the original study, which was focused on the conceptualization and operationalization of quality in mixed methods research. The criticisms were particularly relevant since they were unprompted, that is, they were spontaneously given by the participants in response to questions about another subject, rather than to questions about criticisms. Furthermore, the original study was based on a broad and diverse sample of participants; it included a few researchers from the disciplines of sociology and education known for their criticisms as well as a considerable number of researchers whose publications seemed to be neutral on the subject of criticisms of the mixed methods field. To this latter group of researchers, the interviews afforded an opportunity to express their disagreement with some predominant notions in the mixed methods field which they otherwise might not have published. Another key element of this study is the examination of the relationship between the discipline of the participants and the frequency and type of criticisms they made. This has been possible since the sample is relatively balanced in terms of the number of participants from each of the four disciplines included in the study.

In response to RQ1, participants raised a total of eleven unprompted critical remarks, categorized in the following four domains: the essence of mixed methods (three criticisms), philosophy (four criticisms), procedures (two criticisms), and politics (two criticisms). Nine of the eleven critical comments have been previously mentioned in the literature. For instance, on the essence of mixed methods domain, some authors have alluded to problems such as the narrow definition of mixed methods research [ 15 , 34 – 36 ], the lack of agreement on the terminology used [ 16 , 37 ] and the inappropriateness of considering mixed methods a new methodology [ 35 , 38 , 39 ]. In the philosophical domain, several authors have criticized the dominance of a positivist approach to mixed methods research in some disciplines [ 37 , 40 , 41 ] while some authors have pointed out that considering as a separate or distinct paradigm can lead to an artificial separation of the quantitative and qualitative approaches [ 42 – 45 ]. In the procedures domain, a number of authors cited the problems inherent in conceptualizing mixed methods designs typologically, since such a view is restrictive and unable to reflect the variety of mixed methods designs used in practice [ 38 , 46 , 47 ]. Finally, in the politics domain, a few authors have criticized a tendency, in some of the literature, to homogenize the field [ 48 ], while others have critically noted the occasional adoption of a universalist position based on the idea that the mixed methods approach is inherently superior to monomethod research [ 44 , 47 ]. However, we also identified two criticisms not previously mentioned in the literature: the excessively superficial characterizations of pragmatism (criticism 6) found occasionally in the mixed methods literature and the description of procedures that are not necessarily in line with research practice (criticism 9).

Regarding criticism 6, some participants noted a tendency on the part of some researchers in the field to cite, when writing about pragmatism, what other mixed methods researchers had written about this paradigm rather than citing foundational writings, such as those by John Dewey, William James, or Charles Sanders Peirce. In fact, none of the most influential and most frequently cited textbooks on mixed methods research in the four disciplines we studied cites any work by key authors in the pragmatist tradition. Therefore, it could be useful to learn whether these authors’ highly synthetic explanation of foundational knowledge leads inexperienced researchers to only weakly engage with this paradigm, or, on the other hand, whether this simplification might help them to grasp the basic principles of pragmatism more quickly while leading them to consult first-hand the foundational writings.

With respect to criticism 9, participants reported that researchers do not always follow in practice all of the procedures described in mixed methods textbooks. This disjuncture between textbook guidance and research practice has been described in several methodological reviews of the use of mixed methods in the four disciplines we included in our study. Features such as explicitly stating the mixed methods design used, reporting mixed methods research questions, or explicitly stating the limitations associated with the use of a mixed methods design are regarded by some authors as key characteristics of mixed methods studies (Creswell and Plano Clark, 2018; Plano Clark and Ivankova, 2016; Onwuegbuzie and Corrigan, 2014; O’Cathain et al, 2008). However, Bartholomew & Lockard (2018) reported that very few of the studies included in their review of the use of mixed methods in psychotherapy explicitly stated the mixed methods design used (13%) or reported mixed methods research questions (29%). Additionally, Bressan et al. (2017) and Irvine et al. (2021), in their reviews on mixed methods in nursing, found that most of the studies they included failed to report the limitations associated with the use of a mixed methods design. Therefore, it could be of great interest to study whether the omission of these characteristic features of mixed methods studies reflects the researchers’ view that these features are unimportant, or whether they are unfamiliar with reporting standards. Identifying this latter criticism is a particularly relevant finding of this study, since the intimate context of the interview might have led to the expression of subjective judgments that otherwise might not have come to light (i.e., the participant’s perception that some authors may not habitually carry out empirical research).

With respect to RQ2, we found relevant differences in the type of criticisms raised across disciplines. In fact, one of our key findings is that criticisms in the procedures domain were equally distributed across the four disciplines, while criticisms in the essence of mixed methods, philosophy and politics domains were clearly more common in sociology. First, these findings are consistent with statements made by Plano Clark and Ivankova [ 3 ] regarding the ways in which the sociocultural context of researchers — including the discipline in which they work — can shape their beliefs, knowledge and even experiences with regards to mixed methods. Indeed, the greater number of criticisms made by sociologists categorized in three of the four domains shows how disciplinary conventions might affect how researchers think about mixed methods and judge the acceptability of certain predominant conceptualizations. According to several authors [ 49 – 51 ], critique is a foundational and distinct feature of the discipline of sociology. Therefore, the generalized tendency among sociologists to question traditional assumptions about the order of the world and to detach themselves from predominant belief systems and ideologies might help to explain why many sociologists in our sample criticized ideas such as the conceptualization of quantitative and qualitative research as separate entities and the consideration of mixed methods as inherently better than monomethod research.

Theoretical implications of the study

The findings of our study highlight several differences in opinion in the mixed methods field previously identified by authors such as Greene [ 52 ], Tashakkori and Teddlie [ 17 ], Leech [ 53 ] and Maxwell, Chmiel, and Rogers [ 54 ], among others. Those authors showed that, owing to differences in philosophical and theoretical stances and their adherence to different research cultures, researchers in this field sometimes disagree on foundational issues such as nomenclature, the need for consensus, and the definition of mixed methods research. In our study, the participants, particularly those in the field of sociology, made several criticisms about how some foundational and philosophical aspects of mixed methods research have been conceptualized by the mixed methods community, including how mixed methods has been defined and accorded status as a third paradigm. Furthermore, our findings showed contradictory criticisms formulated by the participants as a group: while some researchers criticized the lack of a consensus in the field on the terminology used to describe mixed methods research (criticism 2), others criticized a tendency by some authors to homogenize terminology (criticism 11).

This divergence of views is consonant with the notion of communities of practice suggested by Denscombe [ 12 ]. Departing from Kuhn’s notion of paradigms as “shared beliefs among the members of a specialty area” (as cited by Morgan [ 10 ]), in Denscombe’s view, the broader mixed methods community is a paradigm encompassing a conglomerate of multiple research communities shaped by the principles, values and interests prevailing in their disciplines and research orientations. In line with that author’s view that methodological decisions and viewpoints “will be shaped by a socialization process involving the influence of peers” [ 12 ], our findings suggest that the disciplinary community of our participants is likely to have informed their criticisms of the mixed methods field. Although this divergence of views suggests that complete agreement and unhindered communication among researchers is not possible [ 55 ], Ghiara [ 56 ] argued that, in Kuhn’s view, some form of communication is always possible; and furthermore, conflicting viewpoints can be reconciled to a certain extent. Similarly, Johnson [ 57 ] has argued that this diversity of critical voices, rather than being a problem, merely indicates that “reality is likely plural”—there is no single set of ontological assumptions underlying mixed methods—and, furthermore, knowledge is articulated on the basis of “multiple standpoints and strategies for learning about our world” that can be reconciled to some degree. An example of the healthy coexistence of divergent viewpoints within the mixed methods community can be found in the Journal of Mixed Methods Research , the leading journal in the field, described in its webpage as a “primary forum for the growing community of international and multidisciplinary scholars of mixed methods research”. The journal publishes a wide range of manuscripts, including articles revealing approaches to mixed methods research that rest on divergent foundational and philosophical perspectives.

The desire for inclusion of divergent viewpoints should not lead researchers to ignore the challenges posed by this divergence. Due to the interdisciplinary nature of mixed methods research, Curry et al. [ 58 ] argue that mixed methods teams often include researchers with different methodological backgrounds and propensities. Occasionally, these differences may pose challenges for establishing effective collaboration and for efficiently integrating research methods. A greater awareness of multiple perspectives on mixed methods research, including divergent critical views like those reported in our study, could help researchers better anticipate difficulties that might present themselves in the course of working with researchers who hold differing viewpoints. Furthermore, as Maxwell, Chmiel, and Rogers [ 54 ] have suggested, a better understanding on the part of mixed methods researchers of the perspectives of others in the field who embrace a differing approach should facilitate the process of integrating quantitative and qualitative methods in studies where different ontological positions coincide. In a similar vein, such an understanding could also help overcome a form of methodological tokenism described by Hancock, Sykes and Verma [ 59 ]. This can occur when mixed methods researchers fail to attend to, and therefore align, the distinct epistemological and ontological premises that underlie the methodological orientations that are integrated in a mixed methods design.

Furthermore, the recognition of mixed methods researchers’ divergence of views should be an integral part in any effort to design and implement a curriculum for mixed methods research. Plano Clark and Ivankova [ 13 ] have pointed out that any lack of clarity concerning the existing disagreements about foundational elements of the mixed methods approach could be confusing to researchers inexperienced in this field. Therefore, it is essential that courses and workshops on mixed methods research take note of these criticisms. This last point is particularly important since the topic of critical viewpoints is not included in any of the mixed methods syllabus exemplars published in the literature, including those by Earley [ 60 ], Christ [ 61 ], and Ivankova and Plano Clark [ 13 ].

Limitations, strengths, and possibilities for future research

Our findings are subject to a few limitations. First of all, the interviews were carried out by Skype and telephone. While these two forms of communication allowed us to interview participants residing in various locations around the world, they limited the possibilities for building the sort of rapport that might have encouraged some participants to elaborate more in their responses. To minimize this limitation, participants were given the opportunity in the member-checking phase to add additional insights to their initial statements. Second, the transferability of our findings is limited by our decision to include only four disciplines while excluding the views of researchers working in other disciplines that also have a high prevalence of mixed methods studies, such as medicine, business, and information science. From our findings alone, it is not possible to infer how frequent these criticisms are and what types of criticisms may be more prevalent in each of the disciplines. Third, multiple correspondence analysis is an exploratory method not appropriate for testing hypotheses or statistical significance. In other words, the method is designed to describe associations between categorical variables rather than to make predictions about a population [ 31 ]. Therefore, in light of the limitations of multiple correspondence analysis for drawing deeper inferences, the findings regarding RQ2 should be considered provisional and subject to further investigation. Finally, since this study is based on a secondary analysis, the interview questions did not specifically prompt the participants to bring up criticisms since the questions were focused on participants’ views related to the quality of mixed methods research. If we had specifically prompted participants to report their own criticisms, it is likely that more critical opinions would have been gathered.

While the use of secondary data entailed certain limitations, it also conferred some advantages. Participants’ critical statements were entirely spontaneous since they were not explicitly solicited. This spontaneity probably helped to reduce the social desirability bias, that is, a presumed tendency for respondents to dissimulate their own critical views in a way that might seem professionally and socially more acceptable. Furthermore, despite the limitations of multiple correspondence analysis, this method allowed us to generate a parsimonious visual representation of the underlying patterns of relationships among the criticisms and the disciplines. This representation helped us to improve our interpretation of the qualitative findings and to identify useful leads for carrying our further analysis. A further strength is that the study included a broad sample of participants in terms of their geographic location, academic position, seniority, and methodological expertise. This diversity probably also afforded us access to a wider range of views. Finally, a key strength of our study is that, to our knowledge, it is the first empirical study that has addressed the topic of criticisms of mixed methods as a field.

This study represents a step towards a better understanding of some current criticisms of mixed methods research. However, further research will be needed to confirm and expand our findings. Future research based on a larger and more diverse sample of mixed methods researchers could extend the scope of our research questions and help researchers generalize from our findings. Such studies might help us discover whether researchers from other disciplines share the criticisms made by the participants in our study and whether those researchers harbor other criticisms of their own. Other analytical tools could be used to examine in greater detail how the circumstances and attributes of researchers — including their disciplinary background, methodological expertise and paradigmatic viewpoints — influence the way these scholars formulate their criticisms of the mixed methods field. Finally, future research is needed to examine the critical accounts of researchers less experienced in methodological writing and probably less exposed to current theoretical debates and developmental issues in the field.

Acknowledgments

The authors would like to acknowledge the help of Dick Edelstein in editing the final manuscript. The first draft of this paper was written while the first author was a visiting scholar in the Mixed Methods Program of the University of Michigan, directed by Michael D. Fetters, John W. Creswell, and Timothy D. Guetterman. This author would like to thank these three scholars for their support and, particularly Michael D. Fetters for his invaluable mentorship over the past few years.

Funding Statement

The authors received no specific funding for this work.

Data Availability

• PLoS One. 2021; 16(7): e0252014.

Decision Letter 0

13 Nov 2020

PONE-D-20-22907

Dear Dr. Fàbregues,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Your manuscript was reviewed by three accomplished scientists with mixed methods expertise. I also reviewed the paper and I concur with the vast majority of all comments, critiques and suggestions raised by the reviewers. This is a highly important manuscript that presents much needed findings in an area seldom studied. In addition to the reviewer comments, I have a major concern about the multiple correspondence analysis. How reproducible are the MCA results? I would suggest that the authors more explicitly acknowledge that this part of the paper is more exploratory than thematic/content analysis. Lastly, I strongly agree with Reviewer 1's concern about use of the term pedagogy and the associated text in the discussion. The dissimilarity between practice and textbook descriptions of mixed methods is notable, but the authors should be more precise with use of language here. 

Please submit your revised manuscript by Dec 28 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at gro.solp@enosolp . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Thank you for your time and dedication in preparing this important work. We look forward to receiving your revised manuscript.

Kind regards,

Adam T. Perzynski, PhD

Academic Editor

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions .

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories .

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Partly

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

Reviewer #3: Yes

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: No

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Please see my full review uploaded as attachment. In summary, this is an interesting study that provides a more transparent window into some of the recurring debates within and among mixed methods researchers and their critics and detractors, but perhaps more importantly, interview data reveals aspects of mixed methods critiques that have not been as explicitly or formally stated in the published research literature, and the findings raise some important operational, conceptual, professional, and practical implications. However, the credibility and validity of the work presented in this manuscript is hampered by the authors’ editorializing recommendations and, at times, apparently unfounded interpretations of the empirical data. The mainly descriptive understanding gained from the study itself warrants dissemination without the need to overstep the bounds of what these empirics allow the authors to infer. The study authors describe was systematic, methodologically sound, and proceeded from a careful research design that affords reasonable confidence in the validity of descriptive results and makes the limits of generalizability and potential sources of bias clear. Yet, subjective interpretations beyond the data amount to a form of advocacy that the study at hand simply does not warrant.

Based on my review of the work in its present form, I recommend that PLOS ONE request that authors Revise & Resubmit, with Major Revisions, Manuscript Number: PONE-D-20-22907, in order to address this main weakness, with a focus on rethinking and rearticulating the nature of the implications of the findings in the current Discussion section (pp.23-29) and Abstract.

Reviewer #2: In this study, the authors reported the result of an analysis of the criticisms of mixed methods research using a cross-national purposive sample of 42 researchers from multiple disciplines (education, nursing, psychology, sociology) who have published on the methods of mixed-methods research. The researchers originally participated in an interview regarding quality in mixed-methods research and were then recontacted and, if they agreed, provided additional information to their original responses. Eleven different responses were identified by the authors and then grouped into 4 domains used by an expert in mixed methods research (Creswell) to map the mixed methods research landscape: essence, philosophy, procedures, and politics. Criticisms related to essence, philosophy, and politics were more prevalent among sociologists. Criticisms related to procedures were equally distributed across the disciplines. Based on their findings, the authors made four recommendations: address procedural criticisms, be more tolerant of the diverse ways to conceptualize mixed methods, account for other disciplines’ criticisms; and take note these mixed methods criticisms in training/educational programs.

Mixed methods have become increasingly more common and promoted in research, and the authors are to be commended for investigating criticisms of these approaches among a sample of investigators from different parts of the world and from different disciplines. This is an interesting study. My concerns mainly revolve around making the language a little more accessible to the wide readership of PLOS by explaining terminology used, and providing additional detail so that readers may better understand the methods and findings. If it is possible (in terms of word limits), the manuscript would be enhanced if the authors utilized more quotations to illustrate their findings and allay any reader concerns that many of the quotations are coming from the same individuals. My specific concerns follow:

1. Line 35: It would be helpful to readers unfamiliar with qualitative work for the authors to describe what is meant by “typologies.”

2. Lines 74-75: Related to points #7 and #21 below, if the secondary data analysis was already “covered” by the IRB, why were participants recontacted? It appears from later in the manuscript that this contact was an opportunity for authors to collect additional information from participants? Please clarify.

3. Line 86: It would be helpful to readers if the authors explained what they meant by criterion sampling. Does this mean there were inclusion criteria?

4. Line 105: This is the data collection procedure for the original study? Please clarify.

5. Lines 106-107: There seems to be some missing text, and there is no (a).

6. Line 109 (two interviews): In other words, each participant was interviewed twice? Please clarify.

7. Related to Point #21 below, the fact that the authors were recontacted and given opportunity to clarify their responses needs to be described here in the data collection. Presumably, it was this opportunity to clarify that necessitated recontacting participants (point 2 above) and that two individuals turned down? Please clarify.

8. Lines 164-165. It would be helpful to readers unfamiliar with Creswell’s publication for the authors to provide more descriptive information about each of the four domains. For example, what do “philosophy” and “politics” refer to here?

9. Criticism #2 in Table 2: I think it should be “proponents.”

10. General concern about the quotations: The quotations are very helpful to illustrate points. Is there a way the authors could identify the individuals, may like “Educationalist #1” or “Sociologist #3” to allay readers’ concerns that all the quotations from sociologists, for example, are not coming from the same sociologist.

11. Lines 257-259: the phrase “…presupposes a strong link between epistemology and method, whereas….” is unclear. Please clarify.

12. Lines 279-281: To help readers unfamiliar with the dominant rules of the 1980s, can the authors explain more about what the rules were.

13. Lines 312-315: For readers unfamiliar with these terms, the authors should explain what is meant by “pragmatism,” “eclecticism,” and “practialism.”

14. Line 343: The authors use the phrase “in some disciplines,” but from the quotation that follows it is not clear whether the participant was referring to entire disciplines or individual researchers. Please clarify.

15. Lines 393-406: It is a little concerning that the authors refer to participants in the plural but provide only one quotation. Is it possible to provide additional quotations to show more clearly to readers that multiple participants shared these views.

16. Lines 427-429: It is interesting that some participants complained about the tendency for homogenized terminologies and procedures, while other participants complained about the variation in terms and lack of agreement (lines 202-206). I might have missed it, but this might be important to include in the discussion: that the criticisms can be contradictory, which might make them difficult to address in a manner that will please everybody.

17. Lines 448-466: The authors need to help readers understand what Figure 1 is showing and what correspondence analysis mapping is doing. What does 92.4% actually refer to? (line 453). What do the values of -1.5 to 1.5 mean on the axes. It will be really helpful to readers if the authors can explain more clearly why a specific discipline is placed where it is relative to the axes’ scales.

18. Lines 486-487 “known for their criticisms”: Were the individuals known for their criticisms all from the same discipline?

19. Line 541 Limitations: The small numbers and the relatively small number of disciplines should be noted as limitations.

20. Line 541: It seems like a limitation that should be noted is that the sample was limited to researchers who have written on the methods of mixed-methods research, especially because one of the criticisms was that the people who are writing about mixed methods aren’t necessarily the ones who are really doing mixed methods research.

21. Line 547-548: The ability to provide additional insights needs to be included in the methods section. Is this what the two persons refused to do?

22. Line 567 “typologies”: Was this typologies of methods? Please clarify.

23. Maybe I missed it, but do the authors have ideas for future research in this area?

24. Figure 1: Shouldn’t the axes be labeled in some way?

Reviewer #3: Thank you for the opportunity to review this manuscript examining criticisms of the field of mixed-methods research. The manuscript is well written and clear, and is grounded in prior articulations of controversies embedded within mixed-methods research. Less clear, however, are the ways in which the authors see this study’s findings advancing the field, and the extent to which the study is sufficiently distinct, as a secondary data analysis, from the primary parent study. Additional details are as follows.

- In the introduction, the authors describe two existing summary overviews of criticism of mixed-methods research but that, to date, no empirical examination of this domain has been carried out. While the fact that prior work, until now, has discussed this topic in terms of theory and methodology is notable, I am not sure that the authors have provided a convincing rationale yet for why this particular study is useful. This seems important to explicate, especially given the nearly complete overlap of the study’s findings with these two prior syntheses.

- Could the authors clarify how they inferred the methodological expertise of the participants?

- From the authors’ stated inclusion criteria, participants from other disciplines that commonly employ mixed methods research (public health, social welfare, etc.) were excluded from the study. Though it may be unrealistic to expect a study like this to include participants from comprehensive disciplines, could the authors add some clarification on their defined focus on education, nursing, psychology, and sociology?

- I recommend that the authors add clarification and explanation of how, exactly, the current study really diverges from the parent study such that it constitutes a qualitative secondary data analysis rather than a presentation of residual findings from the parent analysis already published. The authors may wish to reference Williams and Collins (2002) for an example of structuring an analysis section with more fidelity to the approach of qualitative secondary data analysis:

Williams, C. C., & Collins, A. A. (2002). The social construction of disability in schizophrenia. Qualitative Health Research, 12(3), 297-309.

- Related to the above: at multiple points the authors highlight that the criticisms they are discussing are particularly relevant because they were given spontaneously in the original study. However, the original study appears to focus on evaluations of mixed methods research. This seems like a natural setting in which criticisms and approvals would be raised– all of which is to say, I am not sure that this is as noteworthy a characteristic of the current study as the authors seem to suggest it is. Indeed, at least two of the current study’s identified criticisms were already presented as findings in the original study.

- How transferable do the authors believe their findings are for other researchers who are mixed methodologists? (Keeping in mind that the study likely excluded researchers who work at the intersection of the inclusion criteria disciplines which may affect their views and experiences of mixed methods research.)

- I am not sure that it is clear yet how the current study provides much insight beyond what Creswell and others already have established. As the authors note in the discussion, the vast majority of the criticisms they identified have previously been documented in literature about mixed methods approaches to research. If the authors don’t identify much additional insight, I think that can be ok, as mounting evidence of the need for clarity among mixed-methods research could be seen as important for the continued growth, development, and advancement of the field. But if the authors see other novel conclusions, they should state this clearly.

- Related to the above, in the discussion section the authors dedicate the majority of space to reiterating that most of their findings have been noted in other research. There is comparatively less content discussing and exploring what their two new findings (lines 515-517) mean for the domain of mixed methods research.

- There seem to be some interesting contradictions present in the study’s findings that are not currently attended to or processed in the discussion section. For instance, the finding pertaining to needing consensus on terminology (e.g., lines 225-228) within criticism 2 seems at odds with criticism 11 in which it is described that “participants criticized a tendency in the mixed methods field to homogenize terminology.” Similarly, criticism 8 focuses on the notion that literature on mixed-methods presents typologies that “unnecessarily simplif[y] the process of carrying out a mixed methods study” and “curtail the creativity of researchers,” yet criticism 9 focuses on how researchers often do not align their mixed-methods practice with procedures documented in the literature. It may make sense that a diverse sample may have differing stances, but the discussion section would benefit from explicit attention to and interpretation of these incongruencies.

- Though the discussion section discusses some practical implications of the study’s findings, this section would benefit from additional detail and substance. For example, what do the authors recommend as requisite steps to building awareness (lines 588-591)? How, exactly, are the authors suggesting that procedural criticisms (lines 566-570) should be addressed in order to improve the practice of mixed methods research?

- On page 7, “Questions from the background, (b) participants’ conceptualization of mixed methods research, and (c) how participants perceived the quality of the mixed methods approach in practice” is not a complete sentence.

- In Table 2, 1.2., “proponent” should be “proponents”.

- Criticisms as section headings should be phrased consistently either as the researchers’ belief or the belief they’re taking a stance against. For instance, criticism 4 is “Mixed methods research is not a third paradigm” and seems appropriately worded as a criticism; however, criticism 10 is “Mixed methods research is better than monomethod research” when the criticism itself seems to be that that mixed-methods research is not better than monomethod research.

6. PLOS authors have the option to publish the peer review history of their article ( what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool,  https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at  gro.solp@serugif . Please note that Supporting Information files do not need this step.

Submitted filename: Review_PONE-D-20-2297_2020-09-23_submitted.pdf

Author response to Decision Letter 0

26 Feb 2021

The Response to Reviewers file has been attached to the manuscript.

Submitted filename: Response to Reviewers.docx

Decision Letter 1

10 May 2021

PONE-D-20-22907R1

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Please take note of and make necessary changes according to the minor requests made by Reviewer 3. 

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/ , click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at gro.solp@gnillibrohtua .

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact gro.solp@sserpeno .

Additional Editor Comments (optional):

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

2. Is the manuscript technically sound, and do the data support the conclusions?

3. Has the statistical analysis been performed appropriately and rigorously?

4. Have the authors made all data underlying the findings in their manuscript fully available?

5. Is the manuscript presented in an intelligible fashion and written in standard English?

6. Review Comments to the Author

Reviewer #1: (No Response)

Reviewer #2: I appreciate the authors' effort to answer my concerns. These concerns have been adequately addressed.

Reviewer #3: I appreciate the opportunity to read this edited manuscript. I believe that the authors have addressed reviewers' concerns and have provided appropriate clarification, additional detail, and expanded discussion as recommended. I see this manuscript as an important contribution to the field and look forward to seeing how other scholars interpret and build upon this work.

While I recommend acceptance, I have two very minor suggestions related to Table 1: first, that the authors replace "Gender" with "Sex", and that a "Total" column might be added for a reader's very quick reference.

7. PLOS authors have the option to publish the peer review history of their article ( what does this mean? ). If published, this will include your full peer review and any attached files.

Acceptance letter

30 Jun 2021

Dear Dr. Fàbregues:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact gro.solp@sserpeno .

If we can help with anything else, please email us at gro.solp@enosolp .

Thank you for submitting your work to PLOS ONE and supporting open access.

PLOS ONE Editorial Office Staff

on behalf of

Dr. Adam T. Perzynski

Read our research on: Abortion | Podcasts | Election 2024

Regions & Countries

What the data says about abortion in the u.s..

Pew Research Center has conducted many surveys about abortion over the years, providing a lens into Americans’ views on whether the procedure should be legal, among a host of other questions.

In a  Center survey  conducted nearly a year after the Supreme Court’s June 2022 decision that  ended the constitutional right to abortion , 62% of U.S. adults said the practice should be legal in all or most cases, while 36% said it should be illegal in all or most cases. Another survey conducted a few months before the decision showed that relatively few Americans take an absolutist view on the issue .

Find answers to common questions about abortion in America, based on data from the Centers for Disease Control and Prevention (CDC) and the Guttmacher Institute, which have tracked these patterns for several decades:

How many abortions are there in the U.S. each year?

How has the number of abortions in the u.s. changed over time, what is the abortion rate among women in the u.s. how has it changed over time, what are the most common types of abortion, how many abortion providers are there in the u.s., and how has that number changed, what percentage of abortions are for women who live in a different state from the abortion provider, what are the demographics of women who have had abortions, when during pregnancy do most abortions occur, how often are there medical complications from abortion.

This compilation of data on abortion in the United States draws mainly from two sources: the Centers for Disease Control and Prevention (CDC) and the Guttmacher Institute, both of which have regularly compiled national abortion data for approximately half a century, and which collect their data in different ways.

The CDC data that is highlighted in this post comes from the agency’s “abortion surveillance” reports, which have been published annually since 1974 (and which have included data from 1969). Its figures from 1973 through 1996 include data from all 50 states, the District of Columbia and New York City – 52 “reporting areas” in all. Since 1997, the CDC’s totals have lacked data from some states (most notably California) for the years that those states did not report data to the agency. The four reporting areas that did not submit data to the CDC in 2021 – California, Maryland, New Hampshire and New Jersey – accounted for approximately 25% of all legal induced abortions in the U.S. in 2020, according to Guttmacher’s data. Most states, though,  do  have data in the reports, and the figures for the vast majority of them came from each state’s central health agency, while for some states, the figures came from hospitals and other medical facilities.

Discussion of CDC abortion data involving women’s state of residence, marital status, race, ethnicity, age, abortion history and the number of previous live births excludes the low share of abortions where that information was not supplied. Read the methodology for the CDC’s latest abortion surveillance report , which includes data from 2021, for more details. Previous reports can be found at  stacks.cdc.gov  by entering “abortion surveillance” into the search box.

For the numbers of deaths caused by induced abortions in 1963 and 1965, this analysis looks at reports by the then-U.S. Department of Health, Education and Welfare, a precursor to the Department of Health and Human Services. In computing those figures, we excluded abortions listed in the report under the categories “spontaneous or unspecified” or as “other.” (“Spontaneous abortion” is another way of referring to miscarriages.)

Guttmacher data in this post comes from national surveys of abortion providers that Guttmacher has conducted 19 times since 1973. Guttmacher compiles its figures after contacting every known provider of abortions – clinics, hospitals and physicians’ offices – in the country. It uses questionnaires and health department data, and it provides estimates for abortion providers that don’t respond to its inquiries. (In 2020, the last year for which it has released data on the number of abortions in the U.S., it used estimates for 12% of abortions.) For most of the 2000s, Guttmacher has conducted these national surveys every three years, each time getting abortion data for the prior two years. For each interim year, Guttmacher has calculated estimates based on trends from its own figures and from other data.

The latest full summary of Guttmacher data came in the institute’s report titled “Abortion Incidence and Service Availability in the United States, 2020.” It includes figures for 2020 and 2019 and estimates for 2018. The report includes a methods section.

In addition, this post uses data from StatPearls, an online health care resource, on complications from abortion.

An exact answer is hard to come by. The CDC and the Guttmacher Institute have each tried to measure this for around half a century, but they use different methods and publish different figures.

The last year for which the CDC reported a yearly national total for abortions is 2021. It found there were 625,978 abortions in the District of Columbia and the 46 states with available data that year, up from 597,355 in those states and D.C. in 2020. The corresponding figure for 2019 was 607,720.

The last year for which Guttmacher reported a yearly national total was 2020. It said there were 930,160 abortions that year in all 50 states and the District of Columbia, compared with 916,460 in 2019.

• How the CDC gets its data: It compiles figures that are voluntarily reported by states’ central health agencies, including separate figures for New York City and the District of Columbia. Its latest totals do not include figures from California, Maryland, New Hampshire or New Jersey, which did not report data to the CDC. ( Read the methodology from the latest CDC report .)
• How Guttmacher gets its data: It compiles its figures after contacting every known abortion provider – clinics, hospitals and physicians’ offices – in the country. It uses questionnaires and health department data, then provides estimates for abortion providers that don’t respond. Guttmacher’s figures are higher than the CDC’s in part because they include data (and in some instances, estimates) from all 50 states. ( Read the institute’s latest full report and methodology .)

While the Guttmacher Institute supports abortion rights, its empirical data on abortions in the U.S. has been widely cited by  groups  and  publications  across the political spectrum, including by a  number of those  that  disagree with its positions .

These estimates from Guttmacher and the CDC are results of multiyear efforts to collect data on abortion across the U.S. Last year, Guttmacher also began publishing less precise estimates every few months , based on a much smaller sample of providers.

The figures reported by these organizations include only legal induced abortions conducted by clinics, hospitals or physicians’ offices, or those that make use of abortion pills dispensed from certified facilities such as clinics or physicians’ offices. They do not account for the use of abortion pills that were obtained  outside of clinical settings .

(Back to top)

The annual number of U.S. abortions rose for years after Roe v. Wade legalized the procedure in 1973, reaching its highest levels around the late 1980s and early 1990s, according to both the CDC and Guttmacher. Since then, abortions have generally decreased at what a CDC analysis called  “a slow yet steady pace.”

Guttmacher says the number of abortions occurring in the U.S. in 2020 was 40% lower than it was in 1991. According to the CDC, the number was 36% lower in 2021 than in 1991, looking just at the District of Columbia and the 46 states that reported both of those years.

(The corresponding line graph shows the long-term trend in the number of legal abortions reported by both organizations. To allow for consistent comparisons over time, the CDC figures in the chart have been adjusted to ensure that the same states are counted from one year to the next. Using that approach, the CDC figure for 2021 is 622,108 legal abortions.)

There have been occasional breaks in this long-term pattern of decline – during the middle of the first decade of the 2000s, and then again in the late 2010s. The CDC reported modest 1% and 2% increases in abortions in 2018 and 2019, and then, after a 2% decrease in 2020, a 5% increase in 2021. Guttmacher reported an 8% increase over the three-year period from 2017 to 2020.

As noted above, these figures do not include abortions that use pills obtained outside of clinical settings.

Guttmacher says that in 2020 there were 14.4 abortions in the U.S. per 1,000 women ages 15 to 44. Its data shows that the rate of abortions among women has generally been declining in the U.S. since 1981, when it reported there were 29.3 abortions per 1,000 women in that age range.

The CDC says that in 2021, there were 11.6 abortions in the U.S. per 1,000 women ages 15 to 44. (That figure excludes data from California, the District of Columbia, Maryland, New Hampshire and New Jersey.) Like Guttmacher’s data, the CDC’s figures also suggest a general decline in the abortion rate over time. In 1980, when the CDC reported on all 50 states and D.C., it said there were 25 abortions per 1,000 women ages 15 to 44.

That said, both Guttmacher and the CDC say there were slight increases in the rate of abortions during the late 2010s and early 2020s. Guttmacher says the abortion rate per 1,000 women ages 15 to 44 rose from 13.5 in 2017 to 14.4 in 2020. The CDC says it rose from 11.2 per 1,000 in 2017 to 11.4 in 2019, before falling back to 11.1 in 2020 and then rising again to 11.6 in 2021. (The CDC’s figures for those years exclude data from California, D.C., Maryland, New Hampshire and New Jersey.)

The CDC broadly divides abortions into two categories: surgical abortions and medication abortions, which involve pills. Since the Food and Drug Administration first approved abortion pills in 2000, their use has increased over time as a share of abortions nationally, according to both the CDC and Guttmacher.

The majority of abortions in the U.S. now involve pills, according to both the CDC and Guttmacher. The CDC says 56% of U.S. abortions in 2021 involved pills, up from 53% in 2020 and 44% in 2019. Its figures for 2021 include the District of Columbia and 44 states that provided this data; its figures for 2020 include D.C. and 44 states (though not all of the same states as in 2021), and its figures for 2019 include D.C. and 45 states.

Guttmacher, which measures this every three years, says 53% of U.S. abortions involved pills in 2020, up from 39% in 2017.

Two pills commonly used together for medication abortions are mifepristone, which, taken first, blocks hormones that support a pregnancy, and misoprostol, which then causes the uterus to empty. According to the FDA, medication abortions are safe  until 10 weeks into pregnancy.

Surgical abortions conducted  during the first trimester  of pregnancy typically use a suction process, while the relatively few surgical abortions that occur  during the second trimester  of a pregnancy typically use a process called dilation and evacuation, according to the UCLA School of Medicine.

In 2020, there were 1,603 facilities in the U.S. that provided abortions,  according to Guttmacher . This included 807 clinics, 530 hospitals and 266 physicians’ offices.

While clinics make up half of the facilities that provide abortions, they are the sites where the vast majority (96%) of abortions are administered, either through procedures or the distribution of pills, according to Guttmacher’s 2020 data. (This includes 54% of abortions that are administered at specialized abortion clinics and 43% at nonspecialized clinics.) Hospitals made up 33% of the facilities that provided abortions in 2020 but accounted for only 3% of abortions that year, while just 1% of abortions were conducted by physicians’ offices.

Looking just at clinics – that is, the total number of specialized abortion clinics and nonspecialized clinics in the U.S. – Guttmacher found the total virtually unchanged between 2017 (808 clinics) and 2020 (807 clinics). However, there were regional differences. In the Midwest, the number of clinics that provide abortions increased by 11% during those years, and in the West by 6%. The number of clinics  decreased  during those years by 9% in the Northeast and 3% in the South.

The total number of abortion providers has declined dramatically since the 1980s. In 1982, according to Guttmacher, there were 2,908 facilities providing abortions in the U.S., including 789 clinics, 1,405 hospitals and 714 physicians’ offices.

The CDC does not track the number of abortion providers.

In the District of Columbia and the 46 states that provided abortion and residency information to the CDC in 2021, 10.9% of all abortions were performed on women known to live outside the state where the abortion occurred – slightly higher than the percentage in 2020 (9.7%). That year, D.C. and 46 states (though not the same ones as in 2021) reported abortion and residency data. (The total number of abortions used in these calculations included figures for women with both known and unknown residential status.)

The share of reported abortions performed on women outside their state of residence was much higher before the 1973 Roe decision that stopped states from banning abortion. In 1972, 41% of all abortions in D.C. and the 20 states that provided this information to the CDC that year were performed on women outside their state of residence. In 1973, the corresponding figure was 21% in the District of Columbia and the 41 states that provided this information, and in 1974 it was 11% in D.C. and the 43 states that provided data.

In the District of Columbia and the 46 states that reported age data to  the CDC in 2021, the majority of women who had abortions (57%) were in their 20s, while about three-in-ten (31%) were in their 30s. Teens ages 13 to 19 accounted for 8% of those who had abortions, while women ages 40 to 44 accounted for about 4%.

The vast majority of women who had abortions in 2021 were unmarried (87%), while married women accounted for 13%, according to  the CDC , which had data on this from 37 states.

In the District of Columbia, New York City (but not the rest of New York) and the 31 states that reported racial and ethnic data on abortion to  the CDC , 42% of all women who had abortions in 2021 were non-Hispanic Black, while 30% were non-Hispanic White, 22% were Hispanic and 6% were of other races.

Looking at abortion rates among those ages 15 to 44, there were 28.6 abortions per 1,000 non-Hispanic Black women in 2021; 12.3 abortions per 1,000 Hispanic women; 6.4 abortions per 1,000 non-Hispanic White women; and 9.2 abortions per 1,000 women of other races, the  CDC reported  from those same 31 states, D.C. and New York City.

For 57% of U.S. women who had induced abortions in 2021, it was the first time they had ever had one,  according to the CDC.  For nearly a quarter (24%), it was their second abortion. For 11% of women who had an abortion that year, it was their third, and for 8% it was their fourth or more. These CDC figures include data from 41 states and New York City, but not the rest of New York.

Nearly four-in-ten women who had abortions in 2021 (39%) had no previous live births at the time they had an abortion,  according to the CDC . Almost a quarter (24%) of women who had abortions in 2021 had one previous live birth, 20% had two previous live births, 10% had three, and 7% had four or more previous live births. These CDC figures include data from 41 states and New York City, but not the rest of New York.

The vast majority of abortions occur during the first trimester of a pregnancy. In 2021, 93% of abortions occurred during the first trimester – that is, at or before 13 weeks of gestation,  according to the CDC . An additional 6% occurred between 14 and 20 weeks of pregnancy, and about 1% were performed at 21 weeks or more of gestation. These CDC figures include data from 40 states and New York City, but not the rest of New York.

About 2% of all abortions in the U.S. involve some type of complication for the woman , according to an article in StatPearls, an online health care resource. “Most complications are considered minor such as pain, bleeding, infection and post-anesthesia complications,” according to the article.

The CDC calculates  case-fatality rates for women from induced abortions – that is, how many women die from abortion-related complications, for every 100,000 legal abortions that occur in the U.S .  The rate was lowest during the most recent period examined by the agency (2013 to 2020), when there were 0.45 deaths to women per 100,000 legal induced abortions. The case-fatality rate reported by the CDC was highest during the first period examined by the agency (1973 to 1977), when it was 2.09 deaths to women per 100,000 legal induced abortions. During the five-year periods in between, the figure ranged from 0.52 (from 1993 to 1997) to 0.78 (from 1978 to 1982).

The CDC calculates death rates by five-year and seven-year periods because of year-to-year fluctuation in the numbers and due to the relatively low number of women who die from legal induced abortions.

In 2020, the last year for which the CDC has information , six women in the U.S. died due to complications from induced abortions. Four women died in this way in 2019, two in 2018, and three in 2017. (These deaths all followed legal abortions.) Since 1990, the annual number of deaths among women due to legal induced abortion has ranged from two to 12.

The annual number of reported deaths from induced abortions (legal and illegal) tended to be higher in the 1980s, when it ranged from nine to 16, and from 1972 to 1979, when it ranged from 13 to 63. One driver of the decline was the drop in deaths from illegal abortions. There were 39 deaths from illegal abortions in 1972, the last full year before Roe v. Wade. The total fell to 19 in 1973 and to single digits or zero every year after that. (The number of deaths from legal abortions has also declined since then, though with some slight variation over time.)

The number of deaths from induced abortions was considerably higher in the 1960s than afterward. For instance, there were 119 deaths from induced abortions in  1963  and 99 in  1965 , according to reports by the then-U.S. Department of Health, Education and Welfare, a precursor to the Department of Health and Human Services. The CDC is a division of Health and Human Services.

Note: This is an update of a post originally published May 27, 2022, and first updated June 24, 2022.

Sign up for our weekly newsletter

Fresh data delivered Saturday mornings

Key facts about the abortion debate in America

Public opinion on abortion, three-in-ten or more democrats and republicans don’t agree with their party on abortion, partisanship a bigger factor than geography in views of abortion access locally, do state laws on abortion reflect public opinion, most popular.

About Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of The Pew Charitable Trusts .

Advertisement

Supported by

Use of Abortion Pills Has Risen Significantly Post Roe, Research Shows

By Pam Belluck

Pam Belluck has been reporting about reproductive health for over a decade.

• Share full article

On the eve of oral arguments in a Supreme Court case that could affect future access to abortion pills, new research shows the fast-growing use of medication abortion nationally and the many ways women have obtained access to the method since Roe v. Wade was overturned in June 2022.

The Details

A study, published on Monday in the medical journal JAMA , found that the number of abortions using pills obtained outside the formal health system soared in the six months after the national right to abortion was overturned. Another report, published last week by the Guttmacher Institute , a research organization that supports abortion rights, found that medication abortions now account for nearly two-thirds of all abortions provided by the country’s formal health system, which includes clinics and telemedicine abortion services.

The JAMA study evaluated data from overseas telemedicine organizations, online vendors and networks of community volunteers that generally obtain pills from outside the United States. Before Roe was overturned, these avenues provided abortion pills to about 1,400 women per month, but in the six months afterward, the average jumped to 5,900 per month, the study reported.

Overall, the study found that while abortions in the formal health care system declined by about 32,000 from July through December 2022, much of that decline was offset by about 26,000 medication abortions from pills provided by sources outside the formal health system.

“We see what we see elsewhere in the world in the U.S. — that when anti-abortion laws go into effect, oftentimes outside of the formal health care setting is where people look, and the locus of care gets shifted,” said Dr. Abigail Aiken, who is an associate professor at the University of Texas at Austin and the lead author of the JAMA study.

The co-authors were a statistics professor at the university; the founder of Aid Access, a Europe-based organization that helped pioneer telemedicine abortion in the United States; and a leader of Plan C, an organization that provides consumers with information about medication abortion. Before publication, the study went through the rigorous peer review process required by a major medical journal.

The telemedicine organizations in the study evaluated prospective patients using written medical questionnaires, issued prescriptions from doctors who were typically in Europe and had pills shipped from pharmacies in India, generally charging about $100. Community networks typically asked for some information about the pregnancy and either delivered or mailed pills with detailed instructions, often for free.

Online vendors, which supplied a small percentage of the pills in the study and charged between $39 and $470, generally did not ask for women’s medical history and shipped the pills with the least detailed instructions. Vendors in the study were vetted by Plan C and found to be providing genuine abortion pills, Dr. Aiken said.

The Guttmacher report, focusing on the formal health care system, included data from clinics and telemedicine abortion services within the United States that provided abortion to patients who lived in or traveled to states with legal abortion between January and December 2023.

It found that pills accounted for 63 percent of those abortions, up from 53 percent in 2020. The total number of abortions in the report was over a million for the first time in more than a decade.

Why This Matters

Overall, the new reports suggest how rapidly the provision of abortion has adjusted amid post-Roe abortion bans in 14 states and tight restrictions in others.

The numbers may be an undercount and do not reflect the most recent shift: shield laws in six states allowing abortion providers to prescribe and mail pills to tens of thousands of women in states with bans without requiring them to travel. Since last summer, for example, Aid Access has stopped shipping medication from overseas and operating outside the formal health system; it is instead mailing pills to states with bans from within the United States with the protection of shield laws.

What’s Next

In the case that will be argued before the Supreme Court on Tuesday, the plaintiffs, who oppose abortion, are suing the Food and Drug Administration, seeking to block or drastically limit the availability of mifepristone, the first pill in the two-drug medication abortion regimen.

The JAMA study suggests that such a ruling could prompt more women to use avenues outside the formal American health care system, such as pills from other countries.

“There’s so many unknowns about what will happen with the decision,” Dr. Aiken said.

She added: “It’s possible that a decision by the Supreme Court in favor of the plaintiffs could have a knock-on effect where more people are looking to access outside the formal health care setting, either because they’re worried that access is going away or they’re having more trouble accessing the medications.”

Pam Belluck is a health and science reporter, covering a range of subjects, including reproductive health, long Covid, brain science, neurological disorders, mental health and genetics. More about Pam Belluck