The trustworthy qualitative data tangle

When evaluating our qualitative research data - our evidence - we need to consider what our standards for quality will be... but the terms 'validity' and 'reliability' are not necessarily terms which are used with qualitative approaches.

That is not to say that qualitative researchers should not seek validity and reliability in our research work: it is to say that qualitative terms, meanings and methods seem differ from those used in quantitative work. Validity is a term which Bazeley uses (2013, p. 58, citing Maxwell, 2013), in outlining five key elements required for a sound qualitative research design as "purpose, conceptual framework, research questions, methods, and approach to validity", later implying that 'internal validity' is synonymous with trustworthiness. The biggest proponents of the shift in terminology are Guba & Lincoln (1985, 1989, 2017), who get quite excited about trustworthiness. Further, Bazeley implies that trustworthiness may have a different meaning to validity as she also mentions that qualitative research requires "rigour, reliability, credibility, trustworthiness, and validity" (Bazeley, 2021, p. 492).

So what do these terms mean in qualitative research? Thus far it seems that qualitative researchers are yet to come to a cohesive agreement on what those terms are, and how to define them in the field (Morse, 2015b). In fact, it becomes a tangled web, as illustrated here:

• Rigour. Used synonymously with trustworthiness "to evaluate the credibility, transferability, dependability" of research outcomes (Morse, 2015b, p. 1212, citing Guba & Lincoln, 1989). First tangle.
• Reliability. Or 'dependability': "Attainable through credibility, the use of “overlapping methods” (triangulation), “stepwise replication” (splitting data and duplicating the analysis), and use of an “inquiry audit” (p. 317) or audit trail" (Morse, 2015b, p. 1212, citing Guba & Lincoln, 1985, 1989). Second tangle with credibility.
• Credibility. Third tangle. We can see from above that credibility is related to rigour. This is also used synonymously with quality (Corbin & Strauss, 2014). Fourth tangle: credibility implies 'trustworthiness' in findings where they accurately reflect participant and researcher experiences (Corbin & Strauss, 2014). This can be termed "internal validity [...]: [p]rolonged engagement, persistent observation, triangulation, peer debriefing, negative case analysis, referential adequacy, and member checks" (Morse, 2015b, p. 1212, citing Guba & Lincoln, 1989). Fifth tangle (with validity).
• Trustworthiness. Apparently synonymous with rigour. OK... sixth tangle.
• Validity. Aka 'transferability' or 'generalisability'; and really related to external validity. "Thick description is essential for 'someone interested' (p. 316) to transfer the original findings to another context, or individuals" (Morse, 2015b, p. 1212-13, citing Guba & Lincoln, 1989)

Morse (2015b) gets stroppy and points out the idiocy of all this, suggesting that we just step back into the current 'quantitative' terminology and stop all this faffing about:

1. Validity. "(or internal validity) is usually defined as, the 'degree to which inferences made in a study are accurate and well-founded' (Polit & Beck, 2012, p. 745). Miller (2008b) further defined it as 'the ‘goodness’ or ‘soundness’ of a study' (p. 909). In qualitative inquiry, this is usually 'operationalized' by how well the research represents the actual phenomenon" (Morse, 2015b, p. 1213)
2. Reliability. "'broadly described as the dependability, consistency, and/or repeatability of a project’s data collection, interpretation, and/or analysis' (Miller, 2008a, p. 745). Basically, it is the ability to obtain the same results if the study were to be repeated" (Morse, 2015b, p. 1213)
3. Generalisability. "(or external validity) is 'extending the research results, conclusions, or other accounts that are based on the study of particular individuals, setting, times or institutions, to other individuals, setting, times or institutions than those directly studied' (Maxwell & Chmiel, 2014; Polit & Beck, 2012). In qualitative inquiry, the application of the findings to another situation or population is achieved through decontextualization and abstraction of emerging concepts and theory" (Morse, 2015b, p. 1213)
4. Rigour. If the three elements above are present, then the study should be rigorous. "Both criteria of reliability and validity are intended to make qualitative research rigorous (formerly referred to as trustworthy). Through particular representation, abstraction, and theory development, validity enables qualitative theories to be generalizable and useful when recontextualized and applied to other settings. Reliability makes replication possible, although qualitative researchers themselves recognize induction is difficult" (Morse, 2015b, p. 1213).

It is all research, after all, whether it be qualitative or quantitative. Why not expand the definitions so that they cover both fields?

Sam

References:

Bazeley, P. (2013). Qualitative Data Analysis: Practical strategies (1st ed.). SAGE Publications Ltd.

Bazeley, P. (2021). Qualitative Data Analysis: Practical strategies (2nd ed.). SAGE Publications Ltd.

Corbin, J., & Strauss, A. (2014). Basics of qualitative research techniques and procedures for developing grounded theory (4th ed.). SAGE Publications Ltd.

Denzin, N. K., & Lincoln, Y. S. (Eds.) (2017). The SAGE Handbook of Qualitative Research (5th ed.). SAGE Publications, Inc.

Guba, E. G., & Lincoln, Y. S. (1985). Naturalistic Inquiry. Sage Publications, Inc.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth Generation Evaluation. Sage Publications, Inc.

Morse, J. M. (2015a). Analytic Strategies and Sample Size. Qualitative Health Research, 25(10), 1317-1318. https://doi.org/10.1177/1049732315602867

Morse, J. M. (2015b). Critical Analysis of Strategies for Determining Rigor in Qualitative Inquiry. Qualitative Health Research, 25(9), 1212–1222. https://doi.org/10.1177/104973231558850