Research transparency, openness, rigour, and replicability

In the words of Tara Brabazon, "research is based on transparency, openness, rigour, and repeatability" (Brabazon, 2021, 28:50). Wow, now that is a key set of terms to consider research by!

1. Transparency: is hard to define. There are a number of criteria which have a "cumulative effect on the trustworthiness and replicability of knowledge" (Aguinis & Solarino, 2019, p. 1296), which the authors find equal transparency. These are "practices that increase methodological transparency and thereby increase the replicability of one’s research are essential for trustworthiness" (Pratt et al., 2020, p. 2), and a "means of assessing trustworthy research is via the transparency of its authors" (p. 3).
2. Openness: in order to foster greater transparency, many journals now have openness standards, requiring authors to upload their data, analysis and research plans in advance of their study. This includes "public sharing of materials and data" to hopefully "reduce the number of publications based on poor or even unethical practices" (Pratt et al., 2020, p. 2).
3. Rigour: this refers to the robustness of the data collection methods, analysis and interpretation. Journals are looking for studies to have been carefully planned, with the "key issues involved in designing, conducting, and reporting high quality quantitative [...] research" (Maula & Stam, 2020, p. 1060) well thought through. This equally applies to qualitative research, with the same markers of quality being sought within the paradigm.
4. Repeatability/reproductability/replicability: while the idea of replicability - or repeatability, or reproductability - is a cornerstone of quantitative scientific research, it too should hold true for qualitative research. We should be able to assume that, all other things being equal, the findings of one study - if undertaken following the methodology and methods of a previous study within a similar population group - should be similar to the findings of another. However, a number of high profile papers - all of which have been highly relied upon - have been proven to lack replicability (McVay & Conroy, 2021; Pratt et al., 2020). This is a problem, because if we cannot repeat the study and obtain reasonably similar results, then how much weight can we put on the original findings? The reasons range from post hoc hypothesising, falsification of data, and additional data collection when the findings do not show desired outcomes. Further, journals often won't publish studies which seek similar outcomes to previously published articles, cutting off the ability for academics to publish, and to get grant funding for verification/validation studies. We must note though that "replication does not make sense for qualitative, inductive studies" where 'trustworthiness is a more common measure of quality (Pratt et al., 2020, p. 4).

While these elements fit well with quantitative research, I feel we should at least consider viewing qualitative research through these four lenses as well. While the quantitative research field is not the same as the qualitative, I still think these four characteristics still have something to tell us.

Sam

References:

• Aguinis, H., & Solarino, A. M. (2019). Transparency and replicability in qualitative research: The case of interviews with elite informants. Strategic Management Journal, 40(8), 1291-1315. https://doi.org/10.1002/smj.3015
• Brabazon, T. (19 March 2021). Vlog 261 - The Porous PhD [video]. Office of Graduate Research Flinders University. https://youtu.be/xPhSjQ7NKXo
• Maula, M., & Stam, W. (2020). Enhancing Rigor in Quantitative Entrepreneurship Research. Entrepreneurship Theory and Practice, 44(6), 1059-1090. https://doi.org/10.1177/1042258719891388
• McVay, M. A., & Conroy, D. E. (2021). Transparency and openness in behavioral medicine research. Translational Behavioral Medicine, 11(1), 287-290. https://doi.org/10.1093/tbm/ibz154
• Pratt, M. G., Kaplan, S., & Whittington, R. (2020). Editorial essay: The tumult over transparency: Decoupling transparency from replication in establishing trustworthy qualitative research. Administrative Science Quarterly, 65(1), 1-19. https://doi.org/10.1177/0001839219887663