Data-Analytic Strategies
- Describe the methods and procedures used and for what purpose/goal.
- Explicate in detail the process of analysis, including some discussion of the procedures (e.g., coding, thematic analysis, etc.) with a principle of transparency.
- Describe coders or analysts and their training, if not already described in the researcher description section (e.g., coder selection, collaboration groups).
- Identify whether coding categories emerged from the analyses or were developed a priori.
- Identify units of analysis (e.g., entire transcript, unit, text) and how units were formed, if applicable.
- Describe the process of arriving at an analytic scheme, if applicable (e.g., if one was developed before or during the analysis or was emergent throughout).
- Provide illustrations and descriptions of their development, if relevant.
- Indicate software, if used.
Methodological Integrity
The task force that developed JARS-Qual defines methodological integrity and its component processes as follows:
Methodological integrity can be evaluated through its two composite processes: fidelity to the subject matter and utility in achieving research goals. … Fidelity to the subject matter is the process by which researchers select procedures that develop and maintain allegiance to the phenomenon under study as it is conceived within their approach to inquiry (e.g., the phenomenon might be understood as a social construction). … The second composite process of methodological integrity, utility in achieving research goals is the process by which researchers select procedures that usefully answer their research questions and address their aims (e.g., raising critical consciousness, developing theory, deepening understanding, identifying social practices, forming conceptual frameworks, and developing local knowledge). (Levitt et al. [2018], Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board Task Force report. American Psychologist, 73(1), 26-46. P. 33).
- Demonstrate that the claims made from the analysis are warranted and have produced findings with methodological integrity. The procedures that support methodological integrity (i.e., fidelity and utility) typically are described across the relevant sections of a paper, but they could be addressed in a separate section when elaboration or emphasis would be helpful. Issues of methodological integrity include:
- Assess the adequacy of the data in terms of its ability to capture forms of diversity most relevant to the question, research goals, and inquiry approach.
- Describe how the researchers' perspectives were managed in both the data collection and analysis (e.g., to limit their effect on the data collection, to structure the analysis).
- Demonstrate that findings are grounded in the evidence (e.g., using quotes, excerpts, or descriptions of researchers' engagement in data collection).
- Demonstrate that the contributions are insightful and meaningful (e.g., in relation to the current literature and the study goal).
- Provide relevant contextual information for findings (e.g., setting of study, information about participant, presenting the interview question asked prior to presenting the excerpt, as needed).
- Present findings in a coherent manner that makes sense of contradictions or disconfirming evidence in the data (e.g., reconcile discrepancies, describe why a conflict might exist in the findings).
- Demonstrate consistency with regard to the analytic processes (e.g., analysts may use demonstrations of analyses to support consistency, describe their development of a stable perspective, interrater reliability, consensus) or describe responses to inconsistencies, as relevant (e.g., coders switching mid-analysis, an interruption in the analytic process). If alterations in methodological integrity were made for ethical reasons, explicate those reasons and the adjustments made.
- Describe how support for claims was supplemented by any checks added to the qualitative analysis. Examples of supplemental checks that can strengthen the research may include:
- Transcripts/data collected returned to participants for feedback.
- Triangulation across multiple sources of information, findings, or investigators.
- Checks on the interview thoroughness or interviewer demands.
- Consensus or auditing process.
- Member checks or participant feedback on findings.
- Data displays/matrices.
- In-depth thick description, case examples, illustrations.
- Structured methods of researcher reflexivity (e.g., sending memos, field notes, diary, log books, journals, bracketing).
- Checks on the utility of findings in responding to the study problem (e.g., an evaluation of whether a solution worked).
