How Our AI-Assisted Qualitative Analysis Failed

Abstract

The authors describe how their Generative Artificial Intelligence (GAI)-Assisted qualitative research project failed to produce publishable results. Based on this experience, they argue for the value of embracing and reflecting on failure in GAI-Assisted qualitative research. To frame this argument, they draw on two theories of generative failure: failing forward, which uses failures to iterate on designs to meet existing criteria, and failing sideways, which reconsiders the criteria for success. Using a fail-forward perspective, the authors describe how they might revise their research methods for data preparation, process documentation, and task delegation to create more reliable results. Then, using a fail-sideways perspective, they reexamine criteria for publishable results to reimagine the study more fundamentally.