Using AI to generate formative feedback in doctoral education

Abstract

This paper examines how artificial intelligence (AI) tools enhance feedback practices in doctoral education by providing a supplementary source of formative assessment. It explores the use of Generative AI alongside Grainger’s Formative Assessment Criteria-Based Tool (F.A.C.T.) in a single-subject case study of a pre-confirmation doctoral candidate at an Australian university. The study employs a naturalistic and interpretive approach with a sequential design, exploring interactions between a doctoral student and ChatGPT across multiple sessions where the AI tool evaluated a pre-confirmation thesis. Data collection included deidentified summarised feedback received from AI and an independent academic reviewer. Findings revealed that AI-generated feedback, guided by Grainger’s F.A.C.T. demonstrated thematic alignment with a human reviewer in identifying areas needing improvement, particularly regarding theoretical foundation and contribution to knowledge. However, the human reviewer provided contextually nuanced and discipline-specific feedback with more actionable suggestions. The study illustrates how, when coupled with formative rubrics, generative AI may serve as a supplementary feedback mechanism that may reduce power imbalances inherent in supervisor/reviewer-student relationships while providing expedient formative feedback. This research contributes to understanding how reflective practice in doctoral education may be enhanced through AI integration, addressing gaps in feedback literacy, socialisation, and standardised assessment parameters in doctoral contexts.