Framing Assessment Questions in the Age of Artificial Intelligence: Evidence from ChatGPT 3.5

Abstract

With the rise of artificial intelligence (AI), higher education faces a significant challenge in learning assessment. The emergence of tools like ChatGPT raises concerns regarding the potential for cheating and the reliability of assessment outcomes. This paper aims to address these concerns by proposing a methodology for framing questions that effectively measures learning outcomes while reducing the risk of AI-enabled cheating. To achieve this objective, we employ a methodological approach that involves getting responses from ChatGPT 3.5 to various question prompts across different domains. These responses are then evaluated by faculty members specializing in management education. Through this process, we aim to identify question-framing strategies that effectively assess learning outcomes while minimizing susceptibility to AI Cheating. Our analysis reveals several key findings. Certain question Types (Decision Making, Recent Events, and Experiential Learning) demonstrate greater resilience against AI-generated responses, indicating their potential effectiveness in assessing student learning. This study offers original insights into the challenges and opportunities associated with learning assessment in the context of AI integration. The paper tries to provide valuable guidance for Policymakers, educators & students seeking to enhance the integrity and reliability of their assessment practices.