Artificial Intelligence’s Capabilities, Limitations, and Impact on Accounting Education: Investigating ChatGPT’s Performance on Educational Accounting Cases

Abstract

This study evaluates the capabilities of ChatGPT models 3.5 and 4 to provide solutions to seven educational accounting cases. We find that ChatGPT’s ability to provide accurate solutions varies depending on the type of case requirement, with better performance on tasks requiring elements that require explanation, application of rules, and ethical evaluation using a framework. However, ChatGPT performs relatively poorly on tasks that require financial statement creation, journal entries, or software use. Our study also finds that detection tools provided by ChatGPT’s developer are ineffective in identifying text created by artificial intelligence text generators (AITG). These quantitative results, although limited in generalizability, illustrate the current “state of the art” and allow us to suggest ways in which instructors can structure assignments to reduce the effectiveness of AITGs in subverting the learning process and ways in which instructors can incorporate AITGs into assignment requirements to help students attain desired educational outcomes.