ChatGPT reflects student misconceptions in physics

Abstract

With many students turning to machine learning models such as ChatGPT for homework help, diagnosing the accuracy of the information these technologies purport across different fields is more important than ever. In this study, we find that ChatGPT’s responses to standard physics questions are consistent with an accurate theoretical understanding, but are often incorrect when applied to a given system, reflecting many of the same misconceptions as students.