ChatGPT or Human Mentors? Student Perceptions of Technology Acceptance and Use and the Future of Mentorship in Higher Education

Abstract

GAI technologies are increasingly recognized as mentor-like resources in higher education. While these tools offer academic guidance and personalized feedback, little is known about how students perceive and evaluate AI-generated mentorship. This study investigated how Prior ChatGPT Use, primary mentor identity, mentorship effectiveness, and technology acceptance predict students’ response identification and evaluations of AI- versus human-generated responses. College students (N = 127) completed a survey in which they identified the source of masked responses across different domains and rated each response on helpfulness, caring, and likelihood to reach out again. Binary logistic regression models revealed that Prior ChatGPT Use predicted greater accuracy in identifying AI-generated responses, while mentor identity did not. Multiple linear regression analyses showed that students’ evaluations were influenced by perceived response sources more than actual sources. Participants who viewed human mentorship as effective were less likely to seek support from AI-perceived responses, while those who found ChatGPT useful rated AI-perceived responses more favorably. Technology acceptance factors were positively correlated with ratings of AI-perceived responses. These findings suggest that students’ pre-existing biases shape engagement with AI more than content itself, highlighting the importance of perception and the need to promote AI literacy when integrating ChatGPT as a mentorship tool.