Is AI as trustworthy as human experts? The role of advice source, technology knowledge, and perceptions of AI’s impact in organizational decision-making

Abstract

The rapid digitisation of industries and the emergence of advanced AI tools have reshaped organisational dynamics. This study investigates factors influencing successful human-AI collaboration. In a randomised experiment with 403 participants using the Judge-Advisor System, participants received advice labelled as either AI-generated or from a human expert. Results showed no significant difference in the weight of advice (WOA) given to AI versus human experts, indicating comparable levels of behavioural appreciation. Additionally, individuals with greater knowledge of AI technology and more optimistic perceptions of AI’s impact placed higher weight on AI advice and exhibited increased decision confidence. These findings highlight the significant role of perceptual and emotional factors, along with technological literacy, in shaping individuals’ responses to AI advice. This research contributes to ongoing debates on algorithm aversion and appreciation, emphasises the importance of user training programs and provides indications for improving AI system design for optimising adoption in organisational settings.