Cross-lingual effects of AI-generated content on human work

Abstract

Artificial intelligence (AI) technologies, especially large language models (LLMs), have permeated human work around the globe, but how effective is workers’ usage and application of AI-generated content across language settings? This research examines the quality of (1) AI-generated business recommendations in English, Arabic, and Chinese and (2) business emails written by human participants in those languages. In Study 1, trained evaluators rated the quality of AI-generated content. Study 2 is a randomized experiment in which 480 human participants (160 in each language setting) wrote emails to address business issues, with or without the help of AI-generated content. Trained evaluators then rated the quality of those emails. This research finds that the AI-generated responses were of lower quality in Arabic and Chinese than in English. Importantly, human participants’ usage of AI-generated contentin email-writing was associated with less actionable and less creative work output in Arabic and Chinese than in English. Non-English participants were particularly disadvantaged when dealing with more technical work tasks (e.g., tasks related to scientific discovery or product design). These findings underscore the need for more language-inclusive LLMs to support workers worldwide, as current technologies may inadvertently widen the productivity gap between English and non-English speakers, particularly in more technical domains.