Limits and Possibilities for "Ethical AI" in Open Source: A Study of Deepfakes

Abstract

Open source software communities are a significant site of AI development, but "Ethical AI"discourses largely focus on the problems that arise in software produced by private companies. Design, policy and tooling interventions to encourage "Ethical AI"based on studies in private companies risk being ill-suited for an open source context, which operates under radically different organizational structures, cultural norms, and incentives. In this paper, we show that significant and understudied harms and possibilities originate from differing practices of transparency and accountability in the open source community. We conducted an interview study of an AI-enabled open source Deepfake project to understand how members of that community reason about the ethics of their work. We found that notions of the "Freedom 0"to use code without any restriction, alongside beliefs about technology neutrality and technological inevitability, were central to how community members framed their responsibilities, and the actions they believed were and were not available to them. We propose a continuum between harms resulting from how a system is implemented versus how it is used, and show how commitments to radical transparency in open source allow great ethical scrutiny for harms wrought by implementation bugs, but allow harms through (mis)use to proliferate, requiring a deeper toolbox for disincentivizing harmful use. We discuss how an assumption of control over downstream uses is often implicit in discourses of "Ethical AI", but outline alternative possibilities for action in cases such as open source where this assumption may not hold.