Extrapolation and AI transparency: Why machine learning models should reveal when they make decisions beyond their training

Abstract

The right to artificial intelligence (AI) explainability has consolidated as a consensus in the research community and policy-making. However, a key component of explainability has been missing: extrapolation, which can reveal whether a model is making inferences beyond the boundaries of its training. We report that AI models extrapolate outside their range of familiar data, frequently and without notifying the users and stakeholders. Knowing whether a model has extrapolated or not is a fundamental insight that should be included in explaining AI models in favor of transparency, accountability, and fairness. Instead of dwelling on the negatives, we offer ways to clear the roadblocks in promoting AI transparency. Our commentary accompanies practical clauses useful to include in AI regulations such as the AI Bill of Rights, the National AI Initiative Act in the United States, and the AI Act by the European Commission.