Human Judgment in algorithmic loops: Individual justice and automated decision-making

Abstract

Arguments in favor of tempering algorithmic decision making with human judgment often appeal to concepts and criteria derived from legal philosophy about the nature of law and legal reasoning, arguing that algorithmic systems cannot satisfy them (but humans can). Such arguments often make implicit appeal to the notion that each case needs to be assessed on its own merits, without comparison to or generalization from previous cases. This article argues that this notion of individual justice can only be meaningfully served through human judgment. It distinguishes individual justice and considers how it relates to other dimensions of justice, namely consistency and fairness / nondiscrimination. Finally, it identifies and discussess two challenges: first, how individual justice can be accommodated alongside other dimensions of justice in the socio-technical contexts of humans-in-the-loop; and second, how inequities in individual justice may result from an uneven application of human judgment in algorithmic contexts.