Shifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services

Abstract

Calls for responsible design in algorithmic decision-support systems, especially those used in public services, are increasingly common. While an algorithmic system might promise greater precision and efficiency in domains such as critical care, the same efficiency is difficult to replicate in the public service domain, where caseworkers must exercise discretion in applying complex legal frameworks that directly affect individual lives. In this paper we examine the challenges in responsibly designing such an algorithmic decision-support system. We report findings from a large cross-disciplinary research project, aiming to develop an algorithmic component for municipal decision-support systems for job placement in Denmark. Our data showcases insights from how a team of data scientists, caseworkers, and system developers negotiated notions of value metrics and usefulness in a participatory design set-up. Whereas data scientists expected to focus on profiling individuals, the caseworkers instead pushed for systems that could help mitigate organizational contradictions and obscured processes in casework. We close with a discussion of challenges and future directions for participatory algorithmic systems design in municipal contexts.