Nowotny, Helga (2021). In AI we trust: power, illusion and control of predictive algorithms, Polity, Cambridge, UK, ISBN-13: 978-1509548811

Abstract

This thought-provoking synthesis of progress and the narrative of living with predictive futures should be of interest to those engaged in this field as researchers, students, policy shapers or observers of events. In setting a narrative on In AI we trust, Helga Nowotny argues for the cultivation of wisdom of 'cathedral thinking' to meet the challenges of co-evolutionary trajectory of interdependence between humankind and the digital machine. This she says requires rethinking of the techno-centric narrative of progress, embracing and harnessing uncertainty, and abandoning the fantasy of control over nature and the illusion of techno-centric dominance of smart AIs. This 'cathedral thinking' of narrative of progress requires, says Nowotny, the cultivation of institutional cultures that look forward and backward at the same time, whilst seeking a balance between the urgency of the moment and the long-term view. Any idea of embedding such a cathedral thinking in smart AIs needs to recognise that smart AIs designed for delivering predetermined goals cannot go beyond the quantifiable criteria of efficiency to reach the tacit level human cathedral thinking. And, further, these AIs are limited by their roots in quantification to deal with the richness of human conversation that involves tacit traits of ambiguity, uncertainty, non-verbal cues, and silence. Moreover, the wisdom of cathedral thinking lies in the practice of 'an ethos' that finds ways to tap into knowledge of the past and resources of the present to guide the design of new institutions to meet the societal needs of the present and respond to yet unknown futures. In essence, the author says, we need wisdom 'that acknowledges the limitations of digital technologies and guard against the illusion of control', and against the crave for predictive certainty of the future. Nowotny says that in this illusion and crave lies a 'Paradox' of predictive algorithms , in the sense that the more we crave for future, the more we ignore what the predictions do to us. The more we crave for certainty, the more we seek assurances from algorithmic predictions to cope with uncertainty, and thus more we crave for control of our predetermined futures. But our futures are full of uncertainties, unknowns, ambiguities and algorithmic predictions of bringing the future into the present, confront the past in the predicting the future. In seeking certainty in algorithmic predictions, we are in danger of 'renouncing the inherent uncertainty of the future and replacing it with the dangerous illusion of being in control'. There is also a tacit assumption and misplaced confidence that smart AIs would ultimately take care of the unresolved ethical, transparency and accountability conflicts when we are able to develop computational tools 'to assess the performance and output quality of deep learning algorithms and to optimise their training'. The danger is that 'we end up trusting the automatic pilot while flying blindly in the fog', becoming part of a fine-tuned and interconnected predictive system, thereby diminishing our motivation and ability to stretch the boundaries of imagination. The challenge is how to deactivate the automatic pilot and exercise our own judgment of our action and this goes for institutions when they begin to align their performance with predictive algorithms, often unaware of the unintended consequences. The implication of this alignment is that even public institutions are being allured to governance by numbers in the guise of the umbrella term, 'objectivity'. Nowotny notes that whilst in the recent past systematic management of uncertainty of the natural and social world gave at least a feeling of human being in control of modernity, now predictive analytics are taking over even that human control as tools of management of new uncertainties of the digital world, promising objectivity and efficiency. For example, predictive algorithms are already replacing 'human decision-making in the delivery of public and private services, in the decisions