Using evolutionary learning of behavior to find weaknesses in operating systems

Abstract

System security is an ongoing struggle between system designers and the hacking community. Human creativity within this community pushes software into areas never anticipated by the designers, thus revealing weaknesses. Evolutionary algorithms offer designers a new way to examine the viability of their code. Because of the use of randomness as well as direction based on evaluation, these algorithms help to simulate some aspects of the human creative process. In this work we show that already rather simple evolutionary searches allow us to find weaknesses in an operating system, a Linux version, resulting in a crash of the system and the necessity to reboot - a serious system flaw and security risk. © Springer-Verlag Berlin Heidelberg 2004.