Toward a mathematical understanding of the malware problem

Abstract

Malware plays a significant role in breaching computer systems. Previous research has focused on malware detection even though detection is up against theoretical limits in computer science and current methods are inadequate in practice. We explain the susceptibility of computation to malware as a consequence of the instability of Turing and register machine computation. The behavior of a register machine program can be sabotaged, by making a very small change to the original, uninfected program. Stability has been studied extensively in dynamical systems and in engineering fields such as aerospace. Our primary contribution introduces mathematical tools from topology and dynamical systems to explain why register machine computation is susceptible to malware sabotage. A correspondence is constructed such that one computational step of a Turing machine maps to one iteration of a dynamical system in the x-y plane and vice versa. Using this correspondence, another contribution defines and demonstrates a structural instability in a Universal Turing machine encoding. One research direction proposes to better understand instability in conventional computation by studying non-isolated metrics on the space of Turing machines; another suggests searching for stable computation in unconventional machines.