Filtered nonlinear cryptanalysis of reduced-round serpent, and the wrong-key randomization hypothesis

Abstract

We present a deterministic algorithm to find nonlinear S-box approximations, and a new nonlinear cryptanalytic technique; the "filtered" nonlinear attack, which achieves the lowest data complexity of any known-plaintext attack on reduced-round Serpent so far. We demonstrate that the Wrong-Key Randomization Hypothesis is not entirely valid for attacks on reduced-round Serpent which rely on linear cryptanalysis or a variant thereof, and survey the effects of this on existing attacks (including existing nonlinear attacks) on 11 and 12-round Serpent. © 2013 Springer-Verlag Berlin Heidelberg.