Effect of self-invertible matrix on cipher hexagraphic polyfunction

Abstract

A cryptography system was developed previously based on Cipher Polygraphic Polyfunction transformations, C(t) i×j≡ Ati×iPi×j mod N where Ci×j, Pi×j, Ai×i are cipher text, plain text, and encryption key, respectively. Whereas, (t) is the number of transformations of plain text to cipher text. In this system, the parameters (Ai×i, (t)) are kept in secret by a sender of messages. The security of this system, including its combination with the second order linear recurrence Lucas sequence (LUC) and the Ron Rivest, Adi Shamir and Leonard Adleman (RSA) method, until now is being upgraded by some researchers. The studies found that there is some type of self-invertible A4×4 should be not chosen before transforming a plain text to cipher text in order to enhance the security of Cipher Tetragraphic Trifunction. This paper also seeks to obtain some patterns of self-invertible keys A6×6 and subsequently examine their effect on the system of Cipher Hexagraphic Polyfunction transformation. For that purpose, we need to find some solutions L3×3 for L23×3≡ A3×3 mod N when A3×3 are diagonal and symmetric matrices and subsequently implement the key L3×3 to get the pattern of A6×6 .