Potential weaknesses of the commutator key agreement protocol based on braid groups

Abstract

The braid group with its conjugacy problem is one of the recent hot issues in cryptography. At CT-RSA 2001, Anshel, Anshel, Fisher, and Goldfeld proposed a commutator key agreement protocol (KAP) based on the braid groups and their colored Burau representation. Its security is based on the multiple simultaneous conjugacy problem (MSCP) plus a newly adopted key extractor. This article shows how to reduce finding the shared key of this KAP to the list-MSCPs in a permutation group and in a matrix group over a finite field. We also develop a mathematical algorithm for the MSCP in braid groups. The former implies that the usage of colored Burau representation in the key extractor causes a new weakness, and the latter can be used as a tool to investigate the security level of their KAP.