Residual Vector Capsule: Improving Capsule by Pose Attention

Abstract

The convolutional neural network has significantly improved the accuracy of image recognition; however, it performs in a fragile manner when we apply viewpoint transformation or add noise to the image. Recent studies have proposed new neural networks named capsule. Capsule build the part-whole relationship in the entity through instantiation parameters, and they cluster instantiation parameters layer by layer through routing-by-agreement; therefore, capsule has stronger representational ability and robustness than convolutional neural networks. However, the routing-by-agreement of the capsule network is limited by the prior probability assumption, which performs in an unstable way in recognition accuracy and robustness. To remove the restriction of the prior probability assumption in the routing-by-agreement, we propose a new capsule named the residual vector capsule (RVC), which constructs the routing-by-agreement with self-attention. The experimental results show that compared with other capsule networks, RVC achieves competitive classification accuracy on MNIST, Fashion-MNIST, CIFAR-10 and SVHN, improves the viewpoint invariance of the model on SmallNorb, and significantly improves the robustness of the model against white box attacks on CIFAR-10 and SVHN.