Conv-Attention: A Low Computation Attention Calculation Method for Swin Transformer

Abstract

Transformer networks have excellent performance in various different vision tasks, especially object detection. However, in practical applications, Transformer is difficult to use on-board due to its large computational complexity. In this paper, we propose a new approach for reducing the computation of self-attention, which is called conv-attention. Different from the work of self-attention, conv-attention is inspired by NetVLAD and adopts the probability obtained by soft classification to replace the similarity calculation between query and key. Moreover, we combine the three convolution operations for computing the attention matrix in order to reduce the computational effort. Using the Swin Transformer as a comparison, experiments show that the parameters and FLOPs are reduced by 15% and 16%. Meanwhile, MAP is improved in vehicle object detection, including both FILR and Nuscenses datasets.