CLFM: Few-Shot Object Detection via Low-Resource Contrastive Learning and Fisher Matrix Updating for Overcoming Catastrophic Forgetting

Abstract

Few-shot object detection (FSOD) aims to efficiently detect novel instances by model transferring using a few novel-class samples after the base-class samples are pre-trained. However, catastrophic forgetting occurs when FSOD transfers to the novel-classes, making the transferred model unable to accurately detect base and novel class instances simultaneously. Thus, this paper attempts to extend the general elastic weight curing (EWC) to the field of few-shot transfer detection. An online soft constraint is applied by evaluating the Fisher information matrix of inner-batch samples based on mean squared error (MSE) metric to constrain or encourage model parameters transferring. Also, a momentum update based inter-batch storage mechanism is proposed to alleviate the memory strain caused by the previously applied contrastive learning modules when performing numerous contrastive encoding procedures. Finally, a compact heterogeneous decoupled-head is formulated to remove information entanglement between regression and classification tasks. The experimental results validate that on the PASCAL VOC and COCO datasets, compared with the FSOD baselines of original contrastive learning, the proposed method improves the average accuracy of the novel and the base classes by 1.3%15.1% and 7% separately, both of which reach the SOTA level.