Improving Object Detection with Consistent Negative Sample Mining

Abstract

In object detection, training samples are divided into negatives and positives simply according to their initial positions on images. Samples which have low overlap with ground-truths are assigned to negatives, and positives otherwise. Once allocated, the negative and positive set are fixed in training. A usually overlooked issue is that certain negatives do not stick to their original states as training proceeds. They gradually regress towards foreground objects rather than away from them, which contradicts the nature of negatives. Training with such inconsistent negatives may confuse detectors in distinguishing between foreground and background, and thus makes training less effective. In this paper, we propose a consistent negative sample mining method to filter out biased negatives in training. Specifically, the neural network takes the regression performance into account, and dynamically activates consistent negatives which have both low input IoUs and low output IoUs for training. In the experiments, we evaluate our method on PASCAL VOC and KITTI datasets, and the improvements on both datasets demonstrate the effectiveness of our method.