Recognition of crowd abnormal activities using fusion of handcrafted and deep features

Abstract

Constant vigilance is extremely important at crowded public places where some unusual activities such as sudden dispersion or continuous gathering of people may lead to chaotic and disastrous situations. Automatic recognition of such collective activities of people is indeed an important task to ensure people safety. In this view, we propose a novel approach for automatic recognition of crowd merging and sudden dispersion events. The proposed method detects dispersion and merging using fusion of features extracted by deep networks with a novel set of optical flow based and density based handcrafted features. These proposed features are not affected by occlusion and illumination. These features complement the features extracted by deep network and their fusion improves the performance of event recognition by significant amount. The method is tested profoundly on benchmark public datasets as well as private datasets. Abnormal activity recognition data often suffers from high class imbalance. However, the proposed method could successfully recognize sudden dispersion and merging activities on such very small datasets having class imbalance. This proves the effectiveness and robustness of the proposed features. The proposed method also shows better performance than other state of the art methods based on deep networks.