Multi-layer fusion neural network for deepfake detection

Abstract

Recently, the spread of videos forged by deepfake tools has been widely concerning, and effective ways for detecting them are urgently needed. It is known that such artificial intelligence-aided forgery makes at least three levels of artifacts, which can be named as microcosmic or statistical features, mesoscopic features, and macroscopic or semantic features. However, existing detection methods have not been designed to exploited them all. This work proposes a new approach to more effective detection of deepfake videos. A multi-layer fusion neural network (MFNN) has been designed to capture the artifacts in different levels. Features maps output from specially designed shallow, middle, and deep layers, which are used as statistical, mesoscopic, and semantic features, respectively, are fused together before classification. FaceForensic++ dataset was used to train and test the method. The experimental results show that MFNN outperforms other relevant methods. Particularly, it demonstrates more advantage in detecting low-quality deepfake videos.