Hybrid-Optimized Model for Deepfake Detection

Abstract

The advancement of deep learning models has led to the creation of novel techniques for image and video synthesis. One such technique is the deepfake, which swaps faces among persons and then produces hyper-realistic videos of individuals saying or doing things that they never said or done. These deepfake videos pose a serious risk to everyone and countries if they are exploited for extortion, scamming, political disinformation, or identity theft. This work presents a new methodology based on a hybrid-optimized model for detecting deepfake videos. A Mask Region-based Convolutional Neural Network (Mask R-CNN) is employed to detect human faces from video frames. Then, the optimal bounding box representing the face region per frame is selected, which could help to discover many artifacts. An improved Xception-Network is proposed to extract informative and deep hierarchical representations of the produced face frames. The Bayesian optimization (BO) algorithm is employed to search for the optimal hyperparameters' values in the extreme gradient boosting (XGBoost) classifier model to properly discriminate the deepfake videos from the genuine ones. The proposed method is trained and validated on two different datasets; CelebDF-FaceForencics++ (c23) and FakeAVCeleb, and tested also on various datasets; CelebDF, DeepfakeTIMIT, and FakeAVCeleb. The experimental study proves the superiority of the proposed method over the state-of-the-art methods. The proposed method yielded %97.88 accuracy and %97.65 AUROC on the trained CelebDF-FaceForencics++ (c23) and tested CelebDF datasets. Additionally, it achieved %98.44 accuracy and %98.44 AUROC on the trained CelebDF-FaceForencics++ (c23) and tested DeepfakeTIMIT datasets. Moreover, it yielded %99.50 accuracy and %99.21 AUROC on the FakeAVCeleb visual dataset.