Leveraging Cross-Modals for Cheapfakes Detection

Abstract

Detecting cheapfakes, particularly out-of-context images, is crucial for maintaining the integrity of information and preserving trust in multimedia content. This study proposes a new Cross-modals approach for cheapfakes detection that blends Natural Language Processing and Computer Vision techniques, including Image Captioning, Name Entity Recognition, and Natural Language Inference. Our approach enhances the robustness of the Cross-modals methods by not only considering both the meaning and context of the image titles but also using additional information such as Name Entity. According to the experiments, our method achieved an accuracy of , which can improve previous approaches. This paper highlights the potential of combining Natural Language Processing and Computer Vision techniques to tackle real-world problems, making it a significant advancement in cheapfakes detection.