A Generative Adaptive Context Learning Framework for Large Language Models in Cheapfake Detection

Abstract

Cheapfakes, also known as manipulated media or deceptive content, refer to digital creations that have been altered or fabricated with the intention to deceive or mislead. These can include photos, videos, audio recordings, or any form of media that has been manipulated in a way that distorts its original meaning or context using non-AI techniques. The "ACM ICMR 2024 Grand Challenge on Detecting Cheapfakes" has brought attention to the issue of identifying out-of-context misuse, which can greatly assist fact-checkers in their work. To address this challenge, this paper introduces a method to create the training dataset (i.e image-caption triplets dataset) and a novel approach for detecting cheapfakes using four main components (i.e., Natural Language Inference, Context Generation, Prompt Engineering and Out-of-context Classification). In the testing process, we achieved a result with an accuracy of 88.9% on the public test from COSMOS dataset, which is 6.9% higher than the COSMOS baseline. These results demonstrate the potential of our method as a reliable tool for detecting cheapfakes and aiding in the fight against misinformation.