Detecting Cheapfakes using Self-Query Adaptive-Context Learning

Abstract

Detecting Cheapfakes often requires identifying contextual changes in media resulting from misleading captions. Cheapfake media can be created either by manipulating the content using image or video editing software, or by altering the context of an image or video through misleading claims, without relying on any software. While previous research has shown promising results, these approaches are limited to the data used during training. To overcome this limitation, we propose a Self-Query Adaptive-Context Learning method that is flexible and capable of adapting to new contexts during inference by using image search engine queries to enrich its knowledge. By verifying the context of captions based on the collected information, our approach extends knowledge in a flexible manner. Despite achieving an experimental accuracy of 59.70% on the IEEE ICME 2023 Cheapfakes Challenge dataset, our work has opened up new avenues for detecting out-of-context misuses.