Cross-domain and cross-modality transfer learning for multi-domain and multi-modality event detection

Abstract

Online news media and social media are popular domains for people to acquire real-world event knowledge. In this work, the problem of multi-domain and multi-modality event detection (MMED) is elaborated. We wish to organize the multi-modality data from multiple domains based on real-world events. To this end, a cross-domain and cross-modality transfer learning (CDM) model is proposed. The CDM model aligns the data by exploiting a dictionary-based alignment strategy, and identifies the event labels of the data samples based on the class-specific reconstruction residual. Extensive experiments conducted on real-world data demonstrate the effectiveness of the proposed models. In particular, a benchmark dataset, denoted as MMED100, is released, which can hopefully be used to promote the research on this topic and advance related applications.