Dual Path Interaction Network for Video Moment Localization

51Citations
Citations of this article
28Readers
Mendeley users who have this article in their library.
Get full text

Abstract

Video moment localization aims to localize a specific moment in a video by a natural language query. Previous works either use alignment information to find out the best-matching candidate (i.e., top-down approach) or use discrimination information to predict the temporal boundaries of the match (i.e., bottom-up approach). Little research has taken both the candidate-level alignment information and frame-level boundary information together and considers the complementarity between them. In this paper, we propose a unified top-down and bottom-up approach called Dual Path Interaction Network (DPIN), where the alignment and discrimination information are closely connected to jointly make the prediction. Our model includes a boundary prediction pathway encoding the frame-level representation and an alignment pathway extracting the candidate-level representation. The two branches of our network predict two complementary but different representations for moment localization. To enforce the consistency and strengthen the connection between the two representations, we propose a semantically conditioned interaction module. The experimental results on three popular benchmarks (i.e., TACoS, Charades-STA, and Activity-Caption) demonstrate that the proposed approach effectively localizes the relevant moment and outperforms the state-of-the-art approaches.

Cite

CITATION STYLE

APA

Wang, H., Zha, Z. J., Chen, X., Xiong, Z., & Luo, J. (2020). Dual Path Interaction Network for Video Moment Localization. In MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia (pp. 4116–4124). Association for Computing Machinery, Inc. https://doi.org/10.1145/3394171.3413975

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free