Densely supervised hierarchical policy-value network for image paragraph generation

14Citations
Citations of this article
19Readers
Mendeley users who have this article in their library.

Abstract

Image paragraph generation aims to describe an image with a paragraph in natural language. Compared to image captioning with a single sentence, paragraph generation provides more expressive and fine-grained description for storytelling. Existing approaches mainly optimize paragraph generator towards minimizing word-wise cross entropy loss, which neglects linguistic hierarchy of paragraph and results in “sparse” supervision for generator learning. In this paper, we propose a novel Densely Supervised Hierarchical Policy-Value (DHPV) network for effective paragraph generation. We design new hierarchical supervisions consisting of hierarchical rewards and values at both sentence and word levels. The joint exploration of hierarchical rewards and values provides dense supervision cues for learning effective paragraph generator. We propose a new hierarchical policy-value architecture which exploits compositionality at token-to-token and sentence-to-sentence levels simultaneously and can preserve the semantic and syntactic constituent integrity. Extensive experiments on the Stanford image-paragraph benchmark have demonstrated the effectiveness of the proposed DHPV approach with performance improvements over multiple state-of-the-art methods.

Cite

CITATION STYLE

APA

Wu, S., Zha, Z. J., Wang, Z., Li, H., & Wu, F. (2019). Densely supervised hierarchical policy-value network for image paragraph generation. In IJCAI International Joint Conference on Artificial Intelligence (Vol. 2019-August, pp. 975–981). International Joint Conferences on Artificial Intelligence. https://doi.org/10.24963/ijcai.2019/137

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free