GROOViST: A Metric for Grounding Objects in Visual Storytelling

0Citations
Citations of this article
8Readers
Mendeley users who have this article in their library.

Abstract

A proper evaluation of stories generated for a sequence of images-the task commonly referred to as visual storytelling-must consider multiple aspects, such as coherence, grammatical correctness, and visual grounding. In this work, we focus on evaluating the degree of grounding, that is, the extent to which a story is about the entities shown in the images. We analyze current metrics, both designed for this purpose and for general vision-text alignment. Given their observed shortcomings, we propose a novel evaluation tool, GROOViST, that accounts for cross-modal dependencies, temporal misalignments (the fact that the order in which entities appear in the story and the image sequence may not match), and human intuitions on visual grounding. An additional advantage of GROOViST is its modular design, where the contribution of each component can be assessed and interpreted individually.

Cite

CITATION STYLE

APA

Surikuchi, A. K., Pezzelle, S., & Fernández, R. (2023). GROOViST: A Metric for Grounding Objects in Visual Storytelling. In EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings (pp. 3331–3339). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.emnlp-main.202

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free