Multimedia integration for cooking video indexing

Abstract

We have been working on the integration of video with supplementary documents, such as cooking programs. We propose an integration system that performs semantic segmentations of video and text and associates them together. This association is realized using the ordinal restriction of the recipe, cooccurrences of words in the text and the audio in the video, and the relation between the background in a video and words which describe the situation in a text. In this paper, we will introduce the result of an evaluation experiment and show the effectiveness of the proposed integration method. Through our method, many applications should become possible, such as a cooking navigation software. © Springer-Verlag Berlin Heidelberg 2004.