Layered ontological image for intelligent interaction to extend user capabilities on multimedia systems in a folksonomy driven environment

Abstract

This chapter describes a method for enabling an ontological interaction on video clip shown on ubiquitous systems as a computer monitor, mobile, or tablet. We use a layered representation based on semantic texton forests to obtain spatiotemporal object attributes. The interface is created by extracting object information from the video with a human based computation to obtain a richer semantics of attribute to bridge the semantic gap between words describing an image and its visual features. Users can navigate and manipulate objects displayed on video by associating semantic attributes and comments evaluated by the data and sentiment extraction. Folksonomy tags are extracted from users’ comments to be used in a dynamical driven system (Folksodriven). This chapter documents the advantages on a case application for advertisement inside the objects displayed on a video and argues how the method proposed may be preferred to the more traditional video advertisement.