Enhancing Product Detection with Multicue Optimization for TV Shopping Applications

Abstract

Smart TVs allow consumers to watch TV, interact with applications, and access the Internet, thus enhancing the consumer experience. However, the consumers are still unable to seamlessly interact with the contents being streamed, as it is highlighted by TV-enabled shopping. For example, if a consumer is watching a TV show and is interested in purchasing a product being displayed, the consumer can only go to a store or access the Web to make the purchase. It would be more convenient if the consumer could interact with the TV to purchase interesting items. To realize this use case, products in the content stream must be detected so that the TV system notifies consumers of possibly interesting ones. A practical solution must address the detection of complex products, i.e., those that do not have a rigid form and can appear in various poses, which poses a significant challenge. To this end, a multicue product detection framework is proposed for TV shopping. The framework is generic as it is not tied to specific object detection approaches. Instead, it utilizes appearance, topological, and spatio-temporal cues that make use of a related, easier to detect object class to improve the detection results of the target, more difficult product class. The three cues are jointly considered to select the best path that occurrences of the target product class can follow in the video and thus eliminate false positive occurrences. The empirical results demonstrate the advantages of the proposed approach in improving the precision of the results.