Show me how to tie a tie: Evaluation of cross-lingual video retrieval

Abstract

In this study we investigate the potential of cross-lingual video retrieval for how-to questions. How-to questions are the most frequent among wh-questions and constitute almost 1% of the entire query stream. At the same time, how-to videos are popular on video sharing services. We analyzed a dataset of 500M+ Russian how-to questions. First, we carried out manual labelling of 1,000 queries that shows that about two thirds of all how-to question queries are potentially suitable for answers in the form of video in a language other than the language of the query. Then, we evaluated video retrieval quality for original and machine translated queries on a crowdsourcing platform. The evaluation reveals that machine translated questions yield video search quality comparable to the quality for original questions. Cross-lingual video search for how-to queries can improve recall and diversity of search results, as well as compensate the shortage of original content in emerging markets.