Mining OOV translations from mixed-language web pages for cross language information retrieval

Abstract

Translating Out-Of-Vocabulary (OOV) terms is crucial for Cross Language Information Retrieval (CLIR). In this paper, we propose a method that automatically acquires a large quantity of OOV translations from the web. Different from previous approaches that rely on a finite set of hand-crafted extraction rules, our method adaptively learns translation extraction patterns based on the observation that translation pairs on the same page tend to appear following similar layout patterns. The learned patterns are leveraged in a discriminative translation extraction model that treats translation extraction from a mixed language bilingual web page as a sequence labeling task in order to exploit useful relations among translation pairs on the page. Experiments demonstrate that our proposed method out-performs earlier work with marked improvement on OOV translation mining quality. © 2010 Springer-Verlag Berlin Heidelberg.