Multilingual Named Entity Recognition

Abstract

Mapping companies, such as TomTom, OpenStreetMap, and MapBox, face the difficult issue of how to extract pertinent information from news and social media to automatically update their maps. Tweets about new restaurants or Facebook postings about road construction introduce novel information relevant to map updates. Named Entity Recognition (NER) is a key part of processing and discerning the relevancy of this information. Additionally, given the global and interconnected natures of news and social media, this kind of information may be presented in different languages. We propose a Language-Independent Named Entity Recognition (NER) model using Facebook Multilingual Unsupervised and Supervised Embeddings (MUSE word embeddings) as features for a bi-directional Long Short Term Memory (Bi-LSTM) network with a Conditional Random Field (CRF) classifier. Using the CoNLL shared task corpora as our training data, we train a bi-LSTM to extract features from sentences and implement a classifier to tag words with Named Entity Inner-Outer-Beginning (IOB) tags.