Document classification by inversion of distributed language representations

Abstract

There have been many recent advances in the structure and measurement of dis-tributed language models: those that map from words to a vector-space that is rich in information about word choice and com-position. This vector-space is the dis-tributed language representation. The goal of this note is to point out that any distributed representation can be turned into a classifier through inversion via Bayes rule. The approach is simple and modular, in that it will work with any language representation whose train-ing can be formulated as optimizing a probability model. In our application to 2 million sentences from Yelp reviews, we also find that it performs as well as or bet-ter than complex purpose-built algorithms.