Pairwise Multi-layer nets for learning distributed representation of multi-field categorical data

Abstract

This paper presents a method of pairwise multi-layer networks for multi-field categorical data, which widely exists with various applications such as web search, recommender systems, social link prediction, and computational advertising. The success of non-linear models, e.g., factorization machines, boosted trees, has proved the potential of exploring the interactions among inter-field discrete categories. Inspired by Word2Vec, the distributed representation for natural language, we propose a PMLN (Pairwise Multi-Layer Nets) model to learn the distributed representation for multi-field categorical data. In PMLN, a low-dimensional continuous vector is automatically learned for each category in each field. The interactions among inter-field categories are explored by different neural gates and the most informative ones are selected by pooling layers. Such combined categories can be further explored by performing more gate interactions with another category and then selected by additional pooling operations. In our experiments, with the exploration of the interactions between pairwise categories over layers, the model outperforms state-of-the-art models in a supervised learning task, i.e., ad click prediction, while capturing the most significant interactions from the data in an unsupervised fashion.