A data driven similarity measure and example mapping function for general, unlabelled data sets

Abstract

Deep networks such as autoencoders and deep belief nets are able to construct alternative, and often informative, representations of unlabeled data by searching for (hidden) structure and correlations between the features chosen to represent the data and combining them into new features that allow sparse representations of the data. These representations have been chosen to often increase the accuracy of further classification or regression accuracy when compared to the original, often human chosen representations. In this work, we attempt an investigation of the relation between such discovered representations found using related but differently represented sets of examples. To this end, we combine the cross-domain comparison capabilities of unsupervised manifold alignment with the unsupervised feature construction of deep belief nets, resulting in an example mapping function that allows re-encoding examples from any source to any target task. Using the t-Distributed Stochastic Neighbour Embedding technique to map translated and real examples to a lower dimensional space, we employ KL-divergence to define a dissimilarity measure between data sets enabling us to measure found representation similarities between domains.