Deep generative multi-view learning

Abstract

Deep generative networks has attracted proliferating interests recently. In this work, the linear generative multi-view model is extended to nonlinear multi-views model where the deep neural network is leveraged to model complex latent representation underlying the multi-view observation. The proposed deep multi-view model admits fast stochastic optimization for training the network and offers a model to infer the shared hidden representation and subsequently generate the second view based on the available primary view at the test time. Empirical results prove the merits of the proposed methods. Furthermore, it is shown that the proposed deep model can generate samples in the input space and suppress the background noise or other complex forms of distortions, the abilities that are not naturally available in CCA based methods.