Alternating optimization of unsupervised regression with evolutionary embeddings

Abstract

Unsupervised regression is a dimensionality reduction method that allows embedding high-dimensional patterns in lowdimensional latent spaces. In the line of research on iterative unsupervised regression, numerous methodological variants have been proposed in the recent past. This works extends the set of methods by evolutionary embeddings. We propose to use a (1+λ)-ES with Rechenberg mutation strength control to iteratively embed patterns and show that the learned manifolds are better with regard to the data space reconstruction error than the embeddings generated with naive Gaussian sampling. Further, we introduce a hybrid optimization approach of alternating gradient descent and the iterative evolutionary embeddings. Experimental comparisons on artificial test data sets confirm the expectation that a hybrid approach is superior or at least competitive to known methods like principal component analysis or Hessian local linear embedding.