Assessment of Autoencoder Architectures for Data Representation

Abstract

Efficient representation learning of data distribution is part and parcel of successful execution of any machine learning based model. Autoencoders are good at learning the representation of data with lower dimensions. Traditionally, autoencoders have been widely used for data compression in order to represent the structural data. Data compression is one of the most important tasks in applications based on Computer Vision, Information Retrieval, Natural Language Processing, etc. The aim of data compression is to convert the input data into smaller representation retaining the quality of input data. Many lossy and lossless data compression techniques like Flate/deflate compression, Lempel–Ziv–Welch compression, Huffman compression, Run-length encoding compression, JPEG compression are available. Similarly, autoencoders are unsupervised neural networks used for representing the structural data by data compression. Due to wide availability of high-end processing chips and large datasets, deep learning has gained a lot attention from academia, industries and research centers to solve multitude of problems. Considering the state-of-the-art literature, autoencoders are widely used architectures in many deep learning applications for representation and manifold learning and serve as popular option for dimensionality reduction. Therefore, this chapter aims to shed light upon applicability of variants of autoencoders to multiple application domains. In this chapter, basic architecture and variants of autoencoder viz. Convolutional autoencoder, Variational autoencoder, Sparse autoencoder, stacked autoencoder, Deep autoencoder, to name a few, have been thoroughly studied. How the layer size and depth of deep autoencoder model affect the overall performance of the system has also been discussed. We also outlined the suitability of various autoencoder architectures to different application areas. This would help the research community to choose the suitable autoencoder architecture for the problem to be solved.