Online Incremental Feature Learning with Denoising Autoencoders

  • Zhou G
  • Sohn K
  • Lee H
  • 224

    Readers

    Mendeley users who have this article in their library.
  • 32

    Citations

    Citations of this article.

Abstract

While determining model complexity is an important problem in machine learning, many feature learning algorithms rely on cross-validation to choose an optimal number of features, which is usually challenging for online learning from a massive stream of data. In this paper, we propose an incremental feature learning algorithm to determine the optimal model complexity for large-scale, online datasets based on the denoising autoencoder. This algorithm is composed of two processes: adding features and merging features. Specifically, it adds new features to minimize the objective function’s residual and merges similar features to obtain a compact feature representation and prevent over-fitting. Our experiments show that the proposed model quickly converges to the optimal number of features in a large-scale online setting. In classification tasks, our model outperforms the (non-incremental) denoising autoencoder, and deep networks constructed from our algorithm perform favorably compared to deep belief networks and stacked denoising autoencoders. Further, the algorithm is effective in recognizing new patterns when the data distribution changes over time in the massive online data stream.

Author-supplied keywords

  • Autoencoders
  • Denoising Autoencoders
  • Feature Learning
  • Machine Learning

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

  • SGR: 84954202765
  • PUI: 607769036
  • SCOPUS: 2-s2.0-84954202765
  • ISSN: 15337928

Authors

  • Guanyu Zhou

  • Kihyuk Sohn

  • Honglak Lee

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free