Regularized One-Layer Neural Networks for Distributed and Incremental Environments

Abstract

Deploying machine learning models at scale is still a major challenge; one reason is that performance degrades when they are put into production. It is therefore very important to ensure the maximum possible generalization capacity of the models and regularization plays a key role in avoiding overfitting. We describe Regularized One-Layer Artificial Neural Network (ROLANN), a novel regularized training method for one-layer neural networks. Despite its simplicity, this network model has several advantages: it is noniterative, has low complexity, and is capable of incremental and privacy-preserving distributed learning, while maintaining or improving accuracy over other state- of-the-art methods as demonstrated by the experimental study in which it has been compared with ridge regression, lasso and elastic net over several data sets.