An Overview of Some Issues in the Theory of Deep Networks

Abstract

During the last few years, significant progress has been made in the theoretical understanding of deep networks. We review our contributions in the areas of approximation theory and optimization. We also introduce a new approach based on cross-validation leave-one-out stability to estimate bounds on the expected error of overparametrized classifiers, such as deep networks. © 2020 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.