Eigenvalue distribution of some nonlinear models of random matrices

Abstract

This paper is concerned with the asymptotic empirical eigenvalue distribution of some non linear random matrix ensemble. More precisely we consider M=1mYY∗M=1mYY∗ with Y=f(WX)Y=f(WX) where W and X are random rectangular matrices with I I d. centered entries. The function f is applied pointwise and can be seen as an activation function in (random) neural networks. We compute the asymptotic empirical distribution of this ensemble in the case where W and X have sub-Gaussian tails and f is real analytic. This extends a result of [32] where the case of Gaussian matrices W and X is considered. We also investigate the same questions in the multi-layer case, regarding neural network and machine learning applications.