Sparse Bayesian Broad Learning System for Probabilistic Estimation of Prediction

Abstract

Broad learning system (BLS) is proposed as a competitor to deep neural network, which is constructed using only single hidden layer instead of multilayered structure. It also manages incremental learning with intelligible mathematical derivation by pseudoinverse. Despite its empirical success, the limitations of BLS cannot be overlooked, especially for its sensibility to the number of feature nodes. For this reason, large model is required in training, which may easily cause overfitting. In this paper, a probabilistic model called sparse Bayesian broad learning system (SBBLS) is proposed to dispose of this defect by linearly-weighting a small group of basis functions from an enormous number of candidates within Bayesian framework. Outstanding sparsity and generalization can be observed in SBBLS. More distinctively, SBBLS is capable of outputting probabilistic estimation of prediction for further decision analysis while maintaining inherent properties of BLS simultaneously. Experimental results demonstrate that the proposed SBBLS model could achieve comparable performance to BLS both in regression and classification with probabilistic interpretability.