Sparse Hierarchical Table Ensemble-A Deep Learning Alternative for Tabular Data

Abstract

Deep learning for tabular data is drawing increasing attention, with recent work attempting to boost the accuracy of neuron-based networks. However, such deep learning models are deserted when computational capacity is low, as in Internet of Things (IoT), drone, or Natural User Interface (NUI) applications. We offer to enable deep learning capabilities using ferns (oblivious decision trees) instead of neurons by constructing a Sparse Hierarchical Table Ensemble (S-HTE). S-HTE is dense at the beginning of the training process and becomes gradually sparse using an annealing mechanism, leading to an efficient final predictor. Unlike previous work with ferns, S-HTE learns useful internal representations and earns from increasing depth. Using a standard classification and regression benchmark, we show its accuracy is comparable to alternatives while havingy lower computational complexity. Our PyTorch implementation is available at https://github.com/farjon/HTE_CTE.