Deep Neural Networks (DNNs) are very powerful and successful but suffer from high computation and memory cost. As a useful attempt, binary neural networks represent weights and activations with binary values, which can significantly reduce resource consumption. However, the simultaneous binarization introduces the coupling effect, aggravating the difficulty of training. In this paper, we develop a novel framework named TP-ADMM that decouples the binarization process into two iteratively optimized stages. Firstly, we propose an improved target propagation method to optimize the network with binary activations in a more stable format. Secondly, we apply the alternating direction method (ADMM) with a varying penalty to get the weights binarized, making weights binarization a discretely constrained optimization problem. Experiments on three public datasets for image classification show that the proposed method outperforms the existing methods.
CITATION STYLE
Yuan, Y., Chen, C., Hu, X., & Peng, S. (2019). TP-ADMM: An efficient two-stage framework for training binary neural networks. In Communications in Computer and Information Science (Vol. 1142 CCIS, pp. 580–588). Springer. https://doi.org/10.1007/978-3-030-36808-1_63
Mendeley helps you to discover research relevant for your work.