Accelerate DNN performance with sparse matrix compression in Halide

Abstract

Machine learning nowadays is profoundly impacting every aspect of our lives. With the evolution of the machine learning, many techniques, such as deep learning, improve the accuracy and performance of machine learning. Deep learning is a set of ML techniques that use layers of transformation and consist of neural networks. The power consumption of deep learning becomes a serious problem when it comes to edge computing. One of the most computationally demanding operation of DNN is convolution which preserve the image arrangement and obtain partial image as an input feature. Our goal is to find an effective way for programmers to improve the performance of convolution operation. In this paper, we proposed the design of sparse matrix compression schedule primitives in Halide and find a way to improve convolution operation with im2col method. Halide is an image processing programming language that separates algorithm from its schedule. With this design, we can compress the result of im2col matrix to achieve performance improvements. In our experiments, results show that convolution operation can achieve 20X speedup with our implementation.