ZhuQue: A neural network programming model based on labeled data layout

Abstract

In the last five years, the research of neural network accelerators has made remarkable achievements and provided powerful hardware support for many deep learning algorithms. In order to improve the performance of the neural network accelerator, algorithm optimization and data layout in the neural network development kit (NDK) are indispensable. The rich data types in neural network algorithms determine the diversity of data layout information. How to add complex data layout information to the NDK, to guide the work of all aspects of the software, to avoid user perception and to provide a user-friendly API, has become a series of issues worth studying. This paper implements a neural network development kit based on labeled data layout to solve the above problems, and abstracts a neural network programming model. The programming model establishes a neural network computing graph at “creating time”, “compiling time” sets the data label and “runtime” uses the label to control the data transfer. Compared with the existing NDK, the software has an average performance improvement of 4.76×. In addition, this paper also defines dynamic tags and static tags of neural network data, and proposes a neural network data classification method.