DrowsyNet: Convolutional neural networks with runtime power-accuracy tunability using inference-stage dropout

Abstract

Convolutional neural networks (CNNs) have many future artificial intelligence (AI) applications; CNN hardware is an enabling technology to make this AI future come true. Two main specifications of CNN hardware are power and accuracy, which are determined at the design time by the CNN model, CNN hardware architecture, and the transformation scheme that maps the model onto the hardware. This paper points out that the above three design parameters share one common limitation that they are not flexible enough to offer power-accuracy tunability at deployment time and runtime. In response to this demand, we propose DrowsyNet, which randomly drops out a fraction of convolutional neurons to achieve a power-accuracy tradeoff. DrowsyNet is an inference-stage technique, which is different from traditional training-stage dropout. We further present that the dropout rates would better be set non-uniformly among convolutional layers. Experimental results show that up to 50% savings in the number of convolutions are available by trading away less than 11% of top-5 accuracy.