A Quality-Oriented Reconfigurable Convolution Engine Using Cross-Shaped Sparse Kernels for Highly-Parallel CNN Acceleration

Abstract

Computational imaging CNNs are computationally intensive and need complexity reduction to support high-Throughput applications. However, conventional compact model reduction tends to degrade image quality severely as models become too shallow. On the other hand, irregular pruning-based techniques induce considerable circuit overheads and imbalanced workloads, especially for highly-parallel accelerators. In this paper, we propose cross-shaped sparse kernels to regularly reduce model complexity while preserving image quality well. They improve PSNR (peak signal-To-noise ratio) by 0.03-0.31 dB on classic denoising and super-resolution networks compared to compact depth reduction. Moreover, we design a highly-parallel reconfigurable convolution engine to support three sparsity configurations (0%, 50% and 75% of sparsity) for our complexity-saving method. It can achieve high-quality inference for a wide complexity range with full utilization of MACs. With TSMC 40nm technology, the engine uses 9.85M of logic gates for delivering 8.2 TOPS of inference capability, and only needs 8.4% logic overheads and 14.9% additional power consumption for the quality-oriented reconfigurability. Finally, we do a case study on ERNets for real-Time inference, and this work can achieve 10.114.8x higher area efficiency in terms of Mpixel/s/mm2 compared to SparTen.