Low-Cost Network Scheduling of 3D-CNN Processing for Embedded Action Recognition

Abstract

The recent 3D convolutional neural network (3D-CNN) is a promising candidate for solving the action recognition problem by providing attractive algorithm-level performance. Due to the excessive amount of computational costs, however, it is almost impractical to apply the advanced 3D-CNN architecture to the resource-limited real-time embedded system. In this work, we present several optimization schemes that can relax the complexity of 3D-CNN processing without sacrificing recognition accuracy. More precisely, we first develop several 3D-CNN architectures for exploiting the trade-off between the network complexity and recognition performance. Evaluating the current confidential level, then, the proposed method dynamically changes the network structure to be used for the next clip-level inference. In addition, we introduce a systematic way of managing the network sequence for minimizing the computing overheads while supporting the acceptable algorithm-level performance. Compared to the previous works, as a result, the proposed approaches drastically relax the processing costs as well as the energy consumption by selecting the simplest 3D-CNN architecture at the run time, allowing the cost-effective action recognition for embedded edges.