Exploration and research about key technologies concerning Deep Learning models targeting Mobile Terminals

Abstract

With the rapid development of artificial intelligence and the popularity of mobile devices, mobile deep learning model technology has become a research hotspot in recent years. This paper studies the realization of the mobile terminal deep learning model from the optimization techniques of the deep learning model and the framework of deep learning. This review sorts out the model optimization techniques of pruning, quantization, and model knowledge distillation of deep learning models, and analyzes the lightweight deep learning models and deep learning frameworks suitable for mobile terminals. From the perspective of deep learning model compression, this paper provides multi-granularity pruning, pruning combined with batch normalization factor and filter correlation, joint dynamic pruning, pruning based on cross-entropy; a multi-module feature training method based on knowledge distillation, and an optimized model training strategy based on self-distillation; local quantization, exponential quantization. From the perspective of directly adopting a deep learning framework, this paper compares four different frameworks (Caffe/Caffe2, TensorFlow, Keras and Pytorch) introduced by different companies. The benefits of two other frameworks, TensorFlow Lite and FeatherCNN, are also mentioned. From the perspective of lightweight deep learning model design, this paper analyzes the design of three lightweight models such as SqueezeNet, MobileNet, and ShuffleNet, and compares their performance parameters such as accuracy gap and calculation speed with conventional models such as AlexNet and GoogleNet. Finally, the paper looks ahead to future directions in the field and what the authors believe are important ideas that may help inspire new ideas.