Performance/Resources Comparison of Hardware Implementations on Fully Connected Network Inference

Abstract

Fully Connected Network inference is a complex algorithm that can be accelerated using edge devices like Field Programmable Gate Array (FPGA). One commonly known performance improvement for Fully Connected Network inference is quantization. This technique replaces the floating points weights of the network by integers. Frameworks like Open Neural Network Exchange (ONNX) and Tensorflow Lite provide solutions for this procedure. However, these frameworks have different inference algorithms with different operations and data types. In this article inference algorithms of common Fully Connected Networks in ONNX and Tensorflow Lite have been analysed. A performance and resource usage comparison is tested on Xilinx® Zynq UltraScale+™ MPSoC. Results show that to achieve lower latency is better to avoid floating point operations in the inference algorithm. In terms of FPGA resource usage, an increase is observed when the neural network becomes more complex regardless of the algorithm. This growth in resource usage is framework-dependent.