FPRaker: A processing element for accelerating neural network training

9Citations
Citations of this article
32Readers
Mendeley users who have this article in their library.
Get full text

Abstract

We present FPRaker, a processing element for composing training accelerators. FPRaker processes several floating-point multiplyaccumulation operations concurrently and accumulates their result into a higher precision accumulator. FPRaker boosts performance and energy efficiency during training by taking advantage of the values that naturally appear during training. It processes the significand of the operands of each multiply-accumulate as a series of signed powers of two. The conversion to this form is done on-thefly. This exposes ineffectual work that can be skipped: values when encoded have few terms and some of them can be discarded as they would fall outside the range of the accumulator given the limited precision of floating-point. FPRaker also takes advantage of spatial correlation in values across channels and uses delta-encoding offchip to reduce memory footprint and bandwidth. We demonstrate that FPRaker can be used to compose an accelerator for training and that it can improve performance and energy efficiency compared to using optimized bit-parallel floating-point units under iso-compute area constraints. We also demonstrate that FPRaker delivers additional benefits when training incorporates pruning and quantization. Finally, we show that FPRaker naturally amplifies performance with training methods that use a different precision per layer.

Cite

CITATION STYLE

APA

Awad, O. M., Mahmoud, M., Edo, I., Zadeh, A. H., Bannon, C., Jayarajan, A., … Moshovos, A. (2021). FPRaker: A processing element for accelerating neural network training. In Proceedings of the Annual International Symposium on Microarchitecture, MICRO (pp. 857–869). IEEE Computer Society. https://doi.org/10.1145/3466752.3480106

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free