LECO: Improving Early Exiting via Learned Exits and Comparison-based Exiting Mechanism

4Citations
Citations of this article
6Readers
Mendeley users who have this article in their library.

Abstract

Recently, dynamic early exiting has attracted much attention since it can accelerate the inference speed of pre-trained models (PTMs). However, previous work on early exiting has neglected the intermediate exits' architectural designs. In this work, we propose a novel framework, Learned Exits and COmparison-based early exiting (LECO) to improve PTMs' early exiting performances. First, to fully uncover the potentials of multi-exit BERT, we design a novel search space for intermediate exits and employ the idea of differentiable neural architecture search (DNAS) to design proper exit architectures for different intermediate layers automatically. Second, we propose a simple-yet-effective comparison-based early exiting mechanism (COBEE), which can help PTMs achieve better performance and speedup tradeoffs. Extensive experiments show that our LECO achieves the SOTA performances for multi-exit BERT training and dynamic early exiting.

Cite

CITATION STYLE

APA

Zhang, J., Tan, M., Dai, P., & Zhu, W. (2023). LECO: Improving Early Exiting via Learned Exits and Comparison-based Exiting Mechanism. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Vol. 4, pp. 298–309). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.acl-srw.43

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free