Sylva: Sparse Embedded Adapters via Hierarchical Approximate Second-Order Information

Abstract

Fine-tuning is the gateway to transferring learned knowledge in a pre-trained Large Language Model (LLM) on many downstream applications. To make LLM fine-tuning more affordable, prior works follow two paths: i) adapters freeze the pre-trained LLM weights and inject a small number of trainable weights during fine-tuning, and ii) pruners remove the less important weights in pre-trained LLMs and train the remaining sparse weights during fine-tuning. We find that the former introduces computation overheads due to the injected trainable parameters, while the latter introduces an expensive pre-processing step to identify the important weights and degrades model quality. To get the best of both worlds, we propose Sylva, a novel LLM fine-tuning procedure that provides high system performance during fine-tuning and attains state-of-the-art model quality on downstream applications. Sylva identifies the most important LLM weights via second-order information in a pre-processing step, and significantly reduces the computation and storage costs of the pre-processing step via i) a hierarchical approximation of second-order information, and ii) an online projection and rediagonalization algorithm. Sylva trains only the sparse important weights and embeds these sparse weights into the pre-trained LLM during fine-tuning to provide high system performance. We show that end-to-end fine-tuning with Sylva is, on average, 5.1 × faster than ZeRO and 1.2 × faster than LoRA, the state-of-the-art adapter approach. Sylva's hierarchical approximation reduces the peak GPU memory in the pre-processing step by 2.3 × compared to K-FAC, the most widely used approximation to second-order information. The source code of Sylva is publicly available at https://github.com/CentML/Sylva.