Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications

1Citations
Citations of this article
11Readers
Mendeley users who have this article in their library.

Abstract

Instruction Fine-Tuning (IFT) is a powerful paradigm that strengthens the zero-shot capabilities of Large Language Models (LLMs), but in doing so induces new evaluation metric requirements. We show LLM-based metrics to be well adapted to these requirements, and leverage them to conduct an investigation of task-specialization strategies, quantifying the tradeoffs that emerge in practical industrial settings. Our findings offer practitioners actionable insights for real-world IFT model deployment.

Cite

CITATION STYLE

APA

Faysse, M., Viaud, G., Hudelot, C., & Colombo, P. (2023). Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications. In EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings (pp. 9033–9048). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.emnlp-main.559

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free