CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization

12Citations
Citations of this article
26Readers
Mendeley users who have this article in their library.

Abstract

Hallucination is a known issue for neural abstractive summarization models. Recent work suggests that the degree of hallucination may depend on factual errors in the training data. In this work, we propose a new method called Contrastive Parameter Ensembling (CaPE) to use training data more effectively, utilizing variations in noise in training samples to reduce hallucination. Starting with a base model fine-tuned on an entire dataset, we additionally train expert and anti-expert models on clean and noisy subsets of the data, respectively. We then adjust the parameters of the base model by adding (subtracting) the parameters of the expert (anti-expert), advancing the recent work on additive parameter ensembling approaches. Trained on a much smaller data subset, expert and anti-expert models only fractionally (<14%) increases the total training time. Further, CaPE uses parameter ensembling and does not increase the inference time. Experimental results show that CaPE improves performance across different automatic factual metrics and human evaluation, with a maximum improvement of 16.69% and 15.38% on summary-level dependency-arc entailment accuracy for the XSUM and CNN/DM datasets. The CaPE model performs comparably to the base model on metrics of informativeness such as ROUGE.

References Powered by Scopus

On faithfulness and factuality in abstractive summarization

652Citations
463Readers

Summeval: Re-evaluating summarization evaluation

406Citations
303Readers

Asking and answering questions to evaluate the factual consistency of summaries

281Citations
312Readers

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Choubey, P. K., Fabbri, A. R., Vig, J., Wu, C. S., Liu, W., & Rajani, N. (2023). CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (pp. 10755–10773). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.findings-acl.685

Readers over time

‘22‘23‘24‘250481216

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 8

80%

Lecturer / Post doc 1

10%

Researcher 1

10%

Readers' Discipline

Tooltip

Computer Science 13

93%

Medicine and Dentistry 1

7%

Save time finding and organizing research with Mendeley

Sign up for free
0