Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks

2Citations
Citations of this article
18Readers
Mendeley users who have this article in their library.

Abstract

Despite readily memorizing world knowledge about entities, pre-trained language models (LMs) struggle to compose together two or more facts to perform multi-hop reasoning in question-answering tasks. In this work, we propose techniques that improve upon this limitation by relying on random walks over structured knowledge graphs. Specifically, we use soft prompts to guide LMs to chain together their encoded knowledge by learning to map multi-hop questions to random walk paths that lead to the answer. Applying our methods on two T5 LMs shows substantial improvements over standard tuning approaches in answering questions that require 2-hop reasoning.

Cite

CITATION STYLE

APA

Misra, K., dos Santos, C. N., & Shakeri, S. (2023). Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (pp. 972–985). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.findings-acl.62

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free