Wizard of Search Engine: Access to Information through Conversations with Search Engines

Abstract

Conversational information seeking (CIS) is playing an increasingly important role in connecting people to information. Due to a lackof suitable resources, previous studies on CIS are limited to thestudy of conceptual frameworks, laboratory-based user studies, or a particular aspect of CIS (e.g., asking clarifying questions). In this work, we make three main contributions to facilitate research into CIS: (1) We formulate a pipeline for CIS with six subtasks: intent detection, keyphrase extraction, action prediction, query selection, passage selection, and response generation. (2) We release a benchmark dataset, called wizard of search engine (WISE), which allows for comprehensive and in-depth research on all aspects of CIS. (3) We design a neural architecture capable of training and evaluating both jointly and separately on the six sub-tasks, and devise a pre-train/fine-tune learning scheme, that can reduce the requirements of WISE in scale by making full use of available data. We report useful characteristics of the CIS task based on statistics of the WISE dataset. We also show that our best performing model variant is able to achieve effective CIS. We release the dataset, code as well as evaluation scripts to facilitate future research by measuring further improvements in this important research direction.