GPT Applications in Relevance Model Training in Map Search

Abstract

Understanding map queries and retrieving correct entity results are the two main relevance tasks in Map search. They are usually performed by a set of task specific machine learning models. Collecting large amount of high quality labelled data for training such models is a time-consuming and labor-intensive process. Although various methods have been studied for producing pseudo data labels, they are limited in their effectiveness when applied across different languages or tasks. The recently released Large Language models (LLMs), including ChatGPT and GPT-4 (GPT for short), have demonstrated state-of-the-art performance in text understanding by using simple prompt instructions with only a handful of examples for in-context learning. In this paper, we explore GPT as a cost-effective alternative for both data labeling and synthetic data generation, where we subsequently use data obtained from this approach to train various task specific models such as maps intent detection, address detection, address parsing, geo-entity ranking, and rank scores calibration. GPT demonstrates strong potential in generating otherwise hard-to-synthesize data. We observe significant accuracy and relevance improvement across all task specific models when trained or fine-tuned on data generated by GPT. Lastly, we propose a general framework combining labeled data from GPT with other sources and a prompt fine-tune structure to guide GPT model in completing a given task.