Retrieval-Augmented Text-to-CSEQL Generation for Cross-Platform Cyberspace Assets Query

Abstract

Cyberspace search engines (CSEs) are systems designed to search and index information about cyberspace assets. Effectively mining data across diverse platforms is hindered by the complexity and diversity of different CSE syntaxes. While Text-to-CSEQL offers a promising solution by translating natural language (NL) questions into cyberspace search engine query language (CSEQL), existing prompt-based methods still struggle due to the platform-specific intricacies of CSEQL. To address this limitation, we propose an LLM-based approach leveraging Retrieval-Augmented Generation (RAG). Specifically, to overcome the inability of traditional methods to retrieve relevant syntax fields effectively, we propose a novel hybrid retrieval mechanism combining keyword and dense retrieval, leveraging both field values and their semantic descriptions. Furthermore, we integrate these retrieved fields and the relevant few-shot examples into a redesigned prompt template adapted from the COSTAR framework. For comprehensive evaluation, we construct a Text-to-CSEQL dataset and introduce a new domain-specific metric, field match (FM). Extensive experiments demonstrate our method’s ability to adapt to platform-specific characteristics. Compared to prompt-based methods, it achieves an average accuracy improvement of 43.15% when generating CSEQL queries for diverse platforms. Moreover, our method also outperforms techniques designed for single-platform CSEQL generation.