Hybrid Clustering Framework for Scalable and Robust Query Analysis: Integrating Mini-Batch K-Means with DBSCAN Hybrid Model for Complex Data Clustering

Abstract

Query clustering is a significant task in information retrieval. Research gaps still exist due to high-dimensional datasets, noise detection, and cluster interpretability. Solving these challenges will support large language models with faster and more efficient responses. This study aims to develop a hybrid clustering approach combining Mini-Batch K-means (MBK) and Density-Based Spatial Clustering of Application with Noise (DBSCAN) to cluster large-scale query datasets for information retrieval. The proposed method utilizes a preprocessing technique for data cleaning, extracts meaningful features, and scales all the features from the query dataset. The proposed hybrid clustering framework utilizes preprocessed data for clustering. The clustering algorithms MBK provide fast, scalable clustering, and DBSCAN delivers a precise, density-based refinement to efficiently process large-scale datasets while enhancing cluster boundaries to handle outliers. The proposed hybrid clustering framework effectively performs query analysis in information retrieval with a Silhouette score of 72.14 % and adjusted rand index of 78.23%. Thus, the hybrid clustering approach provides a robust and scalable solution for query analyzing tasks.