A Hybrid Framework for Stock Price Forecasting Using Metaheuristic Feature Selection Approaches and Transformer Models Enhanced by Temporal Embedding and Attention Pruning

Abstract

Accurately predicting stock prices remains a major challenge in financial analytics due to the complexity and noise inherent in market data. Feature selection plays a critical role in improving both computational efficiency and predictive performance. In this study, we introduce a novel hybrid framework that integrates metaheuristic feature-selection algorithms with an enhanced transformer-based prediction model fine-tuned using temporal embedding and adaptive attention pruning. We evaluate and compare the effectiveness of three nature-inspired metaheuristic algorithms: bat algorithm (BAT), gray wolf optimization (GWO), and beluga whale optimization (BWO) for selecting the most informative features from a time-series stock dataset. After feature selection, the optimal subsets are fed into our modified transformer equipped with temporal embeddings and adaptive attention pruning. Extensive experiments conducted on the Bharat Heavy Electricals Limited (BHEL) dataset show that the proposed hybrid framework outperforms traditional methods in terms of predictive accuracy. Among the evaluated approaches, the combination of BWO and the fine-tuned transformer achieves the best performance, yielding a Test RMSE of 0.0030 and a Test MAPE of 0.0108, demonstrating the superiority of BWO in identifying relevant features. This work provides a comprehensive comparative analysis of hybrid metaheuristic–deep learning models for stock price prediction and offers a foundation for integrating more explainable and scalable AI techniques into financial forecasting.