Transfer Learning with Transformer-Based Models for Mine Water Inrush Prediction: A Multivariate Analysis Using Sparse and Imbalanced Monitoring Data

Abstract

Predictions of mining-induced water inrush accidents are challenged by data sparseness and imbalances, as very few high-quality datasets can be obtained for successfully modeling data variation. By using the concept of transfer learning, we employed a well recorded borehole group water level dataset as a source dataset to train a selection of Transformer-based multivariate prediction models with state-of-the-art performance including PatchTST, InFormer, and AutoFormer, to capture data variation patterns in a statistically similar target dataset from a site with similar geological and mining conditions and examined the models' accident prediction performance. Additionally, the frequently used MLP-based Nbeats, RNN-based LSTM, and CNN-based TCN were adopted for the same task. In contrast to models trained merely on the target dataset, the Transformer-based models, especially PatchTST, achieved satisfactory zero-shot prediction performances in terms of accuracy, responsiveness, and anomaly detections for the early warning of accidents, proving their promising generalization capabilities for leveraging existing datasets for forecasting future accidents with data obtained in similar geological conditions. This has broad implications for mining accident prediction and groundwater risk assessment using data-driven approaches.