Towards optimal β-variational autoencoders combined with transformers for reduced-order modelling of turbulent flows

Abstract

Variational autoencoders (VAEs) have shown promising potential as artificial neural networks (NN) for developing reduced-order models (ROMs) in the context of turbulent flows. In this study, we propose a method that combines β-VAEs for modal decomposition and transformer neural networks for temporal-dynamics prediction in the latent space to develop ROMs. We apply our method to an existing database of a turbulent flow around a wall-mounted square cylinder obtained by direct numerical simulation (DNS). A parametric study is performed to investigate the effects of the hyperparameters of the proposed β-VAEs and determine the optimal values. For the first time, we incorporate the consideration of the complexity of architecture into our studies, providing new insights into hyperparameter selection for β-VAEs, which remains a challenging problem for optimising model performance. Results regarding the influence of the different hyperparameters and guidelines to design these architectures are reported. Our optimal model achieves a reconstruction accuracy of 97.18% of the entire dataset using only ten modes. Subsequently, we employ the transformer models to identify latent-space temporal dynamics learned by the optimal β-VAE model and build ROMs to predict instantaneous fields. The resulting model achieves promising accuracy in temporal-dynamics predictions and yields energy reconstruction levels of 96.5% and 83% for a field 25 and 50 steps into the future, respectively, showcasing the potential of the transformer in predicting the temporal dynamics. Overall, the proposed method has potential applications in advanced flow control and fundamental studies of complex turbulent flows.