SEMSDNet: A Multiscale Dense Network with Attention for Remote Sensing Scene Classification

Abstract

Remote sensing image scene classification plays an important role in remote sensing image interpretation. Deep learning brings prosperity to the research in this field, and numerous deep learning models are proposed in order to improve the performance of scene classification. However, images of different remote sensing scenes vary a lot, showing similar or diverse textures and simple or complex contents. Using a fixed convolutional neural network framework to classify scene images is performance-limited and not practice-flexible. To address this issue, in this article, we propose the SEMSDNet (multiscale dense networks with squeeze and excitation attention). The framework multiscale dense convolutional network (MSDNet) with multiple classifiers and dense connections can automatically transform between a small network and a deep network according to the complexity of test samples and the limitation of computational resources. Moreover, in order to extract more effective features, the squeeze-and-excitation (SE) attention mechanism is introduced into the framework to process the features of various scenes self-adaptively. In addition, considering the limited computing resources, we impose two settings with computational constraints at the test time: budgeted batch classification, which is a fixed computational budget setting for sample classification, and anytime prediction, which forces the network to output a prediction at any given point-in-time. Experimental results on several public datasets show that the proposed SEMSDNet method is superior to the state-of-the-art methods on both performance and efficiency. Experiments also reveal its capability to treat samples of different classification difficulties with uneven resource allocation and flexible network architecture, showing its potentials in practical applications.