RIC-NVNet：night-time vehicle enhancement network for vehicle model recognition

Abstract

Objective The recognition of vehicle model based on night-time vehicle images is challenging for such con⁃ straints like weak exposure，irregular distribution of light conditions，and low contrast of night-time images. Specifically，vehicle images taken at night suffer from noise，over-exposure，under-exposure，and additional light source interference，making it difficult for artificial calibration with naked eyes，and recognition with artificial intelligence systems，thus，it directly degrades the performance of intelligent transportation systems. Conventional low-light methods directly brighten the whole image straightforward in terms of histogram equalization，or the correlation of adjacent pixels，which remains being improved on the noise suppression and color distortion. Once the Retinex theory has been proposed，subsequent researches followed the guidance of the theory to decompose the input image into illumination and reflectance components，then enhance the components in a traditional fashion. These methods performed well in the low-light enhancement areas and are limited because of the need for prior knowledge. The emerging deep learning technique-based methods have facili⁃ tated low-light image enhancement to a certain extent. Most of them adopted the U-Net and designed a great variety of loss functions to converge the network for better performance. Multiple categories of datasets were proposed and developed to optimize data-driven methods. However，few methods focused on night-time vehicles in the real scenario. Thus，these methods fail to generalize in night-time vehicle enhancement，especially on the aspect of vehicle light interference or under⁃ exposure of distinctive vehicle parts. Therefore，considering enhancing the night-time vehicle images，this paper proposes a night-time vehicle image enhancement network based on reflectance and illumination components （RIC-NVNet） to enhance the distinctive features so as to improve both the overall enhancement and the correct rate of vehicle model recogni⁃ tion. Method The RIC-NVNet model consists of an information extraction module，a reflection enhancement module，and an illumination enhancement module. Firstly，the RIC-NVNet utilizes an information extraction module based on the U-Net network structure，using a combination of the night-time vehicle image and its grayscale image as input，to extract the reflection and illumination components of the night-time image. Subsequently，the reflection enhancement module，with a skip connection structure，corrects the color distortion and additional noise problems of the reflection component of the night-time image to obtain an enhanced reflection component. Then，the illumination enhancement module，based on a generative adversarial network structure and an adaptive weight coefficient matrix，generates a day-time illumination compo⁃ nent from the illumination component of the night-time image extracted by the information extraction module. Finally，based on the Retinex theory，the enhanced reflection component and the generated daytime illumination component are multiplied to obtain the image after illumination enhancement. To effectively train the RIC-NVNet，we improve the con⁃ straint loss of the illumination component to enhance the component extraction effectiveness of the information extraction network. Also，we use color restoration loss，structure consistency loss，and RGB channel loss to constrain the reflection enhancement module to further improve the model’s performance. In addition，we adopt a generative adversarial loss to constrain the illumination enhancement module and improve its robustness. In summary，the RIC-NVNet is a powerful night-time vehicle image enhancement model that can effectively improve the quality and recognition rate of night-time images. Result On one hand，the performance of RIC-NVNet was evaluated on the simulated night-time vehicle datasets （SNV）and real night-time vehicle datasets（RNV）proposed in this paper. The results showed that using the RIC-NVNet method for low-light image enhancement on these datasets resulted in higher Top1 and Top5 recognition rates obtained by residual neural network-50（ResNet50）compared to other low-light image enhancement methods. In the SNV dataset，the Top1 and Top5 recognition rates of RIC-NVNet were 82. 68% and 94. 92%，respectively，which were about 2% higher than the lower recognition rates of the zero-reference deep curve estimation（Zero-DCE）method. Additionally，the image quality evaluation indices peak signal to noise ratio（PSNR）and structural similarity（SSIM）were also correspondingly improved compared to other methods. Conclusion The experimental results show that the proposed method can solve the problem of low recognition rates of night-time vehicle images caused by weak exposure and multiple interfering light sources. The method combines an information extraction module，a reflection enhancement module，and an illumination enhancement module，and outperforms other low-light enhancement methods in terms of objective recognition rates，image evaluation indices，and subjective overall image quality of the enhanced night-time vehicle images.