Vehicle Damage Severity Estimation for Insurance Operations Using In-The-Wild Mobile Images

Abstract

Following a car accident, an insurance company must assess the level of damage to each vehicle to decide on the compensation paid to the insurance customer. This assessment is usually performed by manual inspection, which is costly and time-consuming. Automatic car damage assessment using image data is an under-addressed problem highly relevant to the insurance industry. Although there have been many attempts at solving particular aspects of this problem, we are unaware of any complete solutions available. In this work, we propose a pipeline that uses photographs of a damaged car, collected by the users from multiple angles, together with structured data about the vehicle, to estimate damage severity following an accident. Our proposed pipeline consists of several computer-vision models for the detection of damage and the determination of its extent. Unlike existing approaches in car damage assessment, we use semantic segmentation to understand which parts of the car are damaged, and to what extent. We then extract computer-vision features, indicating the location and severity of damage to each exterior panel, together with structured data, to arrive at an accurate damage cost estimation. We train and evaluate this model on a large dataset of historical insurance claims with known outcomes, all captured in the wild with mobile-phone hardware.