Object Regression: Multi-Modal Data Enhanced Object Detection for Leasing Vehicle Return Assessment

Various fields and industries have widely adopted Machine Learning (ML) to automate their manual processes and enable data-driven decision making. The Vehicle Leasing Return Assessment (VLRA) process requires all leased vehicles to be appraised for damages at the end of the contract period. These damages need to be classified, and a repair cost needs to be determined. This manual process adds time and labor overhead and introduces a high variance to the final cost due to human biases. A data-driven ML method is needed to automate and streamline VLRA to keep up with the increasing demand and ensure an optimal customer experience. In this work, we present Object Regression, an end-to-end detection and cost prediction model which leverages multi-modal image and vector data for damage detection and cost prediction in a single detection/regression network. Using Faster-RCNN coupled with a ResNet50 backbone, we can extend the capabilities of the standard two-stage object detector to utilize the inherent relationship between different data modalities that are not being leveraged by standalone detection or prediction models. We partner with one of Europe's biggest car manufacturers and detail the process of converting an industrial dataset for a ML task. We also showcase the performance improvements that can be achieved using highly related multi-modal data.