Detecção de defeitos de pavimento usando uma estratégia multimodal baseada em imagens e dados de sensor inercial

Abstract

The monitoring and maintenance of Brazilian highways represent a significant challenge due to their vast extent and the varied conditions of lighting and pavement. Automating this process through object detection models can make it more efficient and standardized. Previous studies have explored the identification of pavement defects using images and inertial sensors, such as accelerometers. This study investigates the hypothesis that a multimodal model, combining images and inertial sensor data, can enhance the detection of pavement failures, specifically potholes, patches, cracks, and wear. To this end, the YOLOv8n-Detect (unimodal) and YOLOv8nMultiDetect (multimodal) models were compared, achieving overall mAP@0.20 averages of 0.759 and 0.809, respectively, and mAP@0.20:0.65 averages of 0.726 and 0.780. These results indicate that feature fusion improves not only object detection but also their localization, as demonstrated by superior performance in both mAP@0.20 and mAP@0.20:0.65, highlighting the potential of multimodal models to optimize the identification of pavement issues on highways.