Development of Deep Learning Pavement Extraction and Crack Segmentation Algorithms to Analyze UAV Images of Roadways

Over the decades, significant efforts have been made in evaluating pavement condition by mounting cameras on a vehicle and automatically analyzing images by incorporating digital image processing techniques. Despite recent advances, there are still some limitations associated with current automated crack collection and analysis systems such as potential risk to vehicular safety, limited coverage by cameras, traffic blocking camera views, and errors in analyzing images for crack extent and severity. Inspired by current developments of deep learning technology such as object classification and sematic segmentation along with unmanned aerial vehicle (UAV) technology, this paper presents a set of comprehensive automated crack analysis algorithms based on a combination of deep learning and UAV images. To extract pavements from UAV images and segment cracks from extracted pavement images, the contracting encoder path of the U-Net model was modified with various deep learning models such as Pre-trained VGG 16, ResNet 50, Inception V3, and DenseNet 169 models as a backbone. Based on the least false negative and false positive outputs, the Inception V3 model with dice loss using a nonaugmented dataset and the Inception V3 model with focal loss using a nonaugmented dataset model showed the best performance for pavement extraction and crack segmentation, respectively. A tile-based pavement crack analysis system was then developed to measure percent cracking and crack widths from segmented crack images. It can be concluded that the developed pavement extraction and crack analysis system using UAV images will help public agencies evaluate pavement conditions in a systematic and cost-effective manner.