Automatic Detection of Water Supply Pipe Defects Based on Underwater Image Enhancement and Improved YOLOX
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 10
Abstract
The water supply pipe system is an important component of the municipal pipe system. However, water supply pipes usually suffer from various defects, such as deposits and infiltrations, which severely affect their performance and result in millions of dollars being wasted on maintenance work. Therefore, timely and effective inspection of water supply pipes is very important. In recent years, automatic detection based on deep learning methods has had the advantages of high efficiency, low cost, and time saving, thus gradually replacing manual inspection for defects in the pipe system. To solve the problem of unclear image acquisition for water supply pipes, this paper proposes a novel automated detection method for water supply pipe defects, mainly involving the use of underwater image enhancement (UIE) algorithms to improve data set image quality, and an attention mechanism was utilized to improve the You Only Look Once X (YOLOX) model for defects detection. Experimental results demonstrate that the improved YOLOX model based on the data set enhanced by underwater image enhancement and attention mechanism achieved an average accuracy [mean average precision (mAP)] value of 92.4% and F1 score of 0.86, which are better than traditional models. Finally, an efficient and accurate automated detection procedure for water supply pipe defects was provided.
Practical Applications
The automatic detection method of water supply pipe defects constructed in this research has the following three significant practical advantages: (1) the UIE algorithm is applied to the water supply pipe image, which improves the quality and quantity of the data set; (2) the method combines the improved UIE data set and attention mechanism to promote the efficiency and accuracy of the object detection model; (3) the research provides a novel procedure for the automatic detection work of water supply pipe defects. New attempts have been made in three aspects—the use of underwater pipeline robots for detection, the establishment of a high-quality data set, and the training or prediction of the object detection model for water supply pipe—and good results have been achieved. For these reasons, this method can greatly reduce the workload of construction personnel, and effectively avoid the occurrence of detection error events caused by the misjudgment of technicians.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research is supported by the Innovation Group Science Foundation of the Natural Science Foundation of Chongqing, China (Grant No. cstc 2020jcyj-cxttX0003), the Key Program of the National Natural Science Foundation of China (52130901), and the Taishan Industry Leading Talents (tscx202306104). The authors express gratitude for their support.
References
Ancuti, C., C. O. Ancuti, T. Haber, and P. Bekaert. 2012. “Enhancing underwater images and videos by fusion.” In Proc., IEEE Conf. Computer Vision Pattern Recognition, 81–88. New York: IEEE. https://doi.org/10.1109/CVPR.2012.624766.
Ancuti, C. O., C. Ancuti, C. D. Vleeschouwer, and P. Bekaert. 2018. “Color balance and fusion for underwater image enhancement.” IEEE Trans. Image Process. 27 (1): 379–393. https://doi.org/10.1109/TIP.2017.2759252.
Bochkovskiy, A., C. Y. Wang, and H. Liao. 2020. “YOLOv4: Optimal speed and accuracy of object detection.” Preprint, submitted April 23, 2020. https://arxiv.org/abs/2004.10934.
Cai, B., X. Xu, K. Jia, C. Qing, and D. Tao. 2016. “DehazeNet: An end-to-end system for single image haze removal.” IEEE Trans. Image Process. 25 (11): 5187–5198. https://doi.org/10.1109/TIP.2016.2598681.
Cheng, J. C. P., and M. Z. Wang. 2018. “Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques.” Autom. Constr. 95 (Apr): 155–171. https://doi.org/10.1016/j.autcon.2018.08.006.
Costello, S. B., D. N. Chapman, C. D. F. Rogers, and N. Metje. 2007. “Underground asset location and condition assessment technologies.” Tunneling Underground Space Technol. 22 (5–6): 524–542. https://doi.org/10.1016/j.tust.2007.06.001.
Dang, L. M., H. Wang, Y. F. Li, T. N. Nguyen, and H. Moon. 2022. “DefectTR: End-to-end defect detection for sewage networks using a transformer.” Constr. Build. Mater. 325 (Jun): 126584. https://doi.org/10.1016/j.conbuildmat.2022.126584.
Dong, C., C. C. Loy, K. He, and X. Tang. 2015. “Image super-resolution using deep convolutional networks.” IEEE Trans. Pattern Anal. Mach. Intell. 38 (2): 295–307. https://doi.org/10.1109/TPAMI.2015.2439281.
Drews, P. L., E. R. Nascimento, and S. S. Botelho. 2016. “Underwater depth estimation and image restoration based on single images.” J. IEEE Comput. Graph. Appl. 36 (2): 24–35. https://doi.org/10.1109/MCG.2016.26.
Duan, H. F., B. Pan, M. L. Wang, L. Chen, F. F. Zheng, and Y. Zhang. 2020. “State-of-the-art review on the transient flow modeling and utilization for urban water supply system (UWSS) management.” J. Water Supply Res Technol. AQUA 69 (8): 858–893. https://doi.org/10.2166/aqua.2020.048.
Fabbri, C., M. J. Islam, and J. Sattar. 2018. “Enhancing underwater imagery using generative adversarial networks.” In Proc., IEEE Int. Conf. Robotics and Automation, 7159–7165. New York: IEEE. https://doi.org/.10.1109/ICRA.2018.8460552.
Fang, S., R. Deng, Y. Cao, and C. Fang. 2013. “Effective single underwater image enhancement by fusion.” J. Comput. 8 (4): 904–911. https://doi.org/10.4304/jcp.8.4.904-911.
Farhadifard, F., Z. Zhou, and U. F. V. Lukas. 2015. “Learning-based underwater image enhancement with adaptive color mapping.” In Proc., IEEE Int. Symp. on Image and Signal Processing and Analysis, 48–53. New York: IEEE. https://doi.org/10.1155/2010/746052.
Fu, X., Y. Liao, D. Zeng, Y. Huang, X. P. Zhang, and X. Ding. 2015. “A probabilistic method for image enhancement with simultaneous illumination and reflectance estimation.” IEEE Trans. Image Process. 24 (12): 4965–4977. https://doi.org/10.1109/TIP.2015.2474701.
Fu, X., P. Zhuang, Y. Huang, Y. Liao, X. P. Zhang, and X. Ding. 2014. “A retinex-based enhancing approach for single underwater image.” In Proc., IEEE Int. Conf. Image Process, 4572–4576. New York: IEEE. https://doi.org/10.1109/ICIP.2014.7025927.
Fu, Z., W. Wang, Y. Huang, X. Ding, and K. K. Ma. 2022. “Uncertainty inspired underwater image enhancement.” Preprint, submitted July 20, 2022. https://arxiv.org/abs/2207.09689.
Galdran, A., D. Pardo, A. Picn, and A. Alvarez-Gila. 2015. “Automatic red-channel underwater image restoration.” J. Visual Commun. Image Represent. 26 (Apr): 132–145. https://doi.org/10.1016/j.jvcir.2014.11.006.
Ge, Z., S. Liu, F. Wang, and Z. Li. 2021. “YOLOX: Exceeding YOLO series in 2021.” Preprint, submitted July 18, 2021. https://arxiv.org/abs/2107.08430.
Girshick, R., J. Donahue, T. Darrell, and J. Malik. 2014. “Rich feature hierarchies for accurate object detection and semantic segmentation.” In Proc., IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 580–587. New York: IEEE. https://doi.org/10.1109/CVPR.2014.81.
Gkioxari, G., R. Girshick, P. Dollár, and K. M. He. 2017. “Detecting and recognizing human-object interactions.” In Proc., 31st IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 8359–8367. New York: IEEE.
Gonzalez, R. C., and R. E. Woods. 1980. “Digital image processing.” Proc. IEEE Trans. Acoust. Speech Signal Process. 28 (4): 484–486. https://doi.org/10.1109/TASSP.1980.1163437.
Goodfellow, I., J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. “Generative adversarial nets.” In Proc., Advances in Neural Information Processing Systems, 2672–2680. La Jolla, CA: Advances in Neural Information Processing Systems. https://doi.org/10.5555/2969033.2969125.
Gu, J., Z. Wang, J. Kuen, L. Ma, and W. Gang. 2018. “Recent advances in convolutional neural networks.” Pattern Recognit. 77 (Apr): 354–377. https://doi.org/10.1016/j.patcog.2017.10.013.
Halfawy, M. R., and J. Hengmeechai. 2014. “Efficient algorithm for crack detection in sewer images from closed-circuit television inspections.” Infrastruct. Syst. 20 (2): 04013014. https://doi.org/https://doi.org/10.1061/(ASCE)IS.1943-555X.0000161.
Harvey, R. R., and E. A. McBean. 2014. “Comparing the utility of decision trees and support vector machines when planning inspections of linear sewer infrastructure.” Hydroinformatics 16 (6): 1265–1279. https://doi.org/10.2166/hydro.2014.007.
He, K., X. Zhang, S. Ren, and J. Sun. 2015. “Spatial pyramid pooling in deep convolutional networks for visual recognition.” IEEE Trans. Pattern Anal. Mach. Intell. 37 (9): 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824.
He, K. M., G. Gkioxari, P. Dollár, and R. Girshick. 2018. “Mask R-CNN.” Preprint, submitted March 20, 2020. https://arxiv.org/abs/2018.2844175.
Hitam, M. S., E. A. Awalludin, W. N. Yussof, and Z. Bachok. 2013. “Mixture contrast limited adaptive histogram equalization for underwater image enhancement.” In Proc., Int. Conf. on Computer Applications Technology, 1–5. New York: IEEE. https://doi.org/10.1109/ISPA.2015.7306031.
Huynh, P., R. Ross, A. Martchenko, and J. Devlin. 2015. “Dou-edge evaluation algorithm for automatic thin crack detection in pipes.” In Proc., 2015 IEEE Int. Conf. on Signal and Image Processing Applications (ICSIPA). New York: IEEE. https://doi.org/10.1109/ICSIPA.2015.7412188.
Iqbal, K., R. A. Salam, M. Osman, and A. Z. Talib. 2007. “Underwater image enhancement using an integrated colour model.” J. Comput. 32 (2): 239–244.
Isola, P., J. Zhu, T. Zhou, and A. A. Efros. 2017. “Image-to-image translation with conditional adversarial networks.” In Proc., IEEE Conf. Computer Vision Pattern Recognition, 1125–1134. New York: IEEE. https://doi.org/10.1109/CVPR.2017.632.
Khatir, S., S. Tiachacht, C. L. Thanh, T. Q. Bui, and M. A. Wahab. 2019. “Damage assessment in composite laminates using ANN-PSO-IGA and Cornwell indicator.” Compos. Struct. 230 (Apr): 111509. https://doi.org/10.1016/j.compstruct.2019.111509.
Kim, Y. T. 1997. “Contrast enhancement using brightness preserving bi-histogram equalization.” IEEE Trans. Consum. Electron. 48 (1): 1–9. https://doi.org/10.1109/TCE.2002.1010085.
Kimmel, R., M. Elad, D. Shaked, R. Keshet, and I. Sobel. 2003. “A variational framework for Retinex.” Comput. Vis. 52 (1): 7–23. https://doi.org/10.1023/A:1022314423998.
Kingma, D., and J. Ba. 2014. “Adam: A method for stochastic optimization.” Preprint, submitted December 22, 2014. https://arxiv.org/abs/1412.6980.
Kuliczkowska, E. 2016. “An analysis of road pavement collapses and traffic safety hazards resulting from leaky sewers.” Balt. J. Road. Bridge Eng. 11 (4): 251–258. https://doi.org/10.3846/bjrbe.2016.29.
Land, E. H. 1964. “The retinex.” Am. Sci. 52 (2): 247–264.
LeCun, Y., Y. Bengio, and G. Hinton. 2015. “Deep learning.” Nature 521 (7553): 436–444. https://doi.org/10.1038/nature14539.
Li, C., S. Anwar, J. Hou, R. Cong, C. Guo, and W. Ren. 2021. “Underwater image enhancement via medium transmission-guided multi-color space embedding.” IEEE Trans. Image Process. 30 (Apr): 4985–5000. https://doi.org/10.1109/TIP.2021.3076367.
Li, C., J. Guo, and C. Guo. 2018. “Emerging from water: Underwater image color correction based on weakly supervised color transfer.” IEEE Signal Process. Lett. 25 (3): 323–327. https://doi.org/10.1109/LSP.2018.2792050.
Li, C. Y., L. L. Li, H. L. Jiang, and K. H. Weng. 2022. “YOLOv6: A single-stage object detection framework for industrial applications.” Preprint, submitted September 7, 2022. https://arxiv.org/abs/2209.02976.
Li, J., K. A. Skinner, R. M. Eustice, and M. Johnson-Roberson. 2018. “WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images.” IEEE Robot. Autom. Lett. 3 (1): 387–394. https://doi.org/10.1109/LRA.2017.2730363.
Lin, T. Y., P. Dollar, R. Girshick, K. M. He, B. Hariharan, and S. Belongie. 2017. “Feature pyramid networks for object detection.” In Proc., 30th IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 936–944. New York: IEEE. https://doi.org/10.1109/CVPR.2017.106.
Liu, E. B., C. J. Li, S. B. Peng, and H. B. Zhang. 2006. “Pipe blockage detection technology based on pressure wave method.” [In Chinese.] Nat. Gas Ind. 26 (4):112–114. https://doi.org/10.3321/j.issn:1000-0976.2006.04.037.
Liu, H., J. Xu, Y. Wu, Q. Guo, B. Ibragimov, and L. Xing. 2018. “Learning deconvolutional deep neural network for high resolution medical image reconstruction.” Inf. Sci. 468 (Apr): 142–154. https://doi.org/10.1016/j.ins.2018.08.022.
Liu, X. M., S. C. Wang, and J. Zhao. 2010. “Image enhancement algorithm based on wavelet transform and fuzzy set theory.” J. Projectiles Rockets Missiles Guid. 30 (4): 183–186. https://doi.org/10.1016/S1876-3804(11)60008-6.
Makar, J. M. 1999. “Diagnostic techniques for sewer systems.” Infrastruct. Syst. 5 (2): 69–78. https://doi.org/10.1061/(ASCE)1076-0342(1999)5:2(69).
Narasimhan, S. G., and S. K. Nayar. 2003. “Contrast restoration of weather degraded images.” IEEE Trans. Pattern Anal. Mach. Intell. 25 (6): 713–724. https://doi.org/10.1109/TPAMI.2003.1201821.
Ng, M. K., and W. Wang. 2011. “A total variation model for Retinex.” SIAM J. Imaging Sci. 4 (1): 345–365. https://doi.org/10.1137/100806588.
Nguyen, D. H., T. T. Bui, G. D. Roeck, and M. A. Wahab. 2019. “Damage detection in Ca-non bridge using transmissibility and artificial neural networks.” Struct. Eng. Mech. 71 (2): 175–183. https://doi.org/10.12989/sem.2019.71.2.175.
Peng, F., Z. Q. Wang, and J. F. Jiang. 2023. “Burst analysis of water supply pipe based on hydrodynamic simulation.” Water Resour. Manage. 37 (5): 2161–2179. https://doi.org/10.1007/s11269-023-03485-1.
Peng, Y. T., K. Cao, and P. C. Cosman. 2018. “Generalization of the dark channel prior for single image restoration.” IEEE Trans. Image Process. 27 (6): 2856–2868. https://doi.org/10.1109/TIP.2018.2813092.
Peng, Y. T., and P. C. Cosman. 2017. “Underwater image restoration based on image blurriness and light absorption.” IEEE Trans. Image Process. 26 (4): 1579–1594. https://doi.org/10.1109/TIP.2017.2663846.
Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2016. “You only look once: Unified, real-time object detection.” In Proc., 2016 IEEE Conf. on Computer Vision and Pattern Recognition, 779–788. New York: IEEE.
Redmon, J., and A. Farhadi. 2017. “YOLO9000: Better, faster, stronger.” In Proc., 30th IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 6517–6525. New York: IEEE.
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An incremental improvement.” Preprint, submitted April 8, 2018. https://arxiv.org/abs/CVPR.2017.690.
Ren, S. Q., K. M. He, R. Girshick, and J. Sun. 2017. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Ronneberger, O., P. Fischer, and T. Brox. 2015. “U-net: Convolutional networks for biomedical image segmentation.” In Proc., Int. Conf. on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 234–241. Rochester, MN: Medical Image Computing and Computer Assisted Intervention Society.
Schechner, Y. Y., and Y. Averbuch. 2007. “Regularized image recovery in scattering media.” IEEE Trans. Pattern Anal. Mach. Intell. 29 (9): 1655–1660. https://doi.org/10.1109/TPAMI.2007.1141.
Shehab, T., and O. Moselhi. 2005. “Automated detection and classification of infiltration in sewer pipes.” Infrastruct. Syst. 11 (3): 165–171. https://doi.org/10.1061/(ASCE)1076-0342(2005)11:3(165).
Tan, Y., R. Y. Cai, and J. R. Li. 2021. “Automatic detection of sewer defects based on improved you only look once algorithm.” Autom. Constr. 131 (Sep): 103912. https://doi.org/10.1016/j.autcon.2021.103912.
Tran-Ngoc, H., S. Khatir, G. D. Roeck, T. Bui-Tien, and M. A. Wahab. 2019. “An efficient artificial neural network for damage detection in bridges and beam-like structures by improving training parameters using cuckoo search algorithm.” Eng. Struct. 199 (Jun): 109637. https://doi.org/10.1016/j.engstruct.2019.109637.
Tzutalin. 2018. “LabelImg.” Accessed March 21, 2018. https://github.com/tzutalin/labelImg.
Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, and A. N. Gomez. 2017. “Attention is all you need.” Preprint, submitted December 4, 2017. https://arxiv.org/abs/1706.03762.
Walski, T., and E. Creaco. 2016. “Selection of pumping configuration for closed water distribution systems.” J. Water Resour. Plann. Manage. 142 (6): 4016009. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000635.
Wang, S., H. Wang, Y. Zhou, J. Liu, P. Dai, X. Du, and M. A. Wahab. 2021. “Automatic laser profile recognition and fast tracking for structured light measurement using deep learning and template matching.” Measurement 169 (Jun): 108362. https://doi.org/10.1016/j.measurement.2020.108362.
Wen, H., Y. Tian, T. Huang, and W. Gao. 2013. “Single underwater image enhancement with a new optical model.” In Proc., IEEE Int. Symp. on Circuits and Systems, 753–756. New York: IEEE. https://doi.org/10.1109/ISCAS.2013.6571956.
Woo, S., J. Park, and J. Y. Lee. 2018. “CBAM: Convolutional block attention module.” Preprint, submitted July 17, 2018. https://arxiv.org/abs/1807.06521.
Wu, W., Z. Liu, and Y. He. 2015. “Classification of defects with ensemble methods in the automated visual inspection of sewer pipes.” Pattern Anal. Appl. 18 (2): 263–276. https://doi.org/10.1007/s10044-013-0355-5.
Yang, M. D., and T. C. Su. 2008. “Automated diagnosis of sewer pipe defects based on machine learning approaches.” Expert Syst. Appl. 35 (3): 1327–1337. https://doi.org/10.1016/j.eswa.2007.08.013.
Yin, X. F., Y. Chen, A. Bouferguene, H. Zaman, M. Al-Hussein, and L. Kurach. 2020. “A deep learning-based framework for an automated defect detection system for sewer pipes.” Autom. Constr. 109 (Jun): 102967. https://doi.org/10.1016/j.autcon.2019.102967.
Yue, H., J. Yang, X. Sun, F. Wu, and C. Hou. 2017. “Contrast enhancement based on intrinsic image decomposition.” IEEE Trans. Image Process. 26 (8): 3981–3994. https://doi.org/10.1109/TIP.2017.2703078.
Zhu, J., T. Park, P. Isola, and A. A. Efros. 2012. “Unpaired image-to-image translation using cycle-consistent adversarial networks.” In Proc., IEEE Conf. Computer Vision Pattern Recognition, 81–88. New York: IEEE.
Zhuang, P., C. Li, and J. Wu. 2021. “Bayesian retinex underwater image enhancement.” Eng. Appl. Artif. Intell. 101 (1): 104171. https://doi.org/10.1016/j.engappai.2021.104171.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 10 • October 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Dec 20, 2023
Accepted: Apr 12, 2024
Published online: Jul 31, 2024
Published in print: Oct 1, 2024
Discussion open until: Dec 31, 2024
ASCE Technical Topics:
- Automation and robotics
- Defects and imperfections
- Engineering fundamentals
- Infrastructure
- Materials characterization
- Materials engineering
- Municipal water
- Pipeline systems
- Pipelines
- Pipes
- Systems engineering
- Underwater pipelines
- Water (by type)
- Water and water resources
- Water management
- Water pipelines
- Water supply
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.