Hyperparameter Optimization and Importance Ranking in Deep Learning–Based Crack Segmentation
Publication: Journal of Computing in Civil Engineering
Volume 38, Issue 2
Abstract
Although deep convolutional neural networks (DCNNs) have been widely adopted for crack segmentation, they often demonstrate performance degradation on data with real-world complexities. To achieve consistent and accurate prediction performance with complex and feature-rich real-world data, DCNN hyperparameters must be properly selected or optimized. The goal of this study is to provide a novel hyperparameter optimization framework for future crack segmentation DCNN designs to follow, and gain insights into hyperparameter importance on segmentation performance. In this study, a Bayesian optimization framework and an accompanying global sensitivity analysis have been proposed to guide the search for optimal crack segmentation DCNNs using real-world 3D roadway range images. The proposed Bayesian optimization framework can determine the optimal configurations for both training- and architecture-related hyperparameters. In addition, the probabilistic models developed during Bayesian optimization are leveraged by the accompanying global sensitivity analysis to interpret and rank the hyperparameter importance on DCNNs’ segmentation accuracy.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some data, models, or codes that support the findings of this study (e.g., the labeled image data for crack segmentation) are available from the corresponding author by request.
Acknowledgments
The authors thank the following support for the research and publication of this article: Alabama Department of Transportation (Project Number 930-930) and Alabama Transportation Institute (Grant Number: 14560). Any opinions, findings, conclusions, or recommendations expressed in this article are those of the authors, and do not necessarily reflect the views of the above-listed agencies.
References
Akaike, H. 1998. “Information theory and an extension of the maximum likelihood principle.” In Selected papers of Hirotugu Akaike, 199–213. New York: Springer.
Alipour, M., D. K. Harris, and G. R. Miller. 2019. “Robust pixel-level crack detection using deep fully convolutional neural networks.” J. Comput. Civ. Eng. 33 (6): 04019040. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000854.
Bang, S., S. Park, H. Kim, and H. Kim. 2019. “Encoder-decoder network for pixel-level road crack detection in black-box images.” Comput.-Aided Civ. Infrastruct. Eng. 34 (8): 713–727. https://doi.org/10.1111/mice.12440.
Bengio, Y. 2012. “Practical recommendations for gradient-based training of deep architectures.” In Neural networks: Tricks of the trade. 2nd ed. 437–478. New York: Springer.
Bergstra, J., and Y. Bengio. 2012. “Random search for hyper-parameter optimization.” J. Mach. Learn. Res. 13 (10): 281–305.
Bhatt, P. M., R. K. Malhan, P. Rajendran, B. C. Shah, S. Thakar, Y. J. Yoon, and S. K. Gupta. 2021. “Image-based surface defect detection using deep learning: A review.” J. Comput. Inf. Sci. Eng. 21 (4): 040801.https://doi.org/10.1115/1.4049535.
Brochu, E., V. M. Cora, and N. D. Freitas. 2010. “A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning.” Preprint, submitted December 12, 2010. http://arxiv.org/abs/1012.2599.
Cao, W., Q. Liu, and Z. He. 2020. “Review of pavement defect detection methods.” IEEE Access 8 (Mar): 14531–14544. https://doi.org/10.1109/ACCESS.2020.2966881.
Cha, Y. J., W. Choi, and O. Büyüköztürk. 2017. “Deep learning-based crack damage detection using convolutional neural networks.” Comput.-Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10.1111/mice.12263.
Cha, Y. J., W. Choi, G. Suh, S. Mahmoudkhani, and O. Büyüköztürk. 2018. “Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types.” Comput.-Aided Civ. Infrastruct. Eng. 33 (9): 731–747. https://doi.org/10.1111/mice.12334.
Chen, K., G. Reichard, X. Xu, and A. Akanmu. 2021. “Automated crack segmentation in close-range building façade inspection images using deep learning techniques.” J. Build. Eng. 43 (Nov): 102913. https://doi.org/10.1016/j.jobe.2021.102913.
Cheng, J., W. Xiong, W. Chen, Y. Gu, and Y. Li. 2018. “Pixel-level crack detection using U-Net.” In Proc., TENCON 2018-2018 IEEE Region 10 Conf., 0462–0466. New York: IEEE.
Choi, W., and Y.-J. Cha. 2020. “SDDNet: Real-time crack segmentation.” IEEE Trans. Ind. Electron. 67 (9): 8016–8025. https://doi.org/10.1109/TIE.2019.2945265.
Claesen, M., and B. Moor. 2015. “Hyperparameter search in machine learning.” Preprint, submitted February 7, 2015. http://arxiv.org/abs/1502.02127.
Csurka, G., D. Larlus, F. Perronnin, and F. Meylan. 2013. “What is a good evaluation measure for semantic segmentation?” In Proc., 24th British Machine Vision Conf. Durham, UK: British Machine Vision Association.
Cui, X., Q. Wang, J. Dai, Y. Xue, and Y. Duan. 2021. “Intelligent crack detection based on attention mechanism in convolution neural network.” Adv. Struct. Eng. 24 (9): 1859–1868. https://doi.org/10.1177/1369433220986638.
D’Amico, G., R. Rabadan, and M. Kleban. 2017. “A random categorization model for hierarchical taxonomies.” Sci. Rep. 7 (1): 17051. https://doi.org/10.1038/s41598-017-17168-6.
Deng, J., Y. Lu, and V. C.-S. Lee. 2020. “Imaging-based crack detection on concrete surfaces using You Only Look Once network.” Struct. Health Monit. 20 (2): 484–499. https://doi.org/10.1177/1475921720938486.
Dumoulin, V., and F. Visin. 2016. “A guide to convolution arithmetic for deep learning.” Preprint, submitted March 23, 2016. http://arxiv.org/abs/1603.07285.
Elsken, T., J. H. Metzen, and F. Hutter. 2019. “Neural architecture search.” In Automated machine learning. New York: Springer.
Fei, Y., K. C. P. Wang, A. Zhang, C. Chen, J. Q. Li, Y. Liu, G. Yang, and B. Li. 2020. “Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based crackNet-V.” IEEE Trans. Intell. Transp. Syst. 21 (1): 273–284. https://doi.org/10.1109/TITS.2019.2891167.
Feurer, M., and F. Hutter. 2019. “Hyperparameter optimization.” In Automated machine learning. 3–33. New York: Springer.
Frazier, P. I. 2018. “A tutorial on Bayesian optimization.” Preprint, submitted July 8, 2018. http://arxiv.org/abs/1804.087381807.02811.
Gelbart, M. A., J. Snoek, and R. P. Adams. 2014. “Bayesian optimization with unknown constraints.” Preprint, submitted March 22, 2014. https://arxiv.org/abs/1403.5607.
Ghosh, R., and O. Smadi. 2021. “Automated detection and classification of pavement distresses using 3D pavement surface images and deep learning.” Transp. Res. Rec. 2675 (9): 1359–1374.https://doi.org/10.1177/03611981211007481.
Goodfellow, I., Y. Bengio, and A. Courville. 2016. Deep learning. Cambridge, UK: MIT Press.
Gopalakrishnan, K. 2018. “Deep learning in data-driven pavement image analysis and automated distress detection: A review.” Data 3 (3): 28. https://doi.org/10.3390/data3030028.
Gopalakrishnan, K., S. K. Khaitan, A. Choudhary, and A. Agrawal. 2017. “Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection.” Constr. Build. Mater. 157 (Mar): 322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110.
He, K., and J. Sun. 2015. “Convolutional neural networks at constrained time cost.” In Proc., 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 5353–5360. New York: IEEE.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image recognition.” In Proc., 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 770–778. New York: IEEE.
Hsieh, Y. A., and Y. J. Tsai. 2020. “Machine learning for crack detection: Review and model performance comparison.” J. Comput. Civ. Eng. 34 (5): 04020038. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000918.
Hutter, F., H. Hoos, and K. Leyton-Brown. 2014. “An efficient approach for assessing hyperparameter importance.” In Proc., 31st Int. Conf. on Machine Learning, edited by P. X. Eric and J. Tony, 754–762. Washington, DC: Proceedings of Machine Learning Research.
Islam, M. M. M., and J. M. Kim. 2019. “Vision-based autonomous crack detection of concrete structures using a fully convolutional encoder-decoder network.” Sensors 19 (19): 4251. https://doi.org/10.3390/s19194251.
Kheradmandi, N., and V. Mehranfar. 2022. “A critical review and comparative study on image segmentation-based techniques for pavement crack detection.” Constr. Build. Mater. 321 (Feb): 126162. https://doi.org/10.1016/j.conbuildmat.2021.126162.
König, J., M. D. Jenkins, M. Mannion, P. Barrie, and G. Morison. 2021. “Optimized deep encoder-decoder methods for crack segmentation.” Digit. Signal Process. 108 (Jan): 102907. https://doi.org/10.1016/j.dsp.2020.102907.
König, J., M. D. Jenkins, M. Mannion, P. Barrie, and G. Morison. 2022. “What’s cracking? A review and analysis of deep learning methods for structural crack segmentation, detection and quantification.” Preprint, submitted February 8, 2022. http://arxiv.org/abs/2202.03714.
Kortmann, F., K. Talits, P. Fassmeyer, A. Warnecke, N. Meier, J. Heger, P. Drews, and B. Funk. 2020. “Detecting various road damage types in global countries utilizing faster R-CNN.” In Proc., 2020 IEEE Int. Conf. on Big Data (Big Data), 5563–5571. New York: IEEE.
Kullback, S., and R. A. Leibler. 1951. “On information and sufficiency.” Ann. Math. Stat. 22 (1): 79–86. https://doi.org/10.1214/aoms/1177729694.
Lau, S. L. H., E. K. P. Chong, X. Yang, and X. Wang. 2020. “Automated pavement crack segmentation using U-net-based convolutional neural network.” IEEE Access 8 (Mar): 114892–114899. https://doi.org/10.1109/ACCESS.2020.3003638.
Li, G., X. Li, J. Zhou, D. Liu, and W. Ren. 2021. “Pixel-level bridge crack detection using a deep fusion about recurrent residual convolution and context encoder network.” Measurement 176 (May):109171. https://doi.org/10.1016/j.measurement.2021.109171.
Liang, X. 2018. “Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization.” Comput.-Aided Civ. Infrastruct. Eng. 34 (5): 415–430. https://doi.org/10.1111/mice.12425.
Liu, C.-Y., and J.-S. Chou. 2023. “Bayesian-optimized deep learning model to segment deterioration patterns underneath bridge decks photographed by unmanned aerial vehicle.” Autom. Constr. 146 (Feb): 104666. https://doi.org/10.1016/j.autcon.2022.104666.
Liu, F., and L. Wang. 2022. “UNet-based model for crack detection integrating visual explanations.” Constr. Build. Mater. 322 (Mar): 126265. https://doi.org/10.1016/j.conbuildmat.2021.126265.
Liu, K., X. Han, and B. M. Chen. 2019a. “Deep learning based automatic crack detection and segmentation for unmanned aerial vehicle inspections.” In Proc., 2019 IEEE Int. Conf. on Robotics and Biomimetics (ROBIO), 381–387. New York: IEEE.
Liu, Y., J. Yao, X. Lu, R. Xie, and L. Li. 2019b. “DeepCrack: A deep hierarchical feature learning architecture for crack segmentation.” Neurocomputing 338 (Mar): 139–153. https://doi.org/10.1016/j.neucom.2019.01.036.
Liu, Z., Y. Cao, Y. Wang, and W. Wang. 2019c. “Computer vision-based concrete crack detection using U-net fully convolutional networks.” Autom. Constr. 104 (Aug): 129–139. https://doi.org/10.1016/j.autcon.2019.04.005.
Maeda, H., Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata. 2018. “Road damage detection and classification using deep neural networks with smartphone images.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1127–1141. https://doi.org/10.1111/mice.12387.
Majidifard, H., P. Jin, Y. Adu-Gyamfi, and W. G. Buttlar. 2020. “Pavement image datasets: A new benchmark dataset to classify and densify pavement distresses.” Transp. Res. Rec. 2674 (2): 328–339. https://doi.org/10.1177/0361198120907283.
Maniat, M., C. V. Camp, and A. R. Kashani. 2021. “Deep learning-based visual crack detection using Google Street View images.” Neural Comput. Appl. 33 (21): 14565–14582.https://doi.org/10.1007/s00521-021-06098-0.
MATHWORKS. 2021a. MATLAB deep learning toolbox. Natick, MA: MathWorks Inc.
MATHWORKS. 2021b. MATLAB statistics and machine learning toolbox. Natick, MA: MathWorks Inc.
Mei, Q., and M. Gül. 2020. “Multi-level feature fusion in densely connected deep-learning architecture and depth-first search for crack segmentation on images collected with smartphones.” Struct. Health Monit. 19 (6): 1726–1744. https://doi.org/10.1177/1475921719896813.
Mei, Q., M. Gül, and M. R. Azim. 2020. “Densely connected deep neural network considering connectivity of pixels for automatic crack detection.” Autom. Constr. 110 (Feb):103018. https://doi.org/10.1016/j.autcon.2019.103018.
Mohammed, M. A., Z. Han, Y. Li, and P. Kłosowski. 2021. “Exploring the detection accuracy of concrete cracks using various CNN models.” Adv. Mater. Sci. Eng. 2021 (Mar): 1–11. https://doi.org/10.1155/2021/9923704.
Murphy, K. P. 2012. Machine learning: A probabilistic perspective. Cambridge, MA: MIT Press.
Naddaf-Sh, M. M., S. Hosseini, J. Zhang, N. A. Brake, and H. Zargarzadeh. 2019. “Real-time road crack mapping using an optimized convolutional neural network.” Complexity 2019 (Sep): 1–17. https://doi.org/10.1155/2019/2470735.
Pan, Y., G. Zhang, and L. Zhang. 2020. “A spatial-channel hierarchical deep learning network for pixel-level automated crack detection.” Autom. Constr. 119 (Nov): 103357. https://doi.org/10.1016/j.autcon.2020.103357.
Park, S., S. Bang, H. Kim, and H. Kim. 2019. “Patch-based crack detection in black box images using convolutional neural networks.” J. Comput. Civ. Eng. 33 (3): 04019017. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000831.
Radosavovic, I., J. Johnson, S. Xie, W.-Y. Lo, and P. Dollar. 2019. “On network design spaces for visual recognition.” In 2019 IEEE/CVF Int. Conf. on Computer Vision (ICCV), 1882–1890. New York: IEEE.
Rasmussen, C. E., and C. K. Williams. 2006. Gaussian processes for machine learning. Cambridge, MA: MIT Press.
Sajedi, S., and X. Liang. 2019. “A convolutional cost-sensitive crack localization algorithm for automated and reliable RC bridge inspection.” In Proc., Risk-Based Bridge Engineering: Proc., 10th New York City Bridge Conf. 229. Boca Raton, FL: CRC Press.
Saltelli, A. 2002. “Making best use of model evaluations to compute sensitivity indices.” Comput. Phys. Commun. 145 (2): 280–297. https://doi.org/10.1016/S0010-4655(02)00280-1.
Saltelli, A., P. Annoni, I. Azzini, F. Campolongo, M. Ratto, and S. Tarantola. 2010. “Variance based sensitivity analysis of model output: Design and estimator for the total sensitivity index.” Comput. Phys. Commun. 181 (2): 259–270. https://doi.org/10.1016/j.cpc.2009.09.018.
Snoek, J., H. Larochelle, and R. P. Adams. 2012. “Practical bayesian optimization of machine learning algorithms.” Preprint, submitted June 13, 2012. https://arxiv.org/abs/1206.2944.
Tan, M., and Q. Le. 2019. “EfficientNet: Rethinking model scaling for convolutional neural networks.” In Proc., 36th Int. Conf. on Machine Learning, edited by C. Kamalika and S. Ruslan, 6105–6114. Washington, DC: Proceedings of Machine Learning Research.
Taylor, R., V. Ojha, I. Martino, and G. Nicosia. 2021. “Sensitivity analysis for deep learning: Ranking hyper-parameter influence.” In Proc., 2021 IEEE 33rd Int. Conf. on Tools with Artificial Intelligence (ICTAI), 512–516. New York: IEEE.
Wang, C. P., and B. Jo. 2013. “Applications of a Kullback-Leibler divergence for comparing non-nested models.” Stat. Modell. 13 (5–6): 409–429. https://doi.org/10.1177/1471082X13494610.
Wang, J. J., Y. F. Liu, X. Nie, and Y. L. Mo. 2021. “Deep convolutional neural networks for semantic segmentation of cracks.” Struct. Control Health Monit. 29 (1): e2850. https://doi.org/10.1002/stc.2850.
Wang, X., and Z. Hu. 2017. “Grid-based pavement crack analysis using deep learning.” In Proc., 2017 4th Int. Conf. on Transportation Information and Safety (ICTIS), 917–924. New York: IEEE. https://doi.org/10.1109/ICTIS.2017.8047878.
Wu, G., H. Zhang, Y. He, X. Bao, L. Li, and X. Hu. 2019. “Learning Kullback-Leibler divergence-based gaussian model for multivariate time series classification.” IEEE Access 7 (Mar): 139580–139591. https://doi.org/10.1109/ACCESS.2019.2943474.
Wu, J., Y. Zhang, and X. Zhao. 2021. “A review of image-based pavement crack detection algorithms.” In Proc., 2021 40th Chinese Control Conf. (CCC), 7300–7306. New York: IEEE. https://doi.org/10.23919/CCC52363.2021.9549966.
Yang, L., and A. Shami. 2020. “On hyperparameter optimization of machine learning algorithms: Theory and practice.” Neurocomputing 415 (Nov): 295–316. https://doi.org/10.1016/j.neucom.2020.07.061.
Yang, X., H. Li, Y. Yu, X. Luo, T. Huang, and X. Yang. 2018. “Automatic pixel-level crack detection and measurement using fully convolutional network.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1090–1109. https://doi.org/10.1111/mice.12412.
Yu, T., and H. Zhu. 2020. “Hyper-parameter optimization: A review of algorithms and applications.” Preprint, submitted March 12, 2020. http://arxiv.org/abs/2003.05689.
Zhang, A., K. C. P. Wang, Y. Fei, Y. Liu, S. Tao, C. Chen, J. Q. Li, and B. Li. 2018. “Deep learning–based fully automated pavement crack detection on 3D asphalt surfaces with an improved Cracknet.” J. Comput. Civ. Eng. 32 (5): 04018041. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000775.
Zhang, A., K. C. P. Wang, B. Li, E. Yang, X. Dai, Y. Peng, Y. Fei, Y. Liu, J. Q. Li, and C. Chen. 2017. “Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network.” Comput.-Aided Civ. Infrastruct. Eng. 32 (10): 805–819. https://doi.org/10.1111/mice.12297.
Zhang, L., J. Shen, and B. Zhu. 2020. “A research on an improved Unet-based concrete crack detection algorithm.” Struct. Health Monit. 20 (4): 1864–1879. https://doi.org/10.1177/1475921720940068.
Zhang, L., F. Yang, Y. D. Zhang, and Y. J. Zhu. 2016. “Road crack detection using deep convolutional neural network.” In Proc., 2016 IEEE Int. Conf. on Image Processing (ICIP), 3708–3712. New York: IEEE.
Zhang, X., D. Rajan, and B. Story. 2019. “Concrete crack detection using context-aware deep semantic segmentation network.” Comput.-Aided Civ. Infrastruct. Eng. 34 (11): 951–971. https://doi.org/10.1111/mice.12477.
Zhao, R., B. Qian, X. Zhang, Y. Li, R. Wei, Y. Liu, and Y. Pan. 2020. “Rethinking dice loss for medical image segmentation.” In Proc., 2020 IEEE Int. Conf. on Data Mining (ICDM), 851–860. New York: IEEE.
Zhou, S., C. Canchila, and W. Song. 2023. “Deep learning-based crack segmentation for civil infrastructure: Data types, architectures, and benchmarked performance.” Autom. Constr. 146 (Feb): 104678. https://doi.org/10.1016/j.autcon.2022.104678.
Zhou, S., and W. Song. 2020a. “Concrete roadway crack segmentation using encoder-decoder networks with range images.” Autom. Constr. 120 (Dec): 103403. https://doi.org/10.1016/j.autcon.2020.103403.
Zhou, S., and W. Song. 2020b. “Deep learning-based roadway crack classification using laser-scanned range images: A comparative study on hyperparameter selection.” Autom. Constr. 114 (Jun): 103171. https://doi.org/10.1016/j.autcon.2020.103171.
Zhou, S., and W. Song. 2020c. “Deep learning–based roadway crack classification with heterogeneous image data fusion.” Struct. Health Monit. 20 (3): 1274–1293. https://doi.org/10.1177/1475921720948434.
Zhou, S., and W. Song. 2020d. “Robust image-based surface crack detection using range data.” J. Comput. Civ. Eng. 34 (2): 04019054. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000873.
Zhou, S., and W. Song. 2021. “Crack segmentation through deep convolutional neural networks and heterogeneous image fusion.” Autom. Constr. 125 (May): 103605. https://doi.org/10.1016/j.autcon.2021.103605.
Zou, Q., Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang. 2018. “Deepcrack: Learning hierarchical convolutional features for crack detection.” IEEE Trans. Image Process. 28 (3): 1498–1512. https://doi.org/10.1109/TIP.2018.2878966.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 38 • Issue 2 • March 2024
Copyright
© 2023 American Society of Civil Engineers.
History
Received: May 17, 2023
Accepted: Oct 5, 2023
Published online: Dec 13, 2023
Published in print: Mar 1, 2024
Discussion open until: May 13, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Hong Lang, Jinsong Qian, Ye Yuan, Jiang Chen, Yingying Xing, Aidi Wang, Automatic Pixel-Level Segmentation of Multiple Pavement Distresses and Surface Design Features with PDSNet II, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5894, 38, 6, (2024).