Geometric Attention Regularization Enhancing Convolutional Neural Networks for Bridge Rubber Bearing Damage Assessment
Publication: Journal of Performance of Constructed Facilities
Volume 35, Issue 5
Abstract
Rubber bearing condition evaluation is crucial for bridge inspection, and the current practice heavily relies on human-vision inspection. Convolutional neural networks (CNNs) have shown great potential for structured damage recognition tasks in recent years; however, this method usually requires a large training data set, which is difficult to collect in practice for rubber bearings. Therefore, methods to improve the performance of CNN for condition classification for elastomeric bearings are necessary. In this paper, a geometric attention regularization (GAR) method is proposed to enhance the performance of CNN for the condition evaluation of rubber bearings. Firstly, the data set of bearings contains different damages that are collected and labeled where the location of the rubber bearing is presented as a bounding box. Then, the location information is utilized to enhance the loss function of CNN in two aspects. On one hand, the bearing location worked as an attention mechanism to indicate the important part of the input image. Besides, it worked as a regularization method to mitigate the effect of overfitting. Experiments using two CNN architectures, including VGG-11 and ResNet-18 trained with transfer learning techniques, are used to evaluate the efficacy of the proposed method. The results show the proposed method is effective to enhance the performance of the CNN model.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or codes used during the study were provided by a third party: all pictures of elastomeric bearings in this study were from Jiangsu Huatong Engineering Company. Direct request for these materials may be made to the provider as indicated in the Acknowledgments.
Acknowledgments
The authors would like to acknowledge the financial support from the National Natural Science Foundation of China (No. 51525801), the Fundamental Research Funds for the Central University, and the Jiangsu Provincial 333 High-Level Talents Educating Project. The authors also gratefully acknowledge the support of Jiangsu Huatong Engineering Company for providing pictures of bridge elastomeric bearings.
References
AASHTO. 2008. Bridging the gap: Restoring and rebuilding the nation’s bridges. Washington, DC: AASHTO.
AASHTO. 2011. AASHTO guide manual for bridge element inspection. Washington, DC: AASHTO.
AASHTO. 2019. Manual for bridge element inspection. Washington, DC: AASHTO.
Abdel-Qader, I., O. Abudayyeh, and M. E. Kelly. 2003. “Analysis of edge-detection techniques for crack identification in bridges.” J. Comput. Civ. Eng. 17 (4): 255–263. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255).
Bang, S., S. Park, H. Kim, and H. Kim. 2019. “Encoder–decoder network for pixel-level road crack detection in black-box images.” Comput. -Aided Civ. Infrastruct. Eng. 34 (8): 713–727. https://doi.org/10.1111/mice.12440.
Beckman, G. H., D. Polyzois, and Y.-J. Cha. 2019. “Deep learning-based automatic volumetric damage quantification using depth camera.” Autom. Constr. 99 (Mar): 114–124. https://doi.org/10.1016/j.autcon.2018.12.006.
Bengio, Y. 2012. “Practical recommendations for gradient-based training of deep architectures.” In Neural networks: Tricks of the trade, 437–478. Berlin: Springer.
Butcher, J. B., C. R. Day, J. C. Austin, P. W. Haycock, D. Verstraeten, and B. Schrauwen. 2014. “Defect detection in reinforced concrete using random neural architectures.” Comput.-Aided Civ. Infrastruct. Eng. 29 (3): 191–207.
Cha, Y. J., W. Choi, and O. Büyüköztürk. 2017. “Deep learning-based crack damage detection using convolutional neural networks.” Comput. -Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10.1111/mice.12263.
Creary, P. A., and F. C. Fang. 2014. “Forecasting long-term bridge deterioration conditions using artificial intelligence techniques.” Int. J. Intell. Syst. Technol. Appl. 13 (4): 280–293. https://doi.org/10.1504/IJISTA.2014.068830.
Dang, J., A. Shrestha, D. Haruta, Y. Tabata, P. Chun, and K. Okubo. 2018. “Site verification tests for UAV bridge inspection and damage image detection based on deep learning.” In Proc., 7th World Conf. on Structural Control and Monitoring. Harbin, China: Harbin Institute of Technology.
Deng, J., W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. “Imagenet: A large-scale hierarchical image database.” In Proc., 2009 IEEE Conf. on Computer Vision and Pattern Recognition, 248–255. New York: IEEE.
Eggert, H., and W. Kauschke. 2002. Structural bearings. Hoboken, NJ: Wiley.
Elsayed, G., S. Kornblith, and Q. V. Le. 2019. “Saccader: Improving accuracy of hard attention models for vision.” In Proc., Advances in Neural Information Processing Systems, 702–714. Vancouver, BC, Canada: NeurIPS.
Everingham, M., L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2010. “The pascal visual object classes (VOC) challenge.” Int. J. Comput. Vison 88 (2): 303–338. https://doi.org/10.1007/s11263-009-0275-4.
Gao, Y., and K. M. Mosalam. 2018. “Deep transfer learning for image-based structural damage recognition.” Comput. -Aided Civ. Infrastruct. Eng. 33 (9): 748–768.
Gopalakrishnan, K., S. K. Khaitan, A. Choudhary, and A. Agrawal. 2017. “Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection.” Constr. Build. Mater. 157 (Dec): 322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110.
Hayashi, K., Y. Adachi, N. Sakamoto, A. Igarashi, and J. Dang. 2015. “Experimental evaluation of seismic residual performance for deteriorated rubber bearings in highway bridges.” In Proc., Joint 6th Int. Conf. on Advancesin Experimental Structural Engineering and Proc., 11th Int. Workshop on Advanced Smart Materials andSmart Structures Technology. Urbana, IL: Univ. of Illinois Urbana-Champaign.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image recognition.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 770–778. New York: IEEE.
Howard, J., and S. Gugger. 2020. “Fastai: A layered API for deep learning.” Information 11 (2): 108. https://doi.org/10.3390/info11020108.
Huang, G., Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. 2017. “Densely connected convolutional networks.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 4700–4708. New York: IEEE.
Huynh, T.-C., J.-H. Park, H.-J. Jung, and J.-T. Kim. 2019. “Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing.” Autom. Constr. 105 (Sep): 102844. https://doi.org/10.1016/j.autcon.2019.102844.
Jaderberg, M., K. Simonyan, and A. Zisserman. 2015. “Spatial transformer networks.” In Proc., Advances in Neural Information Processing Systems, 2017–2025. Vancouver, BC: NeurIPS.
Jiang, X., and H. Adeli. 2007. “Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings.” Int. J. Numer. Methods Eng. 71 (5): 606–629.
Kingma, D. P., and J. Ba. 2014. “ADAM: A method for stochastic optimization.” Accessed December 22, 2014. https://arxiv.org/abs/1412.6980.
Krizhevsky, A., and G. Hinton. 2009. Learning multiple layers of features from tiny images. Toronto: Univ. of Toronto.
LeCun, Y. A., L. Bottou, G. B. Orr, and K.-R. Müller. 2012. “Efficient backprop.” In Neural networks: Tricks of the trade, 9–48. Berlin: Springer.
Lei, B., N. Wang, P. Xu, and G. Song. 2018. “New crack detection method for bridge inspection using UAV incorporating image processing.” J. Aerosp. Eng. 31 (5): 04018058. https://doi.org/10.1061/(ASCE)AS.1943-5525.0000879.
Li, S., X. Zhao, and G. Zhou. 2019. “Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network.” Comput. -Aided Civ. Infrastruct. Eng. 34 (7): 616–634. https://doi.org/10.1111/mice.12433.
Liu, S.-W., J. H. Huang, J.-C. Sung, and C. C. Lee. 2002. “Detection of cracks using neural networks and computational mechanics.” Comput. Methods Appl. Mech. Eng. 191 (25–26): 2831–2845.
Ma, F., H. Li, S. Hou, X. Kang, and G. Wu. 2020. “Defect investigation and replacement implementation of bearings for long-span continuous box girder bridges under operating high-speed railway networks: A case study.” Struct. Infrastruct. Eng. 2020 (Dec): 1–6. https://doi.org/10.1080/15732479.2020.1867589.
Martinez, P., E. Mohamed, O. Mohsen, and Y. Mohamed. 2020. “Comparative study of data mining models for prediction of bridge future conditions.” J. Perform. Constr. Facil. 34 (1): 04019108. https://doi.org/10.1061/(ASCE)CF.1943-5509.0001395.
MLIT (Ministry of Land, Infrastructure, Transport and Tourism). 2015. “White paper on land.” In Infrastructure, transport and tourism in Japan 2015. Tokyo: MLIT.
Mnih, V., N. Heess, and A. Graves. 2014. “Recurrent models of visual attention.” In Proc., Advances in Neural Information Processing Systems, 2204–2212. Vancouver, BC, Canada: NeurIPS.
MOT (Ministry of Transport of the People’s Republic of China). 2019. Traffic survey. Beijing: MOT.
Netzer, Y., T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. 2011. Reading digits in natural images with unsupervised feature learning. Granada, Spain: NeurlPS.
Nishikawa, T., J. Yoshida, T. Sugiyama, and Y. Fujino. 2012. “Concrete crack detection by multiple sequential image filtering.” Comput. -Aided Civ. Infrastruct. Eng. 27 (1): 29–47. https://doi.org/10.1111/j.1467-8667.2011.00716.x.
Pan, S. J., and Q. Yang. 2009. “A survey on transfer learning.” IEEE Trans. Knowl. Data Eng. 22 (10): 1345–1359. https://doi.org/10.1109/TKDE.2009.191.
Simonyan, K., and A. Zisserman. 2014. “Very deep convolutional networks for large-scale image recognition.” Accessed September 4, 2014. https://arxiv.org/abs/1409.1556.
Tang, S., and Z. Chen. 2020. “Scale–Space data augmentation for deep transfer learning of crack damage from small sized datasets.” J. Nondestr. Eval. 39 (3): 1–18. https://doi.org/10.1007/s10921-020-00715-z.
Tieleman, T., and G. Hinton. 2012. “Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude.” COURSERA: Neural Net. Mach. Learn. 4 (2): 26–31.
Tzutalin. 2015. “LabelImg. Git code.” Accessed October 5, 2015. https://github.com/tzutalin/labelImg.
Wah, C., S. Branson, P. Welinder, P. Perona, and S. Belongie. 2011. The Caltech-UCSD birds-200-2011 dataset. Pasadena, CA: California Institute of Technology.
Wang, F., M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang. 2017. “Residual attention network for image classification.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 3156–3164. New York: IEEE.
Wang, N., X. Zhao, P. Zhao, Y. Zhang, Z. Zou, and J. Ou. 2019. “Automatic damage detection of historic masonry buildings based on mobile deep learning.” Autom. Constr. 103 (8): 53–66. https://doi.org/10.1016/j.autcon.2019.03.003.
Yamaguchi, T., and S. Hashimoto. 2010. “Fast crack detection method for large-size concrete surface images using percolation-based image processing.” Mach. Vison Appl. 21 (5): 797–809. https://doi.org/10.1007/s00138-009-0189-8.
Yamaguchi, T., S. Nakamura, R. Saegusa, and S. Hashimoto. 2008. “Image-based crack detection for real concrete surfaces.” Trans. Electr. Electron. Eng. 3 (1): 128–135. https://doi.org/10.1002/tee.20244.
Zhang, K., and H. Cheng. 2017. “A novel pavement crack detection approach using pre-selection based on transfer learning.” In Proc., Int. Conf. on Image and Graphics, 273–283. Berlin: Springer.
Information & Authors
Information
Published In
Journal of Performance of Constructed Facilities
Volume 35 • Issue 5 • October 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Mar 27, 2021
Accepted: May 12, 2021
Published online: Jul 26, 2021
Published in print: Oct 1, 2021
Discussion open until: Dec 26, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Yuchen Liao, Rong Lin, Ruiyang Zhang, Gang Wu, Attention-based LSTM (AttLSTM) neural network for Seismic Response Modeling of Bridges, Computers & Structures, 10.1016/j.compstruc.2022.106915, 275, (106915), (2023).
- Guo-Qing Zhang, Bin Wang, Jun Li, You-Lin Xu, The application of deep learning in bridge health monitoring: a literature review, Advances in Bridge Engineering, 10.1186/s43251-022-00078-7, 3, 1, (2022).
- Fengbo Ma, Lu Zhang, Mida Cui, Xiaoxiang Cheng, Youquan Zou, Yi Cui, Gang Wu, Investigation and treatment of bearing diseases for typical expressway and high-speed railway bridges in Eastern China: a field practice campaign, Structure and Infrastructure Engineering, 10.1080/15732479.2022.2074469, (1-23), (2022).