Computer Vision–Based Counting Model for Dense Steel Pipe on Construction Sites
Publication: Journal of Construction Engineering and Management
Volume 148, Issue 1
Abstract
Building material inventory is routine work of the material delivery process at most construction sites. Manual counting is the conventional manner of taking inventory; however, it is subjective, time consuming, and error prone, especially for densely stacked material. This study proposes a new and accurate counting model based on YOLOv3 to automatically and efficiently count dense steel pipes by images. To promote counting models’ development and verification, a large-scale steel pipe image data set including various on-site conditions was constructed and publicly available. The proposed model was observed to be superior to the original YOLOv3 detector in terms of average precision, mean absolute error, and root-mean-square error based on the steel pipe data set. Furthermore, several improvement measures, split into bag of specials and bag of freebies, were introduced to enhance counting performance further and verified by an ablation study. Comparisons with other popular detectors demonstrate the effectiveness and superiority of the proposed model for counting densely stacked steel pipes. The counting model can be easily extended for other dense material counting and integrated into mobile devices for practical application at construction sites.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request. The data sets are available online at https://github.com/lyang95 after the paper is published.
Acknowledgments
The authors would like to acknowledge the financial support provided by the National Natural Science Foundation of China (Grant No. U1711264). The authors also appreciate the assistance from many student volunteers for annotation work.
References
Alcaide, E. 2018. “E-swish: Adjusting activations to different network depths.” Preprint, submitted January 22, 2018. https://arxiv.org/abs/1801.07145.
Atha, D. J., and M. R. Jahanshahi. 2018. “Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection.” Struct. Health Monit. 17 (5): 1110–1128. https://doi.org/10.1177/1475921717737051.
Baldwin, A. N., A. Thorpe, and J. A. Alkaabi. 1994. “Improved materials management through bar-coding: Results and implications from a feasibility study.” Proc. Inst. Civ. Eng. Civ. Eng. 102 (4): 156–162. https://doi.org/10.1680/icien.1994.27249.
Ben-Baruch, E., T. Ridnik, N. Zamir, A. Noy, I. Friedman, M. Protter, and L. Zelnik-Manor. 2020. “Asymmetric loss for multi-label classification.” Preprint, submitted September 29, 2020. https://arxiv.org/abs/2009.14119.
Bochkovskiy, A., C.-Y. Wang, and H.-Y. M. Liao. 2020. “YOLOv4: Optimal speed and accuracy of object detection.” Preprint, submitted April 23, 2020. https://arxiv.org/abs/2004.10934.
Cai, Z., and N. Vasconcelos. 2019. “Cascade R-CNN: High quality object detection and instance segmentation.” IEEE Trans. Pattern Anal. Mach. Intell. 43 (5): 1483–1498. https://doi.org/10.1111/mice.12263.
Cha, Y., W. Choi, and O. Büyüköztürk. 2017. “Deep learning-based crack damage detection using convolutional neural networks.” Comput.-Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10.1111/mice.12263.
Chen, D., and K. Xu. 2020. “AReLU: Attention-based rectified linear unit.” Preprint, submitted June 24, 2020. https://arxiv.org/abs/2006.13858.
Chen, L.-C., G. Papandreou, F. Schroff, and H. Adam. 2017. “Rethinking atrous convolution for semantic image segmentation.” Preprint, submitted June 17, 2017. https://arxiv.org/abs/1706.05587.
Duan, K. K., and S. Y. Cao. 2020. “Emerging RFID technology in structural engineering—A review.” Structures 28 (Dec): 2404–2414. https://doi.org/10.1016/j.istruc.2020.10.036.
Echeverry, D., and A. Beltran. 1997. “Barcode control of construction field personnel and construction materials.” In Proc., 4th Congress in Computing in Civil Engineering, 341–347. Reston, VA: ASCE.
Everingham, M., L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2010. “The PASCAL visual object classes (VOC) challenge.” Int. J. Comput. Vision 88 (2): 303–338. https://doi.org/10.1007/s11263-009-0275-4.
Fan, Z., J. Lu, B. Qiu, T. Jiang, K. An, A. N. Josephraj, and C. Wei. 2019. “Automated steel bar counting and center localization with convolutional neural networks.” Preprint, submitted June 3, 2019. https://arxiv.org/abs/1906.00891.
Fang, Q., H. Li, X. Luo, L. Ding, H. Luo, T. M. Rose, and W. An. 2018. “Detecting non-hardhat-use by a deep learning method from far-field surveillance videos.” Autom. Constr. 85 (Jan): 1–9. https://doi.org/10.1016/j.autcon.2017.09.018.
George, C., and J. Wolfer. 2006. “A swarm intelligence approach to counting stacked symmetric objects.” In Proc., Artificial Intelligence and Applications, 125–130. Innsbruck, Austria: Acta Press.
Ghazali, M. F., L.-K. Wong, and J. See. 2017. “Automatic detection and counting of circular and rectangular steel bars.” In Proc., 9th Int. Conf. on Robotic, Vision, Signal Processing and Power Applications, 199–207. Singapore: Springer.
Ghiasi, G., T.-Y. Lin, and Q. V. Le. 2018. “DropBlock: A regularization method for convolutional networks.” Preprint, submitted October 30, 2018. https://arxiv.org/abs/1810.12890.
Gopalakrishnan, K., S. K. Khaitan, A. Choudhary, and A. Agrawal. 2017. “Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection.” Constr. Build. Mater. 157 (Dec): 322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110.
Goyal, P., P. Dollár, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. 2017. “Accurate, large minibatch SGD: Training ImageNet in 1 hour.” Preprint, submitted June 8, 2017. https://arxiv.org/abs/1706.02677.
Grau, D., L. Zeng, and Y. Xiao. 2012. “Automatically tracking engineered components through shipping and receiving processes with passive identification technologies.” Autom. Constr. 28 (Dec): 36–44. https://doi.org/10.1016/j.autcon.2012.05.016.
Guo, Y., Y. Xu, and S. Li. 2020. “Dense construction vehicle detection based on orientation-aware feature fusion convolutional neural network.” Autom. Constr. 112 (Apr): 103124. https://doi.org/10.1016/j.autcon.2020.103124.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image recognition.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 770–778. New York: IEEE.
Hendrycks, D., and K. Gimpel. 2016. “Gaussian error linear units (GELUs).” Preprint, submitted June 27, 2016. https://arxiv.org/abs/1606.08415.
Hernández-Ruiz, A. C., J. A. Martínez-Nieto, and J. D. Buldain-Pérez. 2021. “Steel bar counting from images with machine learning.” Electronics 10 (4): 402. https://doi.org/10.3390/electronics10040402.
Hou, W., Z. Duan, and X. Liu. 2011. “A template-covering based algorithm to count the bundled steel bars.” In Proc., 4th Int. Congress on Image and Signal Processing (CISP 2011), 1813–1816. New York: IEEE.
Howard, A., M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, and V. Vasudevan. 2019. “Searching for MobileNetV3.” In Proc., IEEE Int. Conf. on Computer Vision, 1314–1324. New York: IEEE.
Hurtik, P., V. Molek, J. Hula, M. Vajgl, P. Vlasanek, and T. Nejezchleba. 2020. “Poly-YOLO: Higher speed, more precise detection and instance segmentation for YOLOv3.” Preprint, submitted May 27, 2020. https://arxiv.org/abs/2005.13243.
Huyan, J., W. Li, S. Tighe, Z. Xu, and J. Zhai. 2020. “CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection.” Struct. Control. Health Monit. 27 (8): e2551. https://doi.org/10.1002/stc.2551.
Jaselskis, E. J., and T. El-Misalami. 2003. “Implementing radio frequency identification in the construction process.” J. Constr. Eng. Manage. 129 (6): 680–688. https://doi.org/10.1061/(ASCE)0733-9364(2003)129:6(680).
Ju, Y., C. Kim, and H. Kim. 2012. “RFID and CCTV-based material delivery monitoring for cable-stayed bridge construction.” J. Comput. Civ. Eng. 26 (2): 183–190. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000134.
Kar, S., and K. N. Jha. 2020. “Examining the effect of material management issues on the schedule and cost performance of construction projects based on a structural equation model: Survey of Indian experiences.” J. Constr. Eng. Manage. 146 (9): 05020011. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001906.
Kasim, N., N. Sarpin, H. M. Noh, R. Zainal, S. Mohamed, N. Manap, and M. Y. Yahya. 2019. “Automatic materials tracking practices through RFID implementation in construction projects.” In Vol. 266 of Proc., MATEC Web of Conf., 05001. Paris: EDP Sciences. https://doi.org/10.1051/matecconf/201926605001.
Lee, J. H., and S. O. Park. 2019. “Machine learning-based automatic reinforcing bar image analysis system in the internet of things.” Multimedia Tools Appl. 78 (3): 3171–3180. https://doi.org/10.1007/s11042-018-5984-7.
Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollar. 2017. “Focal loss for dense object detection.” In Proc., IEEE Int. Conf. on Computer Vision, 2999–3007. New York: IEEE.
Lin, T. Y., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. 2014. “Microsoft COCO: Common objects in context.” In Proc., European Conf. on Computer Vision, 740–755. Cham, Switzerland: Springer.
Liu, S., D. Huang, and Y. Wang. 2019. “Learning spatial fusion for single-shot object detection.” Preprint, submitted November 21, 2019. https://arxiv.org/abs/1911.09516.
Liu, W., D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg. 2016. “SSD: Single shot multibox detector.” In Proc., European Conf. on Computer Vision, 21–37. Cham, Switzerland: Springer.
Loshchilov, I., and F. Hutter. 2016. “SGDR: Stochastic gradient descent with warm restarts.” Preprint, submitted August 13, 2016. https://arxiv.org/abs/1608.03983.
Lu, W., G. Q. Huang, and H. Li. 2011. “Scenarios for applying RFID technology in construction project management.” Autom. Constr. 20 (2): 101–106. https://doi.org/10.1016/j.autcon.2010.09.007.
Maas, A. L., A. Y. Hannun, and A. Y. Ng. 2013. “Rectifier nonlinearities improve neural network acoustic models.” In Proc., ICML Workshop on Deep Learning for Audio, Speech and Language Processing, 3. Atlanta: JMLR.org.
Misra, D. 2019. “Mish: A self regularized non-monotonic activation function.” Preprint, submitted August 23, 2019. https://arxiv.org/abs/1908.08681.
Misra, D., T. Nalamada, A. U. Arasanipalai, and Q. Hou. 2021. “Rotate to attend: Convolutional triplet attention module.” In Proc., IEEE Winter Conf. on Applications of Computer Vision, 3139–3148. New York: IEEE.
Moon, S., S. Xu, L. Hou, C. Wu, X. Wang, and V. W. Y. Tam. 2018. “RFID-aided tracking system to improve work efficiency of scaffold supplier: Stock management in Australasian supply chain.” J. Constr. Eng. Manage. 144 (2): 04017115. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001432.
Nath, N. D., A. H. Behzadan, and S. G. Paal. 2020. “Deep learning for site safety: Real-time detection of personal protective equipment.” Autom. Constr. 112 (Apr): 103085. https://doi.org/10.1016/j.autcon.2020.103085.
Navon, R., and O. Berkovich. 2005. “Development and on-site evaluation of an automated materials management and control model.” J. Constr. Eng. Manage. 131 (12): 1328–1336. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:12(1328).
Navon, R., and O. Berkovich. 2006. “An automated model for materials management and control.” Construct. Manage. Econ. 24 (6): 635–646. https://doi.org/10.1080/01446190500435671.
Ni, F., J. Zhang, and Z. Chen. 2019. “Pixel-level crack delineation in images with convolutional feature fusion.” Struct. Control. Health Monit. 26 (1): e2286. https://doi.org/10.1002/stc.2286.
Pang, J., K. Chen, J. Shi, H. Feng, W. Ouyang, and D. Lin. 2019. “Libra r-CNN: Towards balanced learning for object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 821–830. New York: IEEE.
Ramachandran, P., B. Zoph, and Q. V. Le. 2017. “Searching for activation functions.” Preprint, submitted October 16, 2017. https://arxiv.org/abs/1710.05941.
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An incremental improvement.” Preprint, submitted April 8, 2018. https://arxiv.org/abs/1804.02767.
Ren, S., K. He, R. Girshick, and J. Sun. 2016. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Ren, Y., and H. Li. 2018. “Building materials management system based on RFID technology.” Int. J. RF Technol. Res. Appl. 9 (1–2): 63–74. https://doi.org/10.3233/RFT-180133.
Ren, Z., C. J. Anumba, and J. Tah. 2011. “RFID-facilitated construction materials management (RFID-CMM)—A case study of water-supply project.” Adv. Eng. Inform. 25 (2): 198–207. https://doi.org/10.1016/j.aei.2010.02.002.
Russakovsky, O., et al. 2015. “ImageNet large scale visual recognition challenge.” Int. J. Comput. Vision 115 (3): 211–252. https://doi.org/10.1007/s11263-015-0816-y.
Simonyan, K., and A. Zisserman. 2014. “Very deep convolutional networks for large-scale image recognition.” Preprint, submitted September 4, 2014. https://arxiv.org/abs/1409.1556.
Song, J., C. T. Haas, C. Caldas, E. Ergen, and B. Akinci. 2006. “Automating the task of tracking the delivery and receipt of fabricated pipe spools in industrial projects.” Autom. Constr. 15 (2): 166–177. https://doi.org/10.1016/j.autcon.2005.03.001.
Su, Z., K. Fang, Z. Peng, and Z. Feng. 2010. “Rebar automatically counting on the product line.” In Proc., 2010 Int. Conf. on Progress in Informatics and Computing, 756–760. New York: IEEE.
Thammasorn, P., S. Boonchu, and A. Kawewong. 2013. “Real-time method for counting unseen stacked objects in mobile.” In Proc., 20th IEEE Int. Conf. on Image Processing, 4103–4107. New York: IEEE.
Tian, Z., C. Shen, H. Chen, and T. He. 2019. “FCOS: Fully convolutional one-stage object detection.” In Proc., IEEE Int. Conf. on Computer Vision, 9627–9636. New York: IEEE.
Torrent, D. G., and C. H. Caldas. 2009. “Methodology for automating the identification and localization of construction components on industrial projects.” J. Comput. Civ. Eng. 23 (1): 3–13. https://doi.org/10.1061/(ASCE)0887-3801(2009)23:1(3).
Wang, C.-Y., H.-Y. M. Liao, Y.-H. Wu, P.-Y. Chen, J.-W. Hsieh, and I.-H. Yeh. 2020. “CSPNet: A new backbone that can enhance learning capability of CNN.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition Workshops, 390–391. New York: IEEE.
Wang, H., J. Polden, J. Jirgens, Z. Yu, and Z. Pan. 2019. “Automatic rebar counting using image processing and machine learning.” In Proc., IEEE 9th Annual Int. Conf. on CYBER Technology in Automation, Control, and Intelligent Systems, 900–904. Piscataway, NJ: IEEE.
Wang, J., H. Chen, and X. Xu. 2011. “Pattern recognition for counting of bounded bar steel.” In Proc., Fourth Int. Conf. on the Applications of Digital Information and Web Technologies (ICADIWT 2011), 173–176. New York: IEEE.
Wang, L. C., Y. C. Lin, and P. H. Lin. 2007. “Dynamic mobile RFID-based supply chain control and management system in construction.” Adv. Eng. Inf. 21 (4): 377–390. https://doi.org/10.1016/j.aei.2006.09.003.
Wu, Y., X. Zhou, and Y. Zhang. 2015. “Steel bars counting and splitting method based on machine vision.” In Proc., 2015 IEEE Int. Conf. on Cyber Technology in Automation, Control, and Intelligent Systems, 420–425. New York: IEEE.
Xiaohu, L., and O. Jineng. 2018. “Research on steel bar detection and counting method based on contours.” In Proc., 2018 Int. Conf. on Electronics Technology, 294–297. New York: IEEE.
Xu, Y., Y. Bao, J. Chen, W. Zuo, and H. Li. 2019a. “Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images.” Struct. Health Monit. 18 (3): 653–674. https://doi.org/10.1177/1475921718764873.
Xu, Y., S. Wei, Y. Bao, and H. Li. 2019b. “Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network.” Struct. Control. Health Monit. 26 (3): e2313. https://doi.org/10.1002/stc.2313.
Ying, X., X. Wei, Y. Pei-xin, H. Qing-da, and C. Chang-hai. 2010. “Research on an automatic counting method for steel bars’ image.” In Proc., 2010 Int. Conf. on Electrical and Control Engineering, 1644–1647. New York: IEEE.
Zhang, C., C. Chang, and M. Jamshidi. 2020a. “Concrete bridge surface damage detection using a single-stage detector.” Comput.-Aided Civ. Infrastruct. Eng. 35 (4): 389–409. https://doi.org/10.1111/mice.12500.
Zhang, D., Z. Xie, and C. Wang. 2008. “Bar section image enhancement and positioning method in on-line steel bar counting and automatic separating system.” In Proc., 1st Int. Congress on Image and Signal Processing, 319–323. New York: IEEE.
Zhang, H., Y. Wang, F. Dayoub, and N. Sünderhauf. 2020b. “VarifocalNet: An IoU-aware dense object detector.” Preprint, submitted August 31, 2020. https://arxiv.org/abs/2008.13367.
Zhang, S., C. Chi, Y. Yao, Z. Lei, and S. Z. Li. 2020c. “Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 9759–9768. New York: IEEE.
Zheng, Z., P. Wang, W. Liu, J. Li, R. Ye, and D. Ren. 2020. “Distance-IoU loss: Faster and better learning for bounding box regression.” In Proc., AAAI Conf. on Artificial Intelligence, 12993–13000. Palo Alto, CA: AAAI Press.
Zhu, Y., C. Tang, H. Liu, and P. Huang. 2020. “End-face localization and segmentation of steel bar based on convolution neural network.” IEEE Access 8 (Apr): 74679–74690. https://doi.org/10.1109/ACCESS.2020.2989300.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 148 • Issue 1 • January 2022
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Mar 30, 2021
Accepted: Sep 17, 2021
Published online: Oct 22, 2021
Published in print: Jan 1, 2022
Discussion open until: Mar 22, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Dena Shamsollahi, Osama Moselhi, Khashayar Khorasani, Automated Detection and Segmentation of Mechanical, Electrical, and Plumbing Components in Indoor Environments by Using the YOLACT++ Architecture, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-15115, 150, 8, (2024).
- Seunghyeon Wang, Mincheol Kim, Hyeonyong Hae, Mengqiu Cao, Juhyung Kim, The Development of a Rebar-Counting Model for Reinforced Concrete Columns: Using an Unmanned Aerial Vehicle and Deep-Learning Approach, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13686, 149, 11, (2023).
- Yoon-Soo Shin, Junhee Kim, Automated Dimension Estimation of Steel Pipes Stacked at Construction Sites Using Euclidean Distances Calculated by Overlapping Segmentation, Sensors, 10.3390/s22124517, 22, 12, (4517), (2022).
- Naveed Ilyas, Zaheer Ahmad, Boreom Lee, Kiseon Kim, An effective modular approach for crowd counting in an image using convolutional neural networks, Scientific Reports, 10.1038/s41598-022-09685-w, 12, 1, (2022).
- Mayank Mishra, Tanmoy Barman, G. V. Ramana, Artificial intelligence-based visual inspection system for structural health monitoring of cultural heritage, Journal of Civil Structural Health Monitoring, 10.1007/s13349-022-00643-8, (2022).