Automatic Stereo Vision-Based Inspection System for Particle Shape Analysis of Coarse Aggregates
Publication: Journal of Computing in Civil Engineering
Volume 36, Issue 2
Abstract
Particle shape analysis of coarse aggregates is important to ensure the quality of cement and asphalt concrete mixtures. Conventional methods for measuring the aggregate particle size, such as manual calipers or mechanical sieving, are time consuming and labor intensive. In addition, the accuracy of image processing techniques is severely limited by shadows and heterogeneous backgrounds. Hence, we developed an automatic stereo vision-based inspection system (SVIS) for the identification and shape analysis of coarse aggregate particles. We integrated a cascaded deep learning model into the SVIS to identify the types of coarse aggregate particles under offsite working conditions. Moreover, we combined deep learning and stereo vision techniques to calculate the unit conversion factors and the thickness of each particle to facilitate particle shape analysis. The precision and recall metrics obtained from the training model were for particle detection and for particle segmentation. In the experiment, the proposed inspection system accurately determined the particle size of coarse aggregates with measurement errors of compared with the ground truth. Thus, the proposed system overcomes the shortcomings of image processing technologies and considerably aids the decision-making process during onsite material inspection.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or codes that support the findings of this study are provided by the corresponding author upon reasonable request.
Acknowledgments
This work was supported by the Architecture & Urban Development Research Program (Grant No. 21 AUDP-B127891-05) and the Innovative Talent Education Program for Smart City funded by the Ministry of Land, Infrastructure, and Transport of the Korean Government.
References
Al-Rousan, T., E. Masad, E. Tutumluer, and T. Pan. 2007. “Evaluation of image analysis techniques for quantifying aggregate shape characteristics.” Constr. Build. Mater. 21 (5): 978–990. https://doi.org/10.1016/j.conbuildmat.2006.03.005.
Anochie-Boateng, J. K., J. J. Komba, and G. M. Mvelase. 2013. “Three-dimensional laser scanning technique to quantify aggregate and ballast shape properties.” Constr. Build. Mater. 43 (Jun): 389–398. https://doi.org/10.1016/j.conbuildmat.2013.02.062.
ASTM. 2019. “Standard test method for flat particles, elongated particles, or flat and elongated particles in coarse aggregate.” Accessed June 17, 2021. https://compass.astm.org/EDIT/html_annot.cgi?D4791+19.
Beshr, H., A. A. Almusallam, and M. Maslehuddin. 2003. “Effect of coarse aggregate quality on the mechanical properties of high strength concrete.” Constr. Build. Mater. 17 (2): 97–103. https://doi.org/10.1016/S0950-0618(02)00097-1.
Brahmbhatt, S. 2013. Practical openCV. New York: Apress.
Brzezicki, J. M., and J. Kasperkiewicz. 1999. “Automatic image analysis in evaluation of aggregate shape.” J. Comput. Civ. Eng. 13 (2): 123–128. https://doi.org/10.1061/(ASCE)0887-3801(1999)13:2(123).
Cai, Z., and N. Vasconcelos. 2018. “Cascade r-CNN: Delving into high quality object detection.” In Proc., 2018 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 6154–6162. New York: IEEE.
Chen, K., J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, and Z. Zhang. 2019. “Mmdetection: Open mmlab detection toolbox and benchmark.” Accessed June 20, 2021. https://arxiv.org/abs/1906.07155.
Dung, C. V., and L. D. Anh. 2019. “Autonomous concrete crack detection using deep fully convolutional neural network.” Autom. Constr. 99 (Mar): 52–58. https://doi.org/10.1016/j.autcon.2018.11.028.
Erdogan, S. T., P. N. Quiroga, D. W. Fowler, H. A. Saleh, R. A. Livingston, E. J. Garboczi, and S. G. Satterfield. 2006. “Three-dimensional shape analysis of coarse aggregates: New techniques for and preliminary results on several different coarse aggregates and reference rocks.” Cem. Concr. Res. 36 (9): 1619–1627. https://doi.org/10.1016/j.cemconres.2006.04.003.
Fernlund, J. M. R. 2005. “Image analysis method for determining 3-D shape of coarse aggregate.” Cem. Concr. Res. 35 (8): 1629–1637. https://doi.org/10.1016/j.cemconres.2004.11.017.
Han, J., K. Wang, X. Wang, and P. J. M. Monteiro. 2016. “2D image analysis method for evaluating coarse aggregate characteristic and distribution in concrete.” Constr. Build. Mater. 127 (Nov): 30–42. https://doi.org/10.1016/j.conbuildmat.2016.09.120.
Hartley, R., and A. Zisserman. 2004. Multiple view geometry in computer vision. Cambridge, UK: Cambridge University Press.
He, K., G. Gkioxari, P. Dollár, and R. B. Girshick. 2017. “Mask R-CNN.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2961–2969. New York: IEEE.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image recognition.” In Proc., 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 770–778. New York: IEEE.
Hentschel, M. L., and N. W. Page. 2003. “Selection of descriptors for particle shape characterization.” Part. Part. Syst. Char. 20 (1): 25–38. https://doi.org/10.1002/ppsc.200390002.
Higashitani, K., H. Makino, and S. Matsusaka. 2019. Powder technology handbook. Boca Raton, FL: CRC Press.
Hirschmuller, H. 2008. “Stereo processing by semiglobal matching and mutual information.” IEEE Trans. Pattern Anal. Mach. Intell. 30 (2): 328–341. https://doi.org/10.1109/TPAMI.2007.1166.
Hoong, J. D. L. H., J. Lux, P. Y. Mahieux, P. Turcry, and A. Aït-Mokhtar. 2020. “Determination of the composition of recycled aggregates using a deep learning-based image analysis.” Autom. Constr. 116 (Aug): 103204. https://doi.org/10.1016/j.autcon.2020.103204.
Huang, H., J. Luo, I. Qamhia, E. Tutumluer, J. M. Hart, and A. J. Stolba. 2021. “I-RIPRAP computer vision software for automated size and shape characterization of riprap in stockpile images.” Transp. Res. Rec. 2021 (Jan): 03611981211001375. https://doi.org/10.1177/03611981211001375.
Huang, H., J. Luo, E. Tutumluer, J. M. Hart, and I. Qamhia. 2020. “Size and shape determination of riprap and large-sized aggregates using field imaging.” Transp. Res. Rec. 2674 (10): 285–298. https://doi.org/10.1177/0361198120943887.
Intel Corporation. 2020. “The OpenCV tutorials.” Accessed June 12, 2021. https://docs.opencv.org/4.5.0/.
Kim, H., H. Kim, Y. W. Hong, and H. Byun. 2018. “Detecting construction equipment using a region-based fully convolutional network and transfer learning.” J. Comput. Civ. Eng. 32 (2): 04017082. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000731.
Kwan, A. K., C. F. Mora, and H. C. Chan. 1999. “Particle shape analysis of coarse aggregate using digital image processing.” Cem. Concr. Res. 29 (9): 1403–1410. https://doi.org/10.1016/S0008-8846(99)00105-2.
LeCun, Y., Y. Bengio, and G. Hinton. 2015. “Deep learning.” Nature 521 (7553): 436–444. https://doi.org/10.1038/nature14539.
Lee, S. J., C. H. Lee, M. Shin, S. Bhattacharya, and Y. F. Su. 2019. “Influence of coarse aggregate angularity on the mechanical performance of cement-based materials.” Constr. Build. Mater. 204 (2019): 184–192. https://doi.org/10.1016/j.conbuildmat.2019.01.135.
Lee, Y. J., and M. Park. 2019. “3D tracking of multiple onsite workers based on stereo vision.” Autom. Constr. 98 (Feb): 146–159. https://doi.org/10.1016/j.autcon.2018.11.017.
Li, C. H., and P. K. S. Tam. 1998. “An iterative algorithm for minimum cross entropy thresholding.” Pattern Recognit. Lett. 19 (8): 771–776. https://doi.org/10.1016/S0167-8655(98)00057-9.
Li, D., Q. Xie, X. Gong, Z. Yu, J. Xu, Y. Sun, and J. Wang. 2021. “Automatic defect detection of metro tunnel surfaces using a vision-based inspection system.” Adv. Eng. Inform. 47 (Jan): 101206. https://doi.org/10.1016/j.aei.2020.101206.
Lin, T. Y., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. 2014. “Microsoft coco: Common objects in context.” In Proc., European Conf. on Computer Vision, 740–755. Berlin: Springer.
Mehta, P. K., and P. J. Monteiro. 2013. Concrete: Microstructure, properties and materials. New York: McGraw-Hill Education.
Mora, C. F., and A. K. H. Kwan. 2000. “Sphericity, shape factor, and convexity measurement of coarse aggregate for concrete using digital image processing.” Cem. Concr. Res. 30 (3): 351–358. https://doi.org/10.1016/S0008-8846(99)00259-8.
Mora, C. F., A. K. H. Kwan, and H. C. Chan. 1998. “Particle size distribution analysis of coarse aggregate using digital image processing.” Cem. Concr. Res. 28 (6): 921–932. https://doi.org/10.1016/S0008-8846(98)00043-X.
Otsu, N. 1979. “A threshold selection method from gray-level histograms.” IEEE Trans. Syst. Man Cybern. 9 (1): 62–66. https://doi.org/10.1109/TSMC.1979.4310076.
Pan, Y., G. Zhang, and L. Zhang. 2020. “A spatial-channel hierarchical deep learning network for pixel-level automated crack detection.” Autom. Constr. 119 (Nov): 103357. https://doi.org/10.1016/j.autcon.2020.103357.
Ruan, X., Y. Li, Z. Jin, Z. Pan, and Z. Yin. 2019. “Modeling method of concrete material at mesoscale with refined aggregate shapes based on image recognition.” Constr. Build. Mater. 204 (Apr): 562–575. https://doi.org/10.1016/j.conbuildmat.2019.01.157.
Russ, J. C. 1998. The image processing handbook. Boca Raton, FL: CRC Press.
Russell, B. C., A. Torralba, K. P. Murphy, and W. T. Freeman. 2008. “LabelMe: A database and web-based tool for image annotation.” Int. J. Comput. Vision 77 (1–3): 157–173. https://doi.org/10.1007/s11263-007-0090-8.
Soltani, M. M., Z. Zhu, and A. Hammad. 2018. “Framework for location data fusion and pose estimation of excavators using stereo vision.” J. Comput. Civ. Eng. 32 (6): 04018045. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000783.
Sung, C., and P. Y. Kim. 2016. “3D terrain reconstruction of construction sites using a stereo camera.” Autom. Constr. 64 (Apr): 65–77. https://doi.org/10.1016/j.autcon.2015.12.022.
Tuan, N. M., Q. V. Hau, S. Chin, and S. Park. 2021. “In-situ concrete slump test incorporating deep learning and stereo vision.” Autom. Constr. 121 (Jan): 103432. https://doi.org/10.1016/j.autcon.2020.103432.
Wu, J., L. Wang, Y. Hou, H. Xiong, Y. Lu, and L. Zhang. 2018. “A digital image analysis of gravel aggregate using CT scanning technique.” Int. J. Pavement Res. Technol. 11 (2): 160–167. https://doi.org/10.1016/j.ijprt.2017.08.002.
Xie, S., R. Girshick, P. Dollár, Z. Tu, and K. He. 2017. “Aggregated residual transformations for deep neural networks.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 1492–1500. New York: IEEE.
Yan, X., H. Zhang, and H. Li. 2019. “Estimating worker-centric 3D spatial crowdedness for construction safety management using a single 2D camera.” J. Comput. Civ. Eng. 33 (5): 04019030. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000844.
Yang, J., and S. Chen. 2017. “An online detection system for aggregate sizes and shapes based on digital image processing.” Mineral. Petrol. 111 (1): 135–144. https://doi.org/10.1007/s00710-016-0458-y.
Zheng, J., and R. D. Hryciw. 2014. “Soil particle size characterization by stereophotography.” In Geo-Congress 2014: Geo-Characterization and Modeling for Sustainability, Geotechnical Special Publication 234, edited by M. Abu-Farsakh, X. Yu, and L. R. Hoyos, 64–73. Reston, VA: ASCE.
Zheng, J., Q. Sun, H. Zheng, D. Wei, Z. Li, and L. Gao. 2020. “Three-dimensional particle shape characterizations from half particle geometries.” Powder Technol. 367 (May): 122–132. https://doi.org/10.1016/j.powtec.2020.03.046.
Zhu, H., H. Fang, Y. Cai, and N. Lv. 2020. “Development of a rapid measurement system for coarse aggregate morphological parameters.” Particuology 50 (Jun): 181–188. https://doi.org/10.1016/j.partic.2019.06.006.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 36 • Issue 2 • March 2022
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Jul 19, 2021
Accepted: Oct 7, 2021
Published online: Nov 23, 2021
Published in print: Mar 1, 2022
Discussion open until: Apr 23, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Jui-Sheng Chou, Chi-Yun Liu, Optimized Lightweight Edge Computing Platform for UAV-Assisted Detection of Concrete Deterioration beneath Bridge Decks, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5905, 39, 1, (2025).
- Hui Guo, Jiahao Wang, Sihan Xu, Jingting Wu, Research on the intelligent detection technology of multi-objective coarse aggregates, Construction and Building Materials, 10.1016/j.conbuildmat.2022.129273, 363, (129273), (2023).
- Chi-Yun Liu, Jui-Sheng Chou, Bayesian-optimized deep learning model to segment deterioration patterns underneath bridge decks photographed by unmanned aerial vehicle, Automation in Construction, 10.1016/j.autcon.2022.104666, 146, (104666), (2023).
- Haoyue Fan, Zhenghong Tian, Xiaobin Xu, Xiao Sun, Yuanshan Ma, Hengrui Liu, Hao Lu, Rockfill material segmentation and gradation calculation based on deep learning, Case Studies in Construction Materials, 10.1016/j.cscm.2022.e01216, 17, (e01216), (2022).