Image-Based Modeling-to-Simulation of Masonry Walls
Publication: Journal of Architectural Engineering
Volume 28, Issue 4
Abstract
This paper presents a streamlined image-based modeling-to-simulation framework to better assess the hazard vulnerability of masonry structures. In this framework, a masonry wall image is used as an input for 3D discrete element modeling and analysis. Therefore, individual bricks are directly detected from the image and the interactions of the bricks are explicitly considered in the numerical simulation of the masonry wall. The bricks in the masonry wall image are segmented first, from which the geometric information of the bricks is obtained using a bespoke algorithm. Segmented bricks are then approximated into n-sided polygons using the splitting and Douglas–Peucker methods to develop simplified brick geometries. This shape simplification is performed to lower the computational cost for the discrete element analysis while reasonably approximating the overall brick geometry and maintaining the simulation fidelity. The simplified polygons are exported to a free and open-source physics engine as scalable vector graphics, from which a 3D masonry model is developed, and rigid body discrete element simulation is performed using the impulse-based dynamics. This paper demonstrates the image-based modeling-to-simulation framework can estimate collapse scenarios from an input masonry wall image and proposes a prototype that transforms visual images into critical domain knowledge that would be useful for disaster preparedness. With enhanced predictive capabilities, this framework can contribute to innovations in the hazard vulnerability assessment of masonry structures.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
This work is sponsored in part by the National Center for Preservation Technology and Training of the US National Park Service under award P19AP00141 and the US National Science Foundation under award 1635378. The opinions, findings, conclusions, or recommendations expressed in this article are solely those of the authors and do not represent the opinions of the funding agencies.
References
Alecci, V., M. Fagone, T. Rotunno, and M. De Stefano. 2013. “Shear strength of brick masonry walls assembled with different types of mortar.” Constr. Build. Mater. 40: 1038–1045. https://doi.org/10.1016/j.conbuildmat.2012.11.107.
Almeida, C., J. P. Guedes, A. Arêde, and A. Costa. 2016. “Geometric indices to quantify textures irregularity of stone masonry walls.” Constr. Build. Mater. 111: 199–208. https://doi.org/10.1016/j.conbuildmat.2016.02.038.
Augarde, C. E., S. J. Lee, and D. Loukidis. 2021. “Numerical modelling of large deformation problems in geotechnical engineering: A state-of-the-art review.” Soils Found. 61 (6): 1718–1735. https://doi.org/10.1016/j.sandf.2021.08.007.
Batar, O. S., E. Tercan, and E. Emsen. 2021. “Ayvalıkemer (Sillyon) historical masonry arch bridge: A multidisciplinary approach for structural assessment using point cloud data obtained by terrestrial laser scanning (TLS).” J. Civ. Struct. Health Monit. 11 (5): 1239–1252. https://doi.org/10.1007/s13349-021-00507-7.
Belemmi, G. 2012. “Construction materials: Stone and ceramics.” Accessed September 18, 2020. https://www.edu.xunta.gal/centros/espazoAbalar/aulavirtual/pluginfile.php/1319/mod_imscp/content/1/construction_materials_stone_and_ceramics.html.
Bender, J., K. Erleben, and J. Trinkle. 2014. “Interactive simulation of rigid body dynamics in computer graphics.” Comput. Graphics Forum 33 (1): 246–270. https://doi.org/10.1111/cgf.12272.
Biscarini, C., I. Catapano, N. Cavalagli, G. Ludeno, F. A. Pepe, and F. Ubertini. 2020. “UAV photogrammetry, infrared thermography and GPR for enhancing structural and material degradation evaluation of the Roman masonry bridge of Ponte Lucano in Italy.” NDT & E Int. 115: 102287. https://doi.org/10.1016/j.ndteint.2020.102287.
Błaszczak-Bąk, W., C. Suchocki, J. Janicka, A. Dumalski, R. Duchnowski, and A. Sobieraj-Żłobińska. 2020. “Automatic threat detection for historic buildings in dark places based on the modified OPTD method.” ISPRS Int. J. Geo-Inf. 9 (2): 123. https://doi.org/10.3390/ijgi9020123.
Bone, P. 2021. “Polygon simplification.” MATLAB Central File Exchange. Accessed March 18, 2021. https://www.mathworks.com/matlabcentral/fileexchange/45342-polygon-simplification.
Bowler, J., C. Brown, M. Capsimalis, R. Cohn, and L. D. Cole. 2001. “Scalable vector graphics (svg) 1.0 specification.” World Wide Web Consortium. Accessed December 2, 2021. https://www.w3.org/TR/2001/REC-SVG-20010904/.
Bui, T. T., A. Limam, V. Sarhosis, and M. Hjiaj. 2017. “Discrete element modelling of the in-plane and out-of-plane behaviour of dry-joint masonry wall constructions.” Eng. Struct. 136: 277–294. https://doi.org/10.1016/j.engstruct.2017.01.020.
Castellazzi, G., A. D’Altri, G. Bitelli, I. Selvaggi, and A. Lambertini. 2015. “From laser scanning to finite element analysis of complex buildings by using a semi-automatic procedure.” Sensors 15 (8): 18360–18380. https://doi.org/10.3390/s150818360.
Cavalagli, N., F. Cluni, and V. Gusella. 2013. “Evaluation of a statistically equivalent periodic unit cell for a quasi-periodic masonry.” Int. J. Solids Struct. 50 (25–26): 4226–4240. https://doi.org/10.1016/j.ijsolstr.2013.08.027.
Cavalagli, N., M. Gioffrè, S. Grassi, V. Gusella, C. Pepi, and G. M. Volpi. 2020. “On the accuracy of UAV photogrammetric survey for the evaluation of historic masonry structural damages.” Procedia Struct. Integr. 29: 165–174. https://doi.org/10.1016/j.prostr.2020.11.153.
Chaiyasarn, K., W. Khan, L. Ali, M. Sharma, D. Brackenbury, and M. DeJong. 2018. “Crack detection in masonry structures using convolutional neural networks and support vector machines.” In Proc., 35th Int. Symp. on Automation and Robotics in Construction, 118–125. London: International Association for Automation and Robotics in Construction.
Chaiyasarn, K., S. Mahat, and S. Prukpratin. 2021. “The application of 3D image-based photogrammetry in Thai heritage sites: A case study of Wat Chai Wattanaram.” In Proc., Int. Conf. on Protection of Historical Constructions, 477–488. Berlin, Germany: Springer.
Chiang, K.-Y., K.-L. Chien, and S.-J. Hwang. 2008. “Study on the characteristics of building bricks produced from reservoir sediment.” J. Hazard. Mater. 159 (2–3): 499–504. https://doi.org/10.1016/j.jhazmat.2008.02.046.
Clayton, P., G. Zalachoris, E. Rathje, T. Bheemasetti, S. Caballero, X. Yu, and S. Bennett. 2016. The geotechnical aspects of the September 3, 2016 M5.8 Pawnee, Oklahoma Earthquake. Berkeley, CA: Geotechnical Extreme Events Reconnaissance Association.
Coumans, E. 2020. “Bullet physics.” Accessed May 23, 2021. http://bulletphysics.org.
Cundall, P. A., and R. D. Hart. 1992. “Numerical modelling of discontinua.” Eng. Comput. 9 (2): 101–113. https://doi.org/10.1108/eb023851.
Cundall, P. A., and O. D. L. Strack. 1979. “A discrete numerical model for granular assemblies.” Géotechnique 29 (1): 47–65. https://doi.org/10.1680/geot.1979.29.1.47.
D’Altri, A. M., G. Milani, S. de Miranda, G. Castellazzi, and V. Sarhosis. 2018. “Stability analysis of leaning historic masonry structures.” Autom. Constr. 92: 199–213. https://doi.org/10.1016/j.autcon.2018.04.003.
Dais, D., İ. E. Bal, E. Smyrou, and V. Sarhosis. 2021. “Automatic crack classification and segmentation on masonry surfaces using convolutional neural networks and transfer learning.” Autom. Constr. 125: 103606. https://doi.org/10.1016/j.autcon.2021.103606.
DeJong, M. J., and C. Vibert. 2012. “Seismic response of stone masonry spires: Computational and experimental modeling.” Eng. Struct. 40: 566–574. https://doi.org/10.1016/j.engstruct.2012.03.001.
Dell’Endice, A., A. Iannuzzo, M. J. DeJong, T. Van Mele, and P. Block. 2021. “Modelling imperfections in unreinforced masonry structures: Discrete element simulations and scale model experiments of a pavilion vault.” Eng. Struct. 228: 111499. https://doi.org/10.1016/j.engstruct.2020.111499.
Dizhur, D., and J. M. Ingham. 2015. Seismic improvement of loadbearing unreinforced masonry cavity walls. Auckland, New Zealand: BRANZ.
Dorji, J., T. Zahra, D. Thambiratnam, and D. Lee. 2021. “Strength assessment of old masonry arch bridges through moderate destructive testing methods.” Constr. Build. Mater. 278: 122391. https://doi.org/10.1016/j.conbuildmat.2021.122391.
Douglas, D. H., and T. K. Peucker. 1973. “Algorithms for the reduction of the number of points required to represent a digitized line or its caricature.” Cartographica: Int. J. Geog. Inf. Geovisualization 10 (2): 112–122. https://doi.org/10.3138/FM57-6770-U75U-7727.
Dunphy, K., and A. Sadhu. 2022. “Autonomous crack detection approach for masonry structures using artificial intelligence.” In Recent developments in structural health monitoring and assessment—Opportunities and challenges: Bridges, buildings and other infrastructures, edited by A. Haldar, 253–283. Singapore: World Scientific.
Ellenberg, A., A. Kontsos, F. Moon, and I. Bartoli. 2016. “Bridge related damage quantification using unmanned aerial vehicle imagery.” Struct. Control Health Monit. 23 (9): 1168–1179. https://doi.org/10.1002/stc.1831.
Erleben, K., J. Sporring, K. Henriksen, and H. Dohlmann. 2005. Physics-based animation. Drive Rockland, MA: Charles River Media.
Falsone, G., and M. Lombardo. 2007. “Stochastic representation of the mechanical properties of irregular masonry structures.” Int. J. Solids Struct. 44 (25–26): 8600–8612. https://doi.org/10.1016/j.ijsolstr.2007.06.030.
Ghaboussi, J. 1988. “Fully deformable discrete element analysis using a finite element approach.” Comput. Geotech. 5 (3): 175–195. https://doi.org/10.1016/0266-352X(88)90001-8.
Ghiassi, B., A. T. Vermelfoort, and P. B. Lourenço. 2019. “Masonry mechanical properties.” In Numerical modeling of masonry and historical structures, edited by B. Ghiassi and G. Milani, 239–261. Amsterdam, Netherlands: Elsevier.
Gonen, S., B. Pulatsu, S. Soyoz, and E. Erdogmus. 2021. “Stochastic discontinuum analysis of unreinforced masonry walls: Lateral capacity and performance assessments.” Eng. Struct. 238: 112175. https://doi.org/10.1016/j.engstruct.2021.112175.
Ham, Y., S. J. Lee, and A. G. Chowdhury. 2017. “Imaging-to-simulation framework for improving disaster preparedness of construction projects and neighboring communities.” In Computing in Civil Engineering 2017, 230–237. Reston, VA: ASCE.
Ioannides, M., D. Fritsch, J. Leissner, R. Davies, F. Remondino, and R. Caffo. 2012. “Progress in cultural heritage preservation.” In Proc., 4th Int. Conf., EuroMed 2012. Berlin, Germany: Springer.
Kassotakis, N., V. Sarhosis, M. V. Peppa, and J. Mills. 2021. “Quantifying the effect of geometric uncertainty on the structural behaviour of arches developed from direct measurement and structure-from-motion (SfM) photogrammetry.” Eng. Struct. 230: 111710. https://doi.org/10.1016/j.engstruct.2020.111710.
Kostack, K. 2021. “Bullet constraints builder for blender.” Accessed August 26, 2021. https://github.com/KaiKostack/bullet-constraints-builder.
Koutsoudis, A., B. Vidmar, G. Ioannakis, F. Arnaoutoglou, G. Pavlidis, and C. Chamzas. 2014. “Multi-image 3D reconstruction data evaluation.” J. Cult. Heritage 15 (1): 73–79. https://doi.org/10.1016/j.culher.2012.12.003.
Lee, J., and H. Kang. 2010. “Flood fill mean shift: A robust segmentation algorithm.” Int. J. Control Autom. Syst. 8 (6): 1313–1319. https://doi.org/10.1007/s12555-010-0617-6.
Lee, S. J. 2014. Developments in large scale discrete element simulations with polyhedral particles. Urbana, IL: Univ. of Illinois at Urbana-Champaign.
Lee, S. J., and Y. M. A. Hashash. 2015. “iDEM: An impulse-based discrete element method for fast granular dynamics.” Int. J. Numer. Methods Eng. 104 (2): 79–103. https://doi.org/10.1002/nme.4923.
Lei, T., P. Liu, X. Jia, X. Zhang, H. Meng, and A. K. Nandi. 2020. “Automatic fuzzy clustering framework for image segmentation.” IEEE Trans. Fuzzy Syst. 28 (9): 2078–2092. https://doi.org/10.1109/TFUZZ.2019.2930030.
Lemos, J. V., and A. Campos Costa. 2017. “Simulation of shake table tests on out-of-plane masonry buildings. V: Discrete element approach.” Int. J. Archit. Heritage 11 (1): 117–124.
Lindenbergh, R., and P. Pietrzyk. 2015. “Change detection and deformation analysis using static and mobile laser scanning.” Appl. Geomatics 7 (2): 65–74. https://doi.org/10.1007/s12518-014-0151-y.
Liu, Z., R. Brigham, E. R. Long, L. Wilson, A. Frost, S. A. Orr, and J. Grau-Bové. 2022. “Semantic segmentation and photogrammetry of crowdsourced images to monitor historic facades.” Heritage Sci. 10 (1): 1–17.
Loverdos, D., and V. Sarhosis. 2022. “Automatic image-based brick segmentation and crack detection of masonry walls using machine learning.” Autom. Constr. 140: 104389. https://doi.org/10.1016/j.autcon.2022.104389.
Malomo, D., M. J. DeJong, and A. Penna. 2019. “Distinct element modelling of the in-plane cyclic response of URM walls subjected to shear-compression.” Earthquake Eng. Struct. Dyn. 48 (12): 1322–1344. https://doi.org/10.1002/eqe.3178.
Masi, F., I. Stefanou, V. Maffi-Berthier, and P. Vannucci. 2020. “A discrete element method based-approach for arched masonry structures under blast loads.” Eng. Struct. 216: 110721. https://doi.org/10.1016/j.engstruct.2020.110721.
MathWorks. 2021a. “MATLAB (Ver. R2021a).” MATLAB: Natick, MA.
MathWorks. 2021b. “Image Segmenter App.” MATLAB (Ver. R2021a). Image processing toolbox. Accessed May 23, 2021. https://www.mathworks.com/help/images/image-segmentation-using-the-image-segmenter-app.html.
MathWorks. 2021c. “Fill image regions and holes—MATLAB imfill.” MATLAB (Ver. R2021a). Image Processing Toolbox. Accessed May 23, 2021. https://www.mathworks.com/help/images/ref/imfill.html.
MathWorks. 2021d. “Trace region boundaries in binary image—MATLAB bwboundaries.” MATLAB (Ver. R2021a). Image Processing Toolbox. Accessed May 23, 2021. https://www.mathworks.com/help/images/ref/bwboundaries.html.
Micelli, F., and A. Cascardi. 2020. “Structural assessment and seismic analysis of a 14th century masonry tower.” Eng. Fail. Anal. 107: 104198. https://doi.org/10.1016/j.engfailanal.2019.104198.
National Park Service. 2014. Keeping watch on surging seas. Experience your America. Key West, FL: National Park Service.
Nayak, S., and S. C. Dutta. 2016. “Failure of masonry structures in earthquake: A few simple cost effective techniques as possible solutions.” Eng. Struct. 106: 53–67. https://doi.org/10.1016/j.engstruct.2015.10.014.
Nghiem, H. L., M. Al Heib, and F. Emeriault. 2015. “Method based on digital image correlation for damage assessment in masonry structures.” Eng. Struct. 86: 1–15. https://doi.org/10.1016/j.engstruct.2014.12.021.
Novikov, A. 2014. “Stone wall with random tiled pattern.” Depositphotos. Accessed August 25, 2022. https://depositphotos.com/56832577/stock-photo-stone-wall-with-random-tiled.html.
Nowak, R., T. Kania, R. Rutkowski, and E. Ekiert. 2022. “Research and TLS (LiDAR) construction diagnostics of clay brick masonry arched stairs.” Materials 15 (2): 552. https://doi.org/10.3390/ma15020552.
Nowak, R., R. Orłowicz, and R. Rutkowski. 2020. “Use of TLS (LiDAR) for building diagnostics with the example of a historic building in Karlino.” Buildings 10 (2): 24. https://doi.org/10.3390/buildings10020024.
Paine, D. J. 2018. “Brown concrete wall photo—Free brick image on unsplash.” Unsplash. Accessed May 23, 2021. https://unsplash.com/photos/DjwwbloDkNg.
Park, H.-J., J.-G. Ha, S.-H. Kim, and S.-S. Jo. 2019. “Seismic performance of ancient masonry structures in Korea rediscovered in 2016 M 5.8 Gyeongju earthquake.” Sustainability 11 (6): 1565. https://doi.org/10.3390/su11061565.
Peng, B., L. Zhang, and J. Yang. 2009. “Iterated graph cuts for image segmentation.” In Asian Conf. on Computer Vision, 677–686. Berlin, Heidelberg: Springer.
Pepe, M., and D. Costantino. 2020. “Techniques, tools, platforms and algorithms in close range photogrammetry in building 3D model and 2D representation of objects and complex architectures.” Comput.-Aided Des. Applic. 18: 42–65. https://doi.org/10.14733/cadaps.2021.42-65.
Pepe, M., and D. Costantino. 2021. “UAV photogrammetry and 3D modelling of complex architecture for maintenance purposes: The case study of the masonry bridge on the sele river, Italy.” Period. Polytech., Civ. Eng. 65 (1): 191–203.
Pikaz, A., and I. Dinstein. 1995. “An algorithm for polygonal approximation based on iterative point elimination.” Pattern Recognit. Lett. 16 (6): 557–563. https://doi.org/10.1016/0167-8655(95)80001-A.
Pulatsu, B., E. Erdogmus, and E. M. Bretas. 2018. “Parametric study on masonry arches using 2D discrete-element modeling.” J. Archit. Eng. 24 (2): 04018005. https://doi.org/10.1061/(ASCE)AE.1943-5568.0000305.
Pushkar, A., M. Senthilvel, and K. Varghese. 2018. “Automated progress monitoring of masonry activity using photogrammetric point cloud.” In Proc., 35th Int. Symp. on Automation and Robotics in Construction. London: International Association for Automation and Robotics in Construction.
Riveiro, B., P. B. Lourenço, D. V. Oliveira, H. González-Jorge, and P. Arias. 2016. “Automatic morphologic analysis of quasi-periodic masonry walls from LiDAR.” Comput.-Aided Civ. Infrastruct. Eng. 31 (4): 305–319. https://doi.org/10.1111/mice.12145.
Santini, S., C. Baggio, V. Sabbatini, and C. Sebastiani. 2022. “Seismic assessment of roman concrete groin vaults through UAV, NDT and 3D analyses.” Heritage 5 (1): 311–331. https://doi.org/10.3390/heritage5010017.
Sauer, J., and E. Schömer. 1998. “A constraint-based approach to rigid body dynamics for virtual reality applications.” In Proc., of the ACM Symp. on Virtual Reality Software and Technology, 153–162. New York: Assoction for Computing Machinery.
Schwanghart, W. 2021. “Line simplification.” MATLAB Central File Exchange. Accessed March 25, 2021. https://www.mathworks.com/matlabcentral/fileexchange/21132-line-simplification.
Sejnoha, J., M. Sejnoha, J. Zeman, J. Sykora, and J. Vorel. 2008. “Mesoscopic study on historic masonry.” Preprint, submitted April 21, 2008. http://arxiv.org/abs/0804.3262v1.
Shaun. 2020. “Wall bricks rustic” Pixabay. Accessed September 11, 2020. https://pixabay.com/photos/wall-brick-rustic-masonry-5508929/.
Stavroulaki, M. E., B. Riveiro, G. A. Drosopoulos, M. Solla, P. Koutsianitis, and G. E. Stavroulakis. 2016. “Modelling and strength evaluation of masonry bridges using terrestrial photogrammetry and finite elements.” Adv. Eng. Software 101: 136–148. https://doi.org/10.1016/j.advengsoft.2015.12.007.
Suchocki, C., J. Katzer, and J. Rapiński. 2018. “Terrestrial laser scanner as a tool for assessment of saturation and moisture movement in building materials.” Period. Polytech., Civ. Eng. 62 (3): 694–699.
The Blender Foundation. 2021. “Blender - A 3D modelling and rendering package.” Accessed May 23, 2021. http://www.blender.org.
Tóth, A. R., Z. Orbán, and K. Bagi. 2009. “Discrete element analysis of a stone masonry arch.” Mech. Res. Commun. 36 (4): 469–480. https://doi.org/10.1016/j.mechrescom.2009.01.001.
Truong-Hong, L., D. F. Laefer, T. Hinks, and H. Carr. 2013. “Combining an angle criterion with voxelization and the flying voxel method in reconstructing building models from LiDAR data.” Comput.-Aided Civ. Infrastruct. Eng. 28 (2): 112–129. https://doi.org/10.1111/j.1467-8667.2012.00761.x.
Tumialan, J. G., F. Micelli, and A. Nanni. 2001. “Strengthening of masonry structures with FRP composites.” In Structures 2001: A Structural Engineering Odyssey, edited by P. C. Chang. Reston, VA: ASCE.
Wang, L., Y. Chang, H. Wang, Z. Wu, J. Pu, and X. Yang. 2017. “An active contour model based on local fitted images for image segmentation.” Inf. Sci. 418–419: 61–73. https://doi.org/10.1016/j.ins.2017.06.042.
Wang, N., Q. Zhao, S. Li, X. Zhao, and P. Zhao. 2018. “Damage classification for masonry historic structures using convolutional neural networks based on still images.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1073–1089. https://doi.org/10.1111/mice.12411.
Wang, N., X. Zhao, P. Zhao, Y. Zhang, Z. Zou, and J. Ou. 2019. “Automatic damage detection of historic masonry buildings based on mobile deep learning.” Autom. Constr. 103 (Jul 2018): 53–66. https://doi.org/10.1016/j.autcon.2019.03.003.
Zhang, X., Y.-W. Chiu, and H. Hao. 2022. “Dynamic tensile properties of clay bricks.” Mech. Mater. 165: 104157. https://doi.org/10.1016/j.mechmat.2021.104157.
Information & Authors
Information
Published In
Journal of Architectural Engineering
Volume 28 • Issue 4 • December 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Jan 7, 2022
Accepted: Jul 18, 2022
Published online: Sep 28, 2022
Published in print: Dec 1, 2022
Discussion open until: Feb 28, 2023
ASCE Technical Topics:
- Bricks
- Building materials
- Computer vision and image processing
- Construction (by type)
- Construction engineering
- Design (by type)
- Discrete element method
- Engineering fundamentals
- Engineering materials (by type)
- Geometrics
- Highway and road design
- Masonry
- Materials engineering
- Methodology (by type)
- Models (by type)
- Numerical methods
- Structural engineering
- Structural members
- Structural models
- Structural systems
- Three-dimensional models
- Walls
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Hongkai Du, Jiaxin Yu, Yuandong Wang, Yuhua Zhu, Yuyang Tang, Haimeng Wang, Visualized Failure Prediction for the Masonry Great Wall, Buildings, 10.3390/buildings12122224, 12, 12, (2224), (2022).