Deep Learning-Based Image Steganography for Visual Data Cybersecurity in Construction Management
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 10
Abstract
The construction industry is increasingly digital and dependent on extensive use of information technologies. However, data exchange in a digital environment makes construction data more vulnerable to cyber risks. For instance, construction videos contain various site information (such as worker privacy, innovative techniques, and infrastructures status), the loss of which may cause financial and safety issues. To ensure the cybersecurity of visual data in construction, this research proposes a deep learning-based image steganography method, which can cover the secret image with an irrelevant image by using a hidden neural network and retrieve the secret image with a reveal neural network. In experiments, a dataset containing 7,000 construction images was used for validating the feasibility of the proposed method. Three evaluation metrics were used to test the performance of proposed method in visual information hiding and recovery. Specifically, the proposed method achieved a peak signal-to-noise ratio of 36.58, a structural similarity index of 97.29%, and a visual information fidelity of 82.57% on average. The test results demonstrate the reliable performance of the proposed method in protecting construction visual data. This research provides a novel way to ensure the cybersecurity of visual data in construction, other than simple password encryptions.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
References
Chen, C., B. Xiao, Y. Zhang, and Z. Zhu. 2023. “Automatic vision-based calculation of excavator earthmoving productivity using zero-shot learning activity recognition.” Autom. Constr. 146 (Mar): 104702. https://doi.org/10.1016/j.autcon.2022.104702.
Chen, C., Z. Zhu, and A. Hammad. 2020. “Automated excavators activity recognition and productivity analysis from construction site surveillance videos.” Autom. Constr. 110 (Mar): 103045. https://doi.org/10.1016/j.autcon.2019.103045.
Chen, C., Z. Zhu, A. Hammad, and M. Akbarzadeh. 2021. “Automatic Identification of idling reasons in excavation operations based on excavator–truck relationships.” J. Comput. Civ. Eng. 35 (5): 04021015. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000981.
Cheng, J. P., P. K. Y. Wong, H. Luo, M. Wang, and P. H. Leung. 2022. “Vision-based monitoring of site safety compliance based on worker re-identification and personal protective equipment classification.” Autom. Constr. 139 (Jul): 104312. https://doi.org/10.1016/j.autcon.2022.104312.
Chow, J. K., K. Liu, P. S. Tan, Z. Su, J. Wu, Z. Li, and Y.-H. Wang. 2021. “Automated defect inspection of concrete structures.” Autom. Constr. 132 (Mar): 103959. https://doi.org/10.1016/j.autcon.2021.103959.
Construction News. 2018. “Cyber-security: What will it take for construction to act?” Accessed February 4, 2023. https://www.constructionnews.co.uk/tech/cyber-security-what-will-it-take-for-construction-to-act-22-01-2018/.
Davies, V. 2023. “Construction and transport sector high cyber targets.” Accessed January 22, 2024. https://cybermagazine.com/articles/construction-and-transport-sector.
Fang, W., L. Ding, H. Luo, and P. E. D. Love. 2018. “Falls from heights: A computer vision-based approach for safety harness detection.” Autom. Constr. 91 (Jun): 53–61. https://doi.org/10.1016/j.autcon.2018.02.018.
Flah, M., A. R. Suleiman, and M. L. Nehdi. 2020. “Classification and quantification of cracks in concrete structures using deep learning image-based techniques.” Cem. Concr. Compos. 114 (Mar): 103781. https://doi.org/10.1016/j.cemconcomp.2020.103781.
Gallagher, U. S. 2020. “Cyber risk within the construction industry.” Accessed February 4, 2023. https://www.ajg.com/us/news-and-insights/2020/jun/cyber-risk-within-construction-industry/.
García de Soto, B., A. Georgescu, B. Mantha, Ž. Turk, A. Maciel, and M. Semih Sonkor. 2022. “Construction cybersecurity and critical infrastructure protection: new horizons for Construction 4.0.” J. Inf. Technol. Constr. 27 (Jun): 571–594. https://doi.org/10.36680/j.itcon.2022.028.
Iglovikov, V., and A. Shvets. 2018. “TernausNet: U-Net with VGG11 encoder pre-trained on imagenet for image segmentation.” Preprint, submitted January 17, 2018. https://arxiv.org/abs/1804.08738.1801.05746.
Kadhim, I. J., P. Premaratne, P. J. Vial, and B. Halloran. 2019. “Comprehensive survey of image steganography: Techniques, Evaluations, and trends in future research.” Neurocomputing 335 (Mar): 299–326. https://doi.org/10.1016/j.neucom.2018.06.075.
Kim, H., K. Kim, and H. Kim. 2016. “Vision-based object-centric safety assessment using fuzzy inference: monitoring struck-by accidents with moving objects.” J. Comput. Civ. Eng. 30 (4): 04015075. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000562.
Kim, J., and S. Chi. 2019. “Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles.” Autom. Constr. 104 (Jan): 255–264. https://doi.org/10.1016/j.autcon.2019.03.025.
Kingma, D. P., and J. Ba. 2017. “Adam: A method for stochastic optimization.” Preprint, submitted December 22, 2014. https://arxiv.org/abs/1412.6980.
Kumar, S., H. Gupta, D. Yadav, I. Ansari, and O. Verma. 2022. “YOLOv4 algorithm for the real-time detection of fire and personal protective equipments at construction sites.” Multimedia Tools Appl. 81 (16): 22163–22183. https://doi.org/10.1007/s11042-021-11280-6.
Kumari, M., S. Gupta, and P. Sardana. 2017. “A survey of image encryption algorithms.” 3D Res. 8 (Jun): 1–35. https://doi.org/10.1007/s13319-017-0148-5.
Le, Q. H., J. W. Lee, and S. Y. Yang. 2017. “Remote control of excavator using head tracking and flexible monitoring method.” Autom. Constr. 81 (Mar): 99–111. https://doi.org/10.1016/j.autcon.2017.06.015.
Long, L., and F. Li. 2018. “A formula adaptive pixel pair matching steganography algorithm.” Adv. Multimedia 2018 (1): 7682098. https://doi.org/10.1155/2018/7682098.
Luo, H., C. Xiong, W. Fang, P. E. D. Love, B. Zhang, and X. Ouyang. 2018a. “Convolutional neural networks: Computer vision-based workforce activity assessment in construction.” Autom. Constr. 94 (Mar): 282–289. https://doi.org/10.1016/j.autcon.2018.06.007.
Luo, X., H. Li, D. Cao, Y. Yu, X. Yang, and T. Huang. 2018b. “Towards efficient and objective work sampling: Recognizing workers’ activities in site surveillance videos with two-stream convolutional networks.” Autom. Constr. 94 (Oct): 360–370. https://doi.org/10.1016/j.autcon.2018.07.011.
Luo, X., H. Li, Y. Yu, C. Zhou, and D. Cao. 2020. “Combining deep features and activity context to improve recognition of activities of workers in groups.” Comput.-Aided Civ. Infrastruct. Eng. 35 (9): 965–978. https://doi.org/10.1111/mice.12538.
Mahmoudpour, S., and M. Kim. 2015. “A study on the relationship between depth map quality and stereoscopic image quality using upsampled depth maps.” In Emerging trends in image processing, computer vision and pattern recognition, 149–160. Amsterdam, Netherlands: Elsevier.
Mantha, B., and B. García de Soto. 2019. “Cyber security challenges and vulnerability assessment in the construction industry.” In Proc., Creative Construction Conf. 2019. Budapest, Hungary: Budapest Univ. of Technology and Economics.
Mantha, B., B. García de Soto, and R. Karri. 2021. “Cyber security threat modeling in the AEC industry: An example for the commissioning of the built environment.” Sustainable Cities Soc. 66 (Mar): 102682. https://doi.org/10.1016/j.scs.2020.102682.
Mantha, B. R. K., and B. García de Soto. 2021. “Assessment of the cybersecurity vulnerability of construction networks.” Eng. Constr. Archit. Manage. 28 (10): 3078–3105. https://doi.org/10.1108/ECAM-06-2020-0400.
MarshMcLennan. 2021. “Cyber risk and the construction supply chain.” Accessed February 10, 2024. https://www.marshmclennan.com/insights/publications/2021/april-/cyber-risk-and-the-construction-supply-chain.html.
McLaughlin, E., N. Charron, and S. Narasimhan. 2020. “Automated defect quantification in concrete bridges using robotics and deep learning.” J. Comput. Civ. Eng. 34 (5): 04020029 https://doi.org/10.1061/(ASCE)CP.1943-5487.0000915.
Miura, H., A. Watanabe, S. Suzuki, and M. Okugawa. 2016. “Field experiment report for tunnel disaster by investigation system with multiple robots.” In Proc., 2016 IEEE Int. Symp. on Safety, Security, and Rescue Robotics (SSRR), 276–277. New York: IEEE.
Obando, S. 2022. “US construction tech firms brace for increased cyberattacks.” Accessed February 4, 2023. https://www.constructiondive.com/news/cybersecurity-experts-automation-materials-worst-case-scenario-hackers/621210/.
Park, M.-W., N. Elsafty, and Z. Zhu. 2015. “Hardhat-wearing detection for enhancing on-site safety of construction workers.” J. Constr. Eng. Manage. 141 (9): 04015024 https://doi.org/10.1061/(ASCE)CO.1943-7862.0000974.
Ronneberger, O., P. Fischer, and T. Brox. 2015. “U-Net: Convolutional networks for biomedical image segmentation.” submitted May 18, 2015. https://doi.org/10.48550/arXiv.1505.04597.
Sekizuka, R., M. Ito, S. Saiki, Y. Yamazaki, and Y. Kurita. 2020. “System to evaluate the skill of operating hydraulic excavators using a remote controlled excavator and virtual reality.” Front. Rob. AI 6 (Jan): 142. https://doi.org/10.3389/frobt.2019.00142.
Sheikh, H. R., and A. C. Bovik. 2006. “Image information and visual quality.” IEEE Trans. Image Proc. 15 (2): 430–444. https://doi.org/10.1109/TIP.2005.859378.
Shi, H., J. Dong, W. Wang, Y. Qian, and X. Zhang. 2018. “SSGAN: Secure steganography based on generative adversarial networks.” submitted July 6, 2017. https://doi.org/10.48550/arXiv.1707.01613.
Sonkor, M. S., and B. García de Soto. 2021. “Is your construction site secure? A view from the cybersecurity perspective.” In Vol. 38 of Proc., Int. Symp. on Automation and Robotics in Construction, 864–871. Edinburg, UK: International Association for Automation and Robotics in Construction Publications.
Subramanian, N., O. Elharrouss, S. Al-Maadeed, and A. Bouridane. 2021. “Image steganography: A review of the recent advances.” IEEE Access 9 (Mar): 23409–23423. https://doi.org/10.1109/ACCESS.2021.3053998.
Tang, W., S. Tan, B. Li, and J. Huang. 2017. “Automatic steganographic distortion learning using a generative adversarial network.” IEEE Signal Process Lett. 24 (10): 1547–1551. https://doi.org/10.1109/LSP.2017.2745572.
Turk, Ž., B. García de Soto, B. R. K. Mantha, A. Maciel, and A. Georgescu. 2022. “A systemic framework for addressing cybersecurity in construction.” Autom. Constr. 133 (Feb): 103988. https://doi.org/10.1016/j.autcon.2021.103988.
Volkhonskiy, D., I. Nazarov, and E. Burnaev. 2019. “Steganographic generative adversarial networks.” In Proc., 12th Int. Conf. on Machine Vision (ICMV 2019). Washington, DC: SPIE.
Wang, Z., A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. “Image quality assessment: from error visibility to structural similarity.” IEEE Trans. Image Proc. 13 (4): 600–612. https://doi.org/10.1109/TIP.2003.819861.
Weng, X., Y. Li, L. Chi, and Y. Mu. 2019. “High-capacity convolutional video steganography with temporal residual modeling.” In Proc., 2019 on Int. Conf. on Multimedia Retrieval, 87–95. New York: Association for Computing Machinery.
Wu, K., and B. García de Soto. 2022. “Current state and future opportunities of data mining for construction 4.0.” In Proc., 39th ISARC, 78–85. Edinburg, UK: International Association for Automation and Robotics in Construction.
Xiao, B., X. Yin, and S.-C. Kang. 2021a. “Vision-based method of automatically detecting construction video highlights by integrating machine tracking and CNN feature extraction.” Autom. Constr. 129 (Mar): 103817. https://doi.org/10.1016/j.autcon.2021.103817.
Xiao, B., Y. Zhang, Y. Chen, and X. Yin. 2021b. “A semi-supervised learning detection method for vision-based monitoring of construction sites by integrating teacher-student networks and data augmentation.” Adv. Eng. Inf. 50 (Mar): 101372. https://doi.org/10.1016/j.aei.2021.101372.
Yao, D., and B. Garcia de Soto. 2022. “A preliminary SWOT evaluation for the applications of ML to cyber risk analysis in the construction industry.” IOP Conf. Ser.: Mater. Sci. Eng. 1218 (1): 012017. https://doi.org/10.1088/1757-899X/1218/1/012017.
Yoo, J.-C., and C. W. Ahn. 2012. “Image matching using peak signal-to-noise ratio-based occlusion detection.” IET Image Proc. 6 (5): 483–495. https://doi.org/10.1049/iet-ipr.2011.0025.
Yu, M., L. Gong, and S. Kollias. 2017. “Computer vision based fall detection by a convolutional neural network.” In Proc., 19th ACM Int. Conf. on Multimodal Interaction, ICMI ’17, 416–420. New York: Association for Computing Machinery.
Zhang, J., S. Qian, and C. Tan. 2022a. “Automated bridge surface crack detection and segmentation using computer vision-based deep learning model.” Eng. Appl. Artif. Intell. 115 (Jun): 105225. https://doi.org/10.1016/j.engappai.2022.105225.
Zhang, L., J. Wang, Y. Wang, H. Sun, and X. Zhao. 2022b. “Automatic construction site hazard identification integrating construction scene graphs with BERT based domain knowledge.” Autom. Constr. 142 (Oct): 104535. https://doi.org/10.1016/j.autcon.2022.104535.
Zhang, R., S. Dong, and J. Liu. 2019. “Invisible steganography via generative adversarial networks.” Multimedia Tools Appl. 78 (7): 8559–8575. https://doi.org/10.1007/s11042-018-6951-z.
Zhu, J., R. Kaplan, J. Johnson, and L. Fei-Fei. 2018. “HiDDeN: Hiding data with deep networks.” In Proc., European Conf. on Computer Vision (ECCV), 657–672. Berlin: Springer.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 10 • October 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Nov 2, 2023
Accepted: Apr 11, 2024
Published online: Jul 17, 2024
Published in print: Oct 1, 2024
Discussion open until: Dec 17, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.