Transformer-Based Semantic Segmentation for Recycling Materials in Construction
Publication: Computing in Civil Engineering 2023
ABSTRACT
With the increasing awareness of sustainable development, reusing and recycling the waste materials are gaining more and more importance on construction sites. The semantic segmentation techniques provide a powerful tool to identify the construction recycling materials automatically for their subsequent management. Among these segmentation techniques, recently proposed transformer-based architecture could understand long-range dependencies and learn highly expressive representations. This paper presented the transformer-based semantic segmentation for recycling materials in construction. It started with an evaluation of two state-of-the-art transformer-based architecture types. The selected architecture types included Twins Transformer and K-Net. Further, the ensemble learning strategy of model averaging was implemented to improve the segmentation performance. The idea of model averaging was to average different checkpoint weights for each transformer-based architecture along a single training trajectory. To evaluate the segmentation performance, a novel dataset containing five classes of recycling materials on construction sites was created and employed as a benchmark. The evaluation results indicated the ensemble models of Twins Transformer and K-Net achieved the mean Intersection over Union (mIoU) of 80.5% and 82.2%, separately, which demonstrated superior performance on segmenting recycling materials in construction. Our segmentation model won the first place in VIMS-IAARC Joint Datathon 2022 Competition.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Atkinson, G. A., Zhang, W., Hansen, M. F., Holloway, M. L., and Napier, A. A. (2020). “Image segmentation of underfloor scenes using a mask regions convolutional neural network with two-stage transfer learning.” Automation in Construction.
Bang, S., Hong, Y., and Kim, H. (2021). “Proactive proximity monitoring with instance segmentation and unmanned aerial vehicle-acquired video-frame prediction.” Computer-Aided Civil and Infrastructure Engineering.
Chen, J., Lu, W., and Xue, F. (2021). “‘Looking beneath the surface’: A visual-physical feature hybrid approach for unattended gauging of construction waste composition.” Journal of Environmental Management.
Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., Xia, H., and Shen, C. (2021). “Twins: Revisiting the Design of Spatial Attention in Vision Transformers.” Advances in Neural Information Processing Systems, 34, 9355–9366.
Dhanachandra, N., Manglem, K., and Chanu, Y. J. (2015). “Image Segmentation Using K-means Clustering Algorithm and Subtractive Clustering Algorithm.” Procedia Computer Science.
Dutta, A., and Zisserman, A. (2019). “The VIA annotation software for images, audio and video.” MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, 2276–2279.
Kabirifar, K., Mojtahedi, M., Wang, C., and Tam, V. W. Y. (2020). “Construction and demolition waste management contributing factors coupled with reduce, reuse, and recycle strategies for effective waste management: A review.” Journal of Cleaner Production.
Kang, B., and Nguyen, T. Q. (2019). “Random Forest with Learned Representations for Semantic Segmentation.” IEEE Transactions on Image Processing.
Ku, Y., Yang, J., Fang, H., Xiao, W., and Zhuang, J. (2021). “Deep learning of grasping detection for a robot used in sorting construction and demolition waste.” Journal of Material Cycles and Waste Management.
Long, J., Shelhamer, E., and Darrell, T. (2015). “Fully convolutional networks for semantic segmentation.” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
Loshchilov, I., and Hutter, F. (2019). “Decoupled weight decay regularization.” 7th International Conference on Learning Representations, ICLR 2019.
Lu, W., Chen, J., and Xue, F. (2022). “Using computer vision to recognize composition of construction waste mixtures: A semantic segmentation approach.” Resources, Conservation and Recycling.
Souly, N., Spampinato, C., and Shah, M. (2017). “Semi Supervised Semantic Segmentation Using Generative Adversarial Network.” Proceedings of the IEEE International Conference on Computer Vision.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). “Attention is all you need.” Advances in Neural Information Processing Systems.
Wang, X., Han, W., Mo, S., Cai, T., Gong, Y., Li, Y., and Zhu, Z. (2023). “Transformer-Based Segmentation for Semantic Understanding in Construction.” Automation in Construction, (In Revision).
Wortsman, M., et al. (2022). “Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time.” Proceedings of the 39th International Conference on Machine Learning, PMLR, 23965–23998.
Xie, X., Cai, J., Wang, H., Wang, Q., Xu, J., Zhou, Y., and Zhou, B. (2022). “Sparse-sensing and superpixel-based segmentation model for concrete cracks.” Computer-Aided Civil and Infrastructure Engineering, John Wiley & Sons, Ltd.
Zhang, W., Pang, J., Chen, K., and Loy, C. C. (2021). “K-Net: Towards Unified Image Segmentation.” Advances in Neural Information Processing Systems, Neural information processing systems foundation, 13, 10326–10338.
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., and Torralba, A. (2017). “Scene parsing through ADE20K dataset.” Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 25 - 33
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Architectural engineering
- Architecture
- Business management
- Construction engineering
- Construction management
- Construction materials
- Construction sites
- Construction wastes
- Engineering materials (by type)
- Environmental engineering
- Materials engineering
- Pollutants
- Practice and Profession
- Recycling
- Solid wastes
- Sustainable development
- Waste management
- Waste sites
- Wastes
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.