Deep Learning-Based Crowdsourced Image Localization in Digital Twin Models for Enhanced Infrastructure Management
Publication: Computing in Civil Engineering 2023
ABSTRACT
With the rapid urbanization and increasing complexity of infrastructure systems, there is a growing need for innovative and efficient methods to monitor and maintain these assets. The advent of digital twin technology and advancements in computer vision have opened new avenues for utilizing crowdsourced data to improve infrastructure management, monitor conditions, and prioritize maintenance activities. This study presents a novel approach for estimating the camera pose of crowdsourced images, enabling their integration into digital twin models without requiring explicit location information. This method mitigates data privacy concerns, promotes cost-effective monitoring, and enhances stakeholder communication by aligning 2D image data with 3D digital twin models. A case study demonstrates the proposed methodology’s effectiveness and adaptability in real-world situations, showcasing its potential for fostering resilient and sustainable urban environments.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Alavi, A. H., and Buttlar, W. G., (2019). “An overview of smartphone technology for citizen-centered, real-time and scalable civil infrastructure monitoring.” Future Gener Comput Syst, 93, pp.651–672.
Alvear, O., Calafate, C. T., Cano, J. C., and Manzoni, P. (2018). “Crowdsensing in smart cities: Overview, platforms, and environment sensing issues.” Sensors 18 (2): 460.
Balakrishnan Selvakumaran, S., and Hall, D. M. (2022). “From crowd to cloud: Simplified automatic reconstruction of digital building assets for facility management.” J. Facil. Manag., 20(3), 401–436.
Bentley. (2023). ContextCapture - 3D reality modelling software. Retrieved from https://www.bentley.com/en/products/brands/contextcapture.
Binalhaj, M., Liu, H., Sulaiman, M., and Abudayyeh, O. (2021). “Mobile crowdsourcing-based data collection for user-centered facility maintenance management.” Can. J. Civ. Eng, 48(12), 1652–1662.
Chen, K., Lu, M., Tan, G., and Wu, J. (2013). “CRSM: Crowdsourcing based road surface monitoring.” In 2013 IEEE 10th int. Conf. Intell. Comput. Commun. pp. 2151–2158. IEEE, 2013.
Chen, J., Li, S., Liu, D., and Lu, W. (2022). Indoor camera pose estimation via style‐transfer 3D models. Comput-Aided. Civ. Inf, 37(3), 335–353.
Choi, J., and Dyke, S. J. (2020). “CrowdLIM: Crowdsourcing to enable lifecycle infrastructure management.” Comput. Ind, 115, 103185.
Crooks, A., et al. (2015). “Crowdsourcing urban form and function”. Int. J. Geogr. Inf. Sci, 29(5), 720–741.
Data Visualization Research Lab. (2023). Point Cloud Unity Plugin -https://ccom.unh.edu/vislab/tools/point_cloud_plugin.
Errandonea, I., Beltrán, S., and Arrizabalaga, S. (2020). “Digital Twin for maintenance: A literature review.” Comput. Ind, 123, 103316.
Escolà, A., et al. (2017). “Mobile terrestrial laser scanner applications in precision fruticulture/horticulture and tools to extract information from canopy point clouds.” Precis. Agric, 18(1)11–132.
Giuliani, M., Castelletti, A., Fedorov, R., and Praternali, P. (2016). “Using crowdsourced web content for informing water systems operations in snow-dominated catchments.” Hydrol. Earth Syst. Sci, 20(12), 5049–5062.
Qi, C. R., et al. (2017). “Pointnet: Deep learning on point sets for 3d classification and segmentation.” CVPR, pp. 652–660.
Juliani, A., et al. (2018). “Unity: A general platform for intelligent agents.”.
Kim, H., Ham, Y., and Kim, H. (2019). “Localizing local vulnerabilities in urban areas using crowdsourced visual data from participatory sensing.” In Comput. Civ. Eng. 2019: Data, sensing, and analytics, pp. 522–529, Reston, VA: ASCE.
Li, E., et al. (2018). “Unified vision‐based methodology for simultaneous concrete defect detection and geolocalization.” Comput-Aided. Civ. Inf, 33(7), 527–544.
Sattler, T., et al. (2019). “Understanding the limitations of cnn-based absolute camera pose regression.” Proc. IEEE CVPR, pp. 3302–3312.
Shavit, Y., Ferens, R., and Keller, Y. (2021). “Learning multi-scene absolute pose regression with transformers.” In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2733–2742).
Vatn, J. (2018). “Industry 4.0 and real-time synchronization of operation and maintenance.” In Safety and Reliability–Safe Societies in a Changing World (pp. 681–686). CRC Press.
Wang, X., Zheng, X., Zhang, Q., Wang, T., and Shen, D. (2016). “Crowdsourcing in ITS: The state of the work and the networking.” IEEE transactions on intelligent transportation systems, 17(6), 1596–1605.
Xue, J., Hou, X., and Zeng, Y. (2021). “Review of image-based 3D reconstruction of building for automated construction progress monitoring.” Applied Sciences, 11(17), 7840.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 562 - 570
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Architectural engineering
- Building information modeling
- Building management
- Business management
- Case studies
- Engineering fundamentals
- Infrastructure
- Innovation
- Management methods
- Methodology (by type)
- Models (by type)
- Practice and Profession
- Research methods (by type)
- Three-dimensional models
- Two-dimensional models
- Urban and regional development
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.