Perception-Aware Tag Placement Planning for Robust Localization of UAVs in Indoor Construction Environments
Publication: Journal of Computing in Civil Engineering
Volume 37, Issue 2
Abstract
Tag-based visual-inertial localization is a lightweight method for enabling autonomous data collection missions of low-cost unmanned aerial vehicles (UAVs) in indoor construction environments. However, finding the optimal tag configuration (i.e., number, size, and location) on dynamic construction sites remains challenging. This work proposes a perception-aware genetic algorithm-based tag placement planner (PGA-TaPP) to determine the optimal tag configuration using four-dimensional (4D) building information models (BIM), considering the project progress, safety requirements, and UAV’s localizability. The proposed method provides a 4D plan for tag placement by maximizing the localizability in user-specified regions of interest (ROIs) while limiting the installation costs. Localizability is quantified using the Fisher information matrix (FIM) and encapsulated in navigable grids. The experimental results show the effectiveness of our method in finding an optimal 4D tag placement plan for the robust localization of UAVs on under-construction indoor sites.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors appreciate the financial support from the Natural Science and Engineering Research Council (NSERC) (Grant No. RGPIN-2017-06792). The opinions, findings, and conclusions presented in this work are those of the authors and do not necessarily reflect the views of the entity mentioned previously.
References
Adán, A., B. Quintana, S. A. Prieto, and F. Bosché. 2020. “An autonomous robotic platform for automatic extraction of detailed semantic models of buildings.” Autom. Constr. 109 (Jan): 102963. https://doi.org/10.1016/j.autcon.2019.102963.
Asadi, K., A. Kalkunte Suresh, A. Ender, S. Gotad, S. Maniyar, S. Anand, M. Noghabaei, K. Han, E. Lobaton, and T. Wu. 2020. “An integrated UGV-UAV system for construction site data collection.” Autom. Constr. 112 (Apr): 103068. https://doi.org/10.1016/j.autcon.2019.103068.
Barfoot, T. D. 2017. State estimation for robotics. Cambridge, UK: Cambridge University Press.
Brommer, C., D. Malyuta, D. Hentzen, and R. Brockers. 2018. “Long-duration autonomy for small rotorcraft UAS including recharging.” In Proc., 2018 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 7252–7258. New York: IEEE.
Cadena, C., L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard. 2016. “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age.” IEEE Trans. Rob. Autom. 32 (6): 1309–1332. https://doi.org/10.1109/TRO.2016.2624754.
Cai, S., Z. Ma, M. J. Skibniewski, and S. Bao. 2019. “Construction automation and robotics for high-rise buildings over the past decades: A comprehensive review.” Adv. Eng. Inf. 42 (Oct): 100989. https://doi.org/10.1016/j.aei.2019.100989.
Deasy, T. P., and W. G. Scanlon. 2004. “Stepwise algorithms for improving the accuracy of both deterministic and probabilistic methods in WLAN-based indoor user localization.” Int. J. Wirel. Inf. Netw. 11 (4): 207–216. https://doi.org/10.1007/s10776-004-1234-1.
Delmerico, J., and D. Scaramuzza. 2018. “A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots.” In Proc., 2018 IEEE Int. Conf. on Robotics and Automation, 2502–2509. New York: IEEE. https://doi.org/10.1109/ICRA.2018.8460664.
Freimuth, H., and M. König. 2018. “Planning and executing construction inspections with unmanned aerial vehicles.” Autom. Constr. 96 (Dec): 540–553. https://doi.org/10.1016/j.autcon.2018.10.016.
Gheisari, M., J. Irizarry, and B. N. Walker. 2014. “UAS4SAFETY: The potential of unmanned aerial systems for construction safety applications.” In Proc., Construction Research Congress 2014, 1801–1810. Reston, VA: ASCE.
Ham, Y., K. K. Han, J. J. Lin, and M. Golparvar-Fard. 2016. “Visual monitoring of civil infrastructure systems via camera-equipped unmanned Aerial vehicles (UAVs): A review of related works.” Visualization Eng. 4 (1): 1. https://doi.org/10.1186/s40327-015-0029-z.
Hamledari, H., B. McCabe, and S. Davari. 2017. “Automated computer vision-based detection of components of under-construction indoor partitions.” Autom. Constr. 74 (Feb): 78–94. https://doi.org/10.1016/j.autcon.2016.11.009.
Hamledari, H., S. Sajedi, B. McCabe, and M. Fischer. 2021. “Automation of inspection mission planning using 4D BIMs and in support of unmanned aerial vehicle-based data collection.” J. Constr. Eng. Manage. 147 (3): 04020179. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001995.
Ibrahim, A., A. Sabet, and M. Golparvar-Fard. 2019. “BIM-driven mission planning and navigation for automatic indoor construction progress detection using robotic ground platform.” In Vol. 1 of Proc., 2019 European Conf. for Computing in Construction, 182–189. Sint-Niklaas, Belgium: European Council on Computing in Construction.
Ibrahim, M., and O. Moselhi. 2016. “Inertial measurement unit based indoor localization for construction applications.” Autom. Constr. 71 (Part 1): 13–20. https://doi.org/10.1016/j.autcon.2016.05.006.
Jin, M., S. Liu, S. Schiavon, and C. Spanos. 2018. “Automated mobile sensing: Towards high-granularity agile indoor environmental quality monitoring.” Build. Environ. 127 (Jan): 268–276. https://doi.org/10.1016/j.buildenv.2017.11.003.
Kayhani, N., A. Heins, W. Zhao, M. Nahangi, B. McCabe, and A. Schoellig. 2019. “Improved tag-based indoor localization of UAVs using extended kalman filter. In Proc., 36th Int. Symp. on Automation and Robotics in Construction, ISARC 2019, 624–631. Banff, AB, Canada: International Association for Automation and Robotics in Construction.
Kayhani, N., B. McCabe, A. Abdelaal, A. Heins, and A. P. Schoellig. 2020. “Tag-based indoor localization of UAVs in construction environments: Opportunities and challenges in practice.” In Proc., Construction Research Congress 2020, 226–235. Reston, VA: ASCE.
Kayhani, N., W. Zhao, B. McCabe, and A. P. Schoellig. 2022. “Tag-based visual-inertial localization of unmanned aerial vehicles in indoor construction environments using an on-manifold extended Kalman filter.” Autom. Constr. 135 (Mar): 104112. https://doi.org/10.1016/j.autcon.2021.104112.
Kielhauser, C., R. Renteria Manzano, J. J. Hoffman, and B. T. Adey. 2020. “Automated construction progress and quality monitoring for commercial buildings with unmanned aerial systems: An application study from Switzerland.” Infrastructures 5 (11): 98. https://doi.org/10.3390/infrastructures5110098.
Kim, P., J. Chen, and Y. K. Cho. 2018. “SLAM-driven robotic mapping and registration of 3D point clouds.” Autom. Constr. 89 (May): 38–48. https://doi.org/10.1016/j.autcon.2018.01.009.
Laakso, A. 2012. “The IFC standard: A review of history, development, and standardization, information technology.” ITcon 17 (9): 134–161.
Lin, J. J., A. Ibrahim, S. Sarwade, and M. Golparvar-Fard. 2021. “Bridge inspection with aerial robots: Automating the entire pipeline of visual data capture, 3D mapping, defect detection, analysis, and reporting.” J. Comput. Civ. Eng. 35 (2): 04020064. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000954.
Liu, T., L. Yang, Q. Lin, Y. Guo, and Y. Liu. 2014. “Anchor-free backscatter positioning for RFID tags with high accuracy.” In Proc., IEEE INFOCOM, 379–387. New York: IEEE. https://doi.org/10.1109/INFOCOM.2014.6847960.
Lozano-Pérez, T., and M. A. Wesley. 1979. “An algorithm for planning collision-free paths among polyhedral obstacles.” Commun. ACM 22 (10): 560–570. https://doi.org/10.1145/359156.359164.
Mantha, B. R. K., C. C. Menassa, and V. R. Kamat. 2018. “Robotic data collection and simulation for evaluation of building retrofit performance.” Autom. Constr. 92 (Aug): 88–102. https://doi.org/10.1016/j.autcon.2018.03.026.
Martinez, J. G., G. Albeaino, M. Gheisari, R. R. A. Issa, and L. F. Alarcón. 2021. “iSafeUAS: An unmanned aerial system for construction safety inspection.” Autom. Constr. 125 (May): 103595. https://doi.org/10.1016/j.autcon.2021.103595.
Moselhi, O., H. Bardareh, and Z. Zhu. 2020. “Automated data acquisition in construction with remote sensing technologies.” Appl. Sci. 10 (8): 2846. https://doi.org/10.3390/app10082846.
Muñoz-Salinas, R., M. J. Marín-Jimenez, and R. Medina-Carnicer. 2019. “SPM-SLAM: Simultaneous localization and mapping with squared planar markers.” Pattern Recognit. 86 (Feb): 156–171. https://doi.org/10.1016/j.patcog.2018.09.003.
Muñoz-Salinas, R., and R. Medina-Carnicer. 2020. “UcoSLAM: Simultaneous localization and mapping by fusion of keypoints and squared planar markers.” Pattern Recognit. 101 (May): 107193. https://doi.org/10.1016/j.patcog.2019.107193.
Mur-Artal, R., J. M. M. Montiel, and J. D. Tardos. 2015. “ORB-SLAM: A versatile and accurate monocular SLAM system.” IEEE Trans. Rob. 31 (5): 1147–1163. https://doi.org/10.1109/TRO.2015.2463671.
Neunert, M., M. Bloesch, and J. Buchli. 2016. “An open source, fiducial based, visual-inertial motion capture system.” In Proc., 2016 19th Int. Conf. on Information Fusion (FUSION), 1523–1530. New York: IEEE.
Pal, A., and S. H. Hsieh. 2021. “Deep-learning-based visual data analytics for smart construction management.” Autom. Constr. 131 (Nov): 103892. https://doi.org/10.1016/j.autcon.2021.103892.
Quigley, M., B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs, E. Berger, R. Wheeler, and A. Ng. 2009. ROS: An open-source robot operating system. New York: IEEE.
Rakha, T., and A. Gorodetsky. 2018. “Review of unmanned aerial system (UAS) applications in the built environment: Towards automated building inspection procedures using drones.” Autom. Constr. 93 (Sep): 252–264. https://doi.org/10.1016/j.autcon.2018.05.002.
Razavi, S. N., and O. Moselhi. 2012. “GPS-less indoor construction location sensing.” Autom. Constr. 28 (Dec): 128–136. https://doi.org/10.1016/j.autcon.2012.05.015.
Siebert, S., and J. Teizer. 2014. “Mobile 3D mapping for surveying earthwork projects using an unmanned aerial vehicle (UAV) system.” Autom. Constr. 41 (May): 1–14. https://doi.org/10.1016/j.autcon.2014.01.004.
Teizer, J. 2015. “Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites.” Adv. Eng. Inf. 29 (2): 225–238. https://doi.org/10.1016/j.aei.2015.03.006.
Wang, J., and E. Olson. 2016. “AprilTag 2: Efficient and robust fiducial detection.” In Proc., IEEE Int. Conf. on Intelligent Robots and Systems, 4193–4198. New York: IEEE.
Wang, Z., and G. Dissanayake. 2008. “Observability analysis of SLAM using Fisher information matrix.” In Proc., 2008 10th Int. Conf. on Control, Automation, Robotics and Vision, ICARCV 2008, 1242–1247. New York: IEEE.
Witrisal, K., and P. Meissner. 2012. “Performance bounds for multipath-assisted indoor navigation and tracking (MINT).” In Proc., IEEE Int. Conf. on Communications, 4321–4325. New York: IEEE.
Xu, L., C. Feng, V. R. Kamat, and C. C. Menassa. 2019. “An occupancy grid mapping enhanced visual SLAM for real-time locating applications in indoor GPS-denied environments.” Autom. Constr. 104 (Aug): 230–245. https://doi.org/10.1016/j.autcon.2019.04.011.
Xu, L., C. Feng, V. R. Kamat, and C. C. Menassa. 2020. “A scene-adaptive descriptor for visual SLAM-based locating applications in built environments.” Autom. Constr. 112 (Apr): 103067. https://doi.org/10.1016/j.autcon.2019.103067.
Yi, S., S. Worrall, and E. Nebot. 2019. “Metrics for the evaluation of localisation robustness.” In Proc., 2019 IEEE Intelligent Vehicles Symp. (5), 1247–1253. New York: IEEE.
Zhang, Z., and D. Scaramuzza. 2019. “Beyond point clouds: Fisher information field for active visual localization.” In Proc., 2019 Int. Conf. on Robotics and Automation (ICRA), 5986–5992. New York: IEEE.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 37 • Issue 2 • March 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Jun 28, 2022
Accepted: Oct 1, 2022
Published online: Dec 21, 2022
Published in print: Mar 1, 2023
Discussion open until: May 21, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.