Autonomous Drones in Urban Navigation: Autoencoder Learning Fusion for Aerodynamics
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 7
Abstract
Drones are becoming indispensable in emergency search and rescue (SAR), particularly in intricate urban areas where rapid and accurate response is crucial. This study addresses the pressing need for enhancing drone navigation in such complex, dynamic urban environments, where obstacles like building layouts and varying wind conditions create unique challenges. Particularly, the need for adapting drone autonomous navigation in correspondence with dynamic wind conditions in urban settings is emphasized because it is important for drones to avoid loss of control or crashes during SAR. This paper introduces a pioneering method integrating multiobjective reinforcement learning (MORL) with a convolutional autoencoder to train autonomous drones in comprehending and reacting to aerodynamic features in urban SAR. MORL enables the drone to optimize multiple goals, whereas the convolutional autoencoder generates synthetic wind simulations with a substantially lower computation cost compared to traditional computational fluid dynamics (CFD) simulations. A unique data transfer structure is also proposed, which fosters a seamless integration of perception and decision-making between machine learning (ML) and reinforcement learning (RL) components. This approach uses imagery data, specific to building layouts, allowing the drone to autonomously formulate policies, prioritize navigation decisions, optimize paths, and mitigate the impact of wind, all while negating the necessity for conventional aerodynamic force sensors. The method was validated with a model of New York City, offering substantial implications for enhancing automation algorithms in urban SAR. This innovation enables the possibility of more efficient, precise, and timely drone SAR operations within intricate urban landscapes.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This material is supported by the Air Force Office of Scientific Research (AFOSR) under Grant FA9550-22-1-0492. Any opinions, findings, conclusions, or recommendations expressed in this article are those of the authors and do not reflect the views of the AFOSR.
References
Arafat, M. Y., M. M. Alam, and S. Moh. 2023. “Vision-based navigation techniques for unmanned aerial vehicles: Review and challenges.” Drones 7 (2): 89. https://doi.org/10.3390/drones7020089.
Arshad, M. A., S. H. Khan, S. Qamar, M. W. Khan, I. Murtza, J. Gwak, and A. Khan. 2022. “Drone navigation using region and edge exploitation-based deep CNN.” IEEE Access 10 (Sep): 95441–95450. https://doi.org/10.1109/ACCESS.2022.3204876.
Aslan, S., and T. Erkin. 2023. “A multi-population immune plasma algorithm for path planning of unmanned combat aerial vehicle.” Adv. Eng. Inf. 55 (Jan): 101829. https://doi.org/10.1016/j.aei.2022.101829.
Biao, L., J. Cunyan, W. Lu, C. Weihua, and L. Jing. 2019. “A parametric study of the effect of building layout on wind flow over an urban area.” Build. Environ. 160 (Aug): 106160. https://doi.org/10.1016/j.buildenv.2019.106160.
Blocken, B., T. Stathopoulos, and J. Carmeliet. 2008. “Wind environmental conditions in passages between two long narrow perpendicular buildings.” J. Aerosp. Eng. 21 (4): 280–287. https://doi.org/10.1061/(ASCE)0893-1321(2008)21:4(280).
Bolognini, M., and L. Fagiano. 2020. “LiDAR-based navigation of tethered drone formations in an unknown environment.” IFAC-PapersOnLine 53 (2): 9426–9431. https://doi.org/10.1016/j.ifacol.2020.12.2413.
Calzolari, G., and W. Liu. 2021. “Deep learning to replace, improve, or aid CFD analysis in built environment applications: A review.” Build. Environ. 206 (Dec): 108315. https://doi.org/10.1016/j.buildenv.2021.108315.
Capolupo, A., S. Pindozzi, C. Okello, N. Fiorentino, and L. Boccia. 2015. “Photogrammetry for environmental monitoring: The use of drones and hydrological models for detection of soil contaminated by copper.” Sci. Total Environ. 514 (May): 298–306. https://doi.org/10.1016/j.scitotenv.2015.01.109.
Chakravarty, P., K. Kelchtermans, T. Roussel, S. Wellens, T. Tuytelaars, and L. Van Eycken. 2017. “CNN-based single image obstacle avoidance on a quadrotor.” In Proc., 2017 IEEE Int. Conf. on Robotics and Automation (ICRA), 6369–6374. New York: IEEE.
Chen, Q., J. Chen, and W. Huang. 2022. “Pathfinding method for an indoor drone based on a BIM-semantic model.” Adv. Eng. Inf. 53 (Aug): 101686. https://doi.org/10.1016/j.aei.2022.101686.
Chu, T., M. J. Starek, J. Berryhill, C. Quiroga, and M. Pashaei. 2021. “Simulation and characterization of wind impacts on sUAS flight performance for crash scene reconstruction.” Drones 5 (3): 67. https://doi.org/10.3390/drones5030067.
Du, J. 2021. “VR Drone. YouTube.” Accessed November 17, 2023. https://youtu.be/vS9vtHbjhDg.
Elmokadem, T., and A. V. Savkin. 2021. “A hybrid approach for autonomous collision-free UAV navigation in 3D partially unknown dynamic environments.” Drones 5 (3): 57. https://doi.org/10.3390/drones5030057.
El-Sheimy, N., and Y. Li. 2021. “Indoor navigation: State of the art and future trends.” Satell. Navig. 2 (1): 1–23. https://doi.org/10.1186/s43020-021-00041-3.
Entrop, A., and A. Vasenev. 2017. “Infrared drones in the construction industry: Designing a protocol for building thermography procedures.” Energy Procedia 132 (Oct): 63–68. https://doi.org/10.1016/j.egypro.2017.09.636.
Esposito, M., M. Crimaldi, V. Cirillo, F. Sarghini, and A. Maggio. 2021. “Drone and sensor technology for sustainable weed management: A review.” Chem. Biol. Technol. Agric. 8 (Dec): 1–11. https://doi.org/10.1186/s40538-021-00217-8.
Floreano, D., and R. J. Wood. 2015. “Science, technology and the future of small autonomous drones.” Nature 521 (7553): 460–466. https://doi.org/10.1038/nature14542.
García Carrillo, L. R., A. E. Dzul López, R. Lozano, and C. Pégard. 2012. “Combining stereo vision and inertial navigation system for a quad-rotor UAV.” J. Intell. Rob. Syst. 65 (1–4): 373–387. https://doi.org/10.1007/s10846-011-9571-7.
Giersch, S., O. El Guernaoui, S. Raasch, M. Sauer, and M. Palomar. 2022. “Atmospheric flow simulation strategies to assess turbulent wind conditions for safe drone operations in urban environments.” J. Wind Eng. Ind. Aerodyn. 229 (Oct): 105136. https://doi.org/10.1016/j.jweia.2022.105136.
Hochreiter, S., and J. Schmidhuber. 1997. “Long short-term memory.” Neural Comput. 9 (8): 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.
Hodge, V. J., R. Hawkins, and R. Alexander. 2021. “Deep reinforcement learning for drone navigation using sensor data.” Neural Comput. Appl. 33 (6): 2015–2033. https://doi.org/10.1007/s00521-020-05097-x.
Huang, C., X. Zhou, X. Ran, J. Wang, H. Chen, and W. Deng. 2023. “Adaptive cylinder vector particle swarm optimization with differential evolution for UAV path planning.” Eng. Appl. Artif. Intell. 121 (May): 105942. https://doi.org/10.1016/j.engappai.2023.105942.
Jeong, S., K. You, and D. Seok. 2021. “Hazardous flight region prediction for a small UAV operated in an urban area using a deep neural network.” Aerosp. Sci. Technol. 118 (Nov): 107060. https://doi.org/10.1016/j.ast.2021.107060.
Jiang, Y., and Y. Bai. 2020. “Estimation of construction site elevations using drone-based orthoimagery and deep learning.” J. Constr. Eng. Manage. 146 (8): 04020086. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001869.
Kaelbling, L. P., M. L. Littman, and A. W. Moore. 1996. “Reinforcement learning: A survey.” J. Artif. Intell. Res. 4 (May): 237–285. https://doi.org/10.1613/jair.301.
Khan, M. T. R., M. Muhammad Saad, Y. Ru, J. Seo, and D. Kim. 2021. “Aspects of unmanned aerial vehicles path planning: Overview and applications.” Int. J. Commun. Syst. 34 (10): e4827. https://doi.org/10.1002/dac.4827.
Kwak, J., and Y. Sung. 2018. “Autonomous UAV flight control for GPS-based navigation.” IEEE Access 6 (Jul): 37947–37955. https://doi.org/10.1109/ACCESS.2018.2854712.
Lee, S., D. Har, and D. Kum. 2016. “Drone-assisted disaster management: Finding victims via infrared camera and LiDAR sensor fusion.” In Proc., 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE), 84–89. New York: IEEE.
Lee, T., S. Mckeever, and J. Courtney. 2021. “Flying free: A research overview of deep learning in drone navigation autonomy.” Drones 5 (2): 52. https://doi.org/10.3390/drones5020052.
Liu, D., X. Xia, J. Chen, and S. Li. 2021. “Integrating building information model and augmented reality for drone-based building inspection.” J. Comput. Civ. Eng. 35 (2): 04020073. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000958.
Liu, Y., Q. Wang, H. Hu, and Y. He. 2018. “A novel real-time moving target tracking and path planning system for a quadrotor UAV in unknown unstructured outdoor scenes.” IEEE Trans. Syst. Man Cybern.: Syst. 49 (11): 2362–2372. https://doi.org/10.1109/TSMC.2018.2808471.
Ma, H., Y. Zhang, N. Thuerey, X. Hu, and O. J. Haidn. 2021. “Physics-driven learning of the steady Navier–Stokes equations using deep convolutional neural networks.” Preprint, submitted June 17, 2021. https://arxiv.org/abs/2106.09301.
Mayer, S., L. Lischke, and P. W. Woźniak. 2019. “Drones for search and rescue.” In Proc., 1st Int. Workshop on Human-Drone Interaction. Toulouse, France: Ecole Nationale de l'Aviation Civile.
Milla-Val, J., C. Montañés, and N. Fueyo. 2024. “Economical microscale predictions of wind over complex terrain from mesoscale simulations using machine learning.” Model. Earth Syst. Environ. 10 (1): 1407–1421. https://doi.org/10.1007/s40808-023-01851-x.
Mishra, B., D. Garg, P. Narang, and V. Mishra. 2020. “Drone-surveillance for search and rescue in natural disaster.” Comput. Commun. 156 (Apr): 1–10. https://doi.org/10.1016/j.comcom.2020.03.012.
Muñoz, G., C. Barrado, E. Çetin, and E. Salami. 2019. “Deep reinforcement learning for drone delivery.” Drones 3 (3): 72. https://doi.org/10.3390/drones3030072.
Paz, C., E. Suárez, C. Gil, and J. Vence. 2021. “Assessment of the methodology for the CFD simulation of the flight of a quadcopter UAV.” J. Wind Eng. Ind. Aerodyn. 218 (Nov): 104776. https://doi.org/10.1016/j.jweia.2021.104776.
Pham, H. X., H. M. La, D. Feil-Seifer, and L. V. Nguyen. 2018. “Autonomous UAV navigation using reinforcement learning.” Preprint, submitted January 16, 2018. https://arxiv.org/abs/1801.05086.
Qu, C., F. B. Sorbelli, R. Singh, P. Calyam, and S. K. Das. 2023. “Environmentally-aware and energy-efficient multi-drone coordination and networking for disaster response.” IEEE Trans. Network Serv. Manage. 20 (2): 1093–1109. https://doi.org/10.1109/TNSM.2023.3243543.
Ragi, S., and E. K. Chong. 2013. “UAV path planning in a dynamic environment via partially observable Markov decision process.” IEEE Trans. Aerosp. Electron. Syst. 49 (4): 2397–2412. https://doi.org/10.1109/TAES.2013.6621824.
Ramezani Dooraki, A., and D.-J. Lee. 2022. “A multi-objective reinforcement learning based controller for autonomous navigation in challenging environments.” Machines 10 (7): 500. https://doi.org/10.3390/machines10070500.
Restas, A. 2015. “Drone applications for supporting disaster management.” World J. Eng. Technol. 3 (3): 316. https://doi.org/10.4236/wjet.2015.33C047.
Ribeiro, M. D., A. Rehman, S. Ahmed, and A. Dengel. 2020. “DeepCFD: Efficient steady-state laminar flow approximation with deep convolutional neural networks.” Preprint, submitted April 19, 2020. https://arxiv.org/abs/2004.08826.
Rohman, B. P., M. B. Andra, H. F. Putra, D. H. Fandiantoro, and M. Nishimoto. 2019. “Multisensory surveillance drone for survivor detection and geolocalization in complex post-disaster environment.” In Proc., IGARSS 2019–2019 IEEE Int. Geoscience and Remote Sensing Symp., 9368–9371. New York: IEEE.
Sani, M. F., and G. Karimian. 2017. “Automatic navigation and landing of an indoor AR. Drone quadrotor using ArUco marker and inertial sensors.” In Proc., 2017 Int. Conf. on Computer and Drone Applications (IConDA), 102–107. New York: IEEE.
Schulman, J., F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. 2017. “Proximal policy optimization algorithms.” Preprint, submitted July 20, 2017. https://arxiv.org/abs/1707.06347.
Shahoud, A., D. Shashev, and S. Shidlovskiy. 2022. “Visual navigation and path tracking using street geometry information for image alignment and servoing.” Drones 6 (5): 107. https://doi.org/10.3390/drones6050107.
Shantia, A., R. Timmers, Y. Chong, C. Kuiper, F. Bidoia, L. Schomaker, and M. Wiering. 2021. “Two-stage visual navigation by deep neural networks and multi-goal reinforcement learning.” Rob. Auton. Syst. 138 (Apr): 103731. https://doi.org/10.1016/j.robot.2021.103731.
Shavarani, S. M., M. G. Nejad, F. Rismanchian, and G. Izbirak. 2018. “Application of hierarchical facility location problem for optimization of a drone delivery system: A case study of Amazon prime air in the city of San Francisco.” Int. J. Adv. Manuf. Technol. 95 (Apr): 3141–3153. https://doi.org/10.1007/s00170-017-1363-1.
Suleiman, A., Z. Zhang, L. Carlone, S. Karaman, and V. Sze. 2019. “Navion: A 2-mw fully integrated real-time visual-inertial odometry accelerator for autonomous navigation of nano drones.” IEEE J. Solid-State Circuits 54 (4): 1106–1119. https://doi.org/10.1109/JSSC.2018.2886342.
Sun, Q., M. Li, T. Wang, and C. Zhao. 2018. “UAV path planning based on improved rapidly-exploring random tree.” In Proc., 2018 Chinese Control and Decision Conf. (CCDC), 6420–6424. New York: IEEE.
Vanegas, F., K. J. Gaston, J. Roberts, and F. Gonzalez. 2019. “A framework for UAV navigation and exploration in GPS-denied environments.” In Proc., 2019 IEEE Aerospace Conf., 1–6. New York: IEEE.
von Stumberg, L., V. Usenko, J. Engel, J. Stückler, and D. Cremers. 2017. “From monocular SLAM to autonomous drone exploration.” In Proc., 2017 European Conf. on Mobile Robots (ECMR), 1–8. New York: IEEE.
Vuppala, R. K., and K. Kara. 2021. “A novel approach in realistic wind data generation for the safe operation of small unmanned aerial systems in urban environment.” In Proc., AIAA Aviation 2021 Forum, 2505. Reston, VA: American Institute of Aeronautics and Astronautics.
Wang, Z., Y. Wu, and Q. Niu. 2019. “Multi-sensor fusion in automated driving: A survey.” IEEE Access 8 (Dec): 2847–2868. https://doi.org/10.1109/ACCESS.2019.2962554.
Wartman, J., J. W. Berman, A. Bostrom, S. Miles, M. Olsen, K. Gurley, J. Irish, L. Lowes, T. Tanner, and J. Dafni. 2020. “Research needs, challenges, and strategic approaches for natural hazards and disaster reconnaissance.” Front. Built Environ. 6 (Nov): 573068. https://doi.org/10.3389/fbuil.2020.573068.
Weng, Y., and S. G. Paal. 2023. “Physics-informed few-shot learning for wind pressure prediction of low-rise buildings.” Adv. Eng. Inf. 56 (Apr): 102000. https://doi.org/10.1016/j.aei.2023.102000.
Yousef, M., F. Iqbal, and M. Hussain. 2020. “Drone forensics: A detailed analysis of emerging DJI models.” In Proc., 2020 11th Int. Conf. on Information and Communication Systems (ICICS), 66–71. New York: IEEE.
Zhang, M., M. Zhang, Y. Chen, and M. Li. 2021. “IMU data processing for inertial aided navigation: A recurrent neural network based approach.” In Proc., 2021 IEEE Int. Conf. on Robotics and Automation (ICRA), 3992–3998. New York: IEEE.
Zhang, S., S. M. Bogus, C. D. Lippitt, V. Kamat, and S. Lee. 2022. “Implementing remote-sensing methodologies for construction research: An unoccupied airborne system perspective.” J. Constr. Eng. Manage. 148 (9): 03122005. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002347.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 7 • July 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Nov 17, 2023
Accepted: Feb 9, 2024
Published online: Apr 30, 2024
Published in print: Jul 1, 2024
Discussion open until: Sep 30, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.