Reinforcement Learning–Based Adaptive Attitude Control Method for a Class of Hypersonic Flight Vehicles Subject to Nonaffine Structure and Unmatched Disturbances
Publication: Journal of Aerospace Engineering
Volume 37, Issue 2
Abstract
This paper proposes a reinforcement learning–based adaptive attitude control (RLAC) method for a class of hypersonic flight vehicles (HFVs) output constrained nonaffine attitude control problems subject to unmatched disturbances. First, by considering the strong coupling of HFVs attitude dynamics, the uncertainty of aerodynamic parameters and the complexity of the flight environment, a second-order multivariable nonaffine nonlinear control system is obtained. Then, by introducing specific nonlinear function and coordinate transformation techniques, the output constrained nonaffine control problem is transformed into a stabilization problem of several new variables. Moreover, dual actor-critic networks and their adaptive weight update laws are designed to cope with unknown unmatched and matched structural uncertainties. Meanwhile, two super-twisting disturbance observers integrated with dual actor-critic networks are designed to compensate unknown unmatched and matched external disturbances. With the help of the Lyapunov direct method, output constraint, convergence of the estimated weights, and stability of the system are proved. Finally, the validity as well as superiority of the proposed method are verified by numerical simulations.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was supported by the Foundation of National Key Laboratory of Science and Technology on Test Physics and Numerical Mathematics, China.
References
Amin, A. A., and K. M. Hasan. 2019. “A review of fault tolerant control systems: Advancements and applications.” Measurement 143 (Sep): 58–68. https://doi.org/10.1016/j.measurement.2019.04.083.
An, H., C. Wang, and B. Fidan. 2017. “Sliding mode disturbance observer-enhanced adaptive control for the air-breathing hypersonic flight vehicle.” Acta Astronaut. 139 (Oct): 111–121. https://doi.org/10.1016/j.actaastro.2017.06.026.
Bu, X., C. Hua, M. Lv, and Z. Wu. 2023. “Flight control of waverider vehicles with fragility-avoidance prescribed performance.” IEEE Trans. Aerosp. Electron. Syst. 59 (5): 5248–5261. https://doi.org/10.1109/TAES.2023.3251314.
Bu, X., B. Jiang, and H. Lei. 2022a. “Nonfragile quantitative prescribed performance control of waverider vehicles with actuator saturation.” IEEE Trans. Aerosp. Electron. Syst. 58 (4): 3538–3548. https://doi.org/10.1109/TAES.2022.3153429.
Bu, X., B. Jiang, and H. Lei. 2022b. “Performance guaranteed finite-time non-affine control of waverider vehicles without function-approximation.” IEEE Trans. Intell. Transp. Syst. 24 (3): 3252–3262. https://doi.org/10.1109/TITS.2022.3224424.
Bu, X., X. Wu, G. He, and J. Huang. 2016. “Novel adaptive neural control design for a constrained flexible air-breathing hypersonic vehicle based on actuator compensation.” Acta Astronaut. 120 (Mar–Apr): 75–86. https://doi.org/10.1016/j.actaastro.2015.12.004.
Chung, S. C.-Y., and C.-L. Lin. 1999. “A transformed lure problem for sliding mode control and chattering reduction.” IEEE Trans. Autom. Control 44 (3): 563–568. https://doi.org/10.1109/9.751351.
Dong, C., Y. Hou, Y. Zhang, and Q. Wang. 2010. “Model reference adaptive switching control of a linearized hypersonic flight vehicle model with actuator saturation.” Proc. Inst. Mech. Eng. Part I: J. Syst. Control Eng. 224 (3): 289–303. https://doi.org/10.1243/09596518JSCE829.
Freidovich, L. B., and H. K. Khalil. 2008. “Performance recovery of feedback-linearization-based designs.” IEEE Trans. Autom. Control 53 (10): 2324–2334. https://doi.org/10.1109/TAC.2008.2006821.
Gao, H., and Y. Cai. 2016. “Nonlinear disturbance observer-based model predictive control for a generic hypersonic vehicle.” Proc. Inst. Mech. Eng., Part I: J. Syst. Control Eng. 230 (1): 3–12. https://doi.org/10.1177/0959651815605664.
Giap, V.-N., and S.-C. Huang. 2020. “Effectiveness of fuzzy sliding mode control boundary layer based on uncertainty and disturbance compensator on suspension active magnetic bearing system.” Meas. Control 53 (5–6): 934–942. https://doi.org/10.1177/0020294020905044.
Guo, Z., J. Zhou, J. Guo, J. Cieslak, and J. Chang. 2017. “Coupling-characterization-based robust attitude control scheme for hypersonic vehicles.” IEEE Trans. Ind. Electron. 64 (8): 6350–6361. https://doi.org/10.1109/TIE.2017.2682031.
He, W., and Y. Dong. 2017. “Adaptive fuzzy neural network control for a constrained robot using impedance learning.” IEEE Trans. Neural Networks Learn. Syst. 29 (4): 1174–1186. https://doi.org/10.1109/TNNLS.2017.2665581.
He, W., S. S. Ge, Y. Li, E. Chew, and Y. S. Ng. 2015. “Neural network control of a rehabilitation robot by state and output feedback.” J. Intell. Robotic Syst. 80 (1): 15–31. https://doi.org/10.1007/s10846-014-0150-6.
Hou, Z., and S. Xiong. 2019. “On model-free adaptive control and its stability analysis.” IEEE Trans. Autom. Control 64 (11): 4555–4569. https://doi.org/10.1109/TAC.2019.2894586.
Kai, X., D. Yanli, Z. Peng, and L. Haibing. 2019. “Adaptive backstepping control for hypersonic vehicles with actuator amplitude and rate saturation.” Trans. Nanjing Univ. Aeronaut. Astronaut. 36 (2): 242–252.
Li, A., S. Liu, and X. Hu. 2023. “Fault-tolerant attitude control for hypersonic flight vehicle subject to actuators constrain: A model predictive static programming approach.” IEEE J. Miniaturization Air Space Syst. 4 (2): 136–145. https://doi.org/10.1109/JMASS.2023.3241566.
Liu, C., C. Dong, Z. Zhou, and Z. Wang. 2020. “Barrier lyapunov function based reinforcement learning control for air-breathing hypersonic vehicle with variable geometry inlet.” Aerosp. Sci. Technol. 96 (Jan): 105537. https://doi.org/10.1016/j.ast.2019.105537.
Liu, H., W. Bao, H. Li, and Y. Liao. 2019. “Multivariable disturbance observer–based fuzzy fast terminal sliding mode attitude control for a hypersonic vehicle.” J. Aerosp. Eng. 32 (2): 04018152. https://doi.org/10.1061/(ASCE)AS.1943-5525.0000991.
Liu, Y., H. Wang, T. Wu, Y. Lun, J. Fan, and J. Wu. 2022. “Attitude control for hypersonic reentry vehicles: An efficient deep reinforcement learning method.” Appl. Soft Comput. 123 (Jul): 108865. https://doi.org/10.1016/j.asoc.2022.108865.
Lu, Y., Z. Jia, X. Liu, and K. Lu. 2022. “Output feedback fault-tolerant control for hypersonic flight vehicles with non-affine actuator faults.” Acta Astronaut. 193 (Apr): 324–337. https://doi.org/10.1016/j.actaastro.2022.01.023.
Lu, Y., X. Liu, and Z. Jia. 2021. “An ESO-based attitude control method for non-affine hypersonic flight vehicles.” In Proc., 2021 40th Chinese Control Conf. (CCC), 7736–7741. New York: IEEE.
Ma, Y.-S., W.-W. Che, C. Deng, and Z.-G. Wu. 2023. “Distributed model-free adaptive control for learning nonlinear mass under dos attacks.” IEEE Trans. Neural Networks Learn. Syst. 34 (3): 1146–1155. https://doi.org/10.1109/TNNLS.2021.3104978.
Nagesh, I., and C. Edwards. 2014. “A multivariable super-twisting sliding mode approach.” Automatica 50 (3): 984–988. https://doi.org/10.1016/j.automatica.2013.12.032.
Nie, Y., Y. Zhang, W. Bai, Y. Cao, and C. Yu. 2023. “Reinforcement learning control for hypersonic morphing flight vehicle with identification of dynamic parameter.” In Proc., 2022 Int. Conf. on Guidance, Navigation and Control: Advances in Guidance, Navigation and Control, 2564–2574. Berlin: Springer.
Ouyang, Y., L. Dong, Y. Wei, and C. Sun. 2020. “Neural network based tracking control for an elastic joint robot with input constraint via actor-critic design.” Neurocomputing 409 (Oct): 286–295. https://doi.org/10.1016/j.neucom.2020.05.067.
Speier, R. H., G. Nacouzi, C. Lee, and R. M. Moore. 2017. Hypersonic missile nonproliferation: Hindering the spread of a new class of weapons. Berlin: Rand.
Tan, S., H. Lei, and P. Wang. 2018. “Constrained adaptive neural control for air-breathing hypersonic vehicles without backstepping.” Math. Problems Eng. 2018 (Aug): 1–13. https://doi.org/10.1155/2018/6968526.
Ur Rehman, O., I. R. Petersen, and B. Fidan. 2013. “Feedback linearization-based robust nonlinear control design for hypersonic flight vehicles.” Proc. Inst. Mech. Eng., Part I: J. Syst. Control Eng. 227 (1): 3–11. https://doi.org/10.1177/0959651812447722.
Wang, P., C. Luo, and J. Xu. 2021. “Research on the development and defense strategy for hypersonic missiles.” Tactical Missile Technol. 2021 (1): 60–66. https://doi.org/10.16358/j.issn.1009-1300.2021.1.148.
Wang, Y., M. Chen, Q. Wu, and J. Zhang. 2018. “Fuzzy adaptive non-affine attitude tracking control for a generic hypersonic flight vehicle.” Aerosp. Sci. Technol. 80 (Sep): 56–66. https://doi.org/10.1016/j.ast.2018.06.033.
Wang, Y., C. Jiang, and Q. Wu. 2013. “Attitude tracking control for variable structure near space vehicles based on switched nonlinear systems.” Chin. J. Aeronaut. 26 (1): 186–193. https://doi.org/10.1016/j.cja.2012.12.009.
Wang, Y., and Q. Wu. 2017. “Adaptive non-affine control for the short-period model of a generic hypersonic flight vehicle.” Aerosp. Sci. Technol. 66 (Jul): 193–202. https://doi.org/10.1016/j.ast.2017.03.005.
Wenfeng, M., W. Peng, and T. Guojian. 2017. “Adaptive control for hypersonic vehicle based on error characteristic model.” In Proc., 2017 36th Chinese Control Conf. (CCC), 3496–3503. New York: IEEE.
Wu, T., H. Wang, Y. Yu, Y. Liu, and J. Wu. 2021. “Quantized fixed-time fault-tolerant attitude control for hypersonic reentry vehicles.” Appl. Math. Modell. 98 (Oct): 143–160. https://doi.org/10.1016/j.apm.2021.04.033.
Wu, Y., A. Isidori, R. Lu, and H. K. Khalil. 2019. “Performance recovery of dynamic feedback-linearization methods for multivariable nonlinear systems.” IEEE Trans. Autom. Control 65 (4): 1365–1380. https://doi.org/10.1109/TAC.2019.2924176.
Xingling, S., and W. Honglun. 2014. “Sliding mode based trajectory linearization control for hypersonic reentry vehicle via extended disturbance observer.” ISA Trans. 53 (6): 1771–1786. https://doi.org/10.1016/j.isatra.2014.09.021.
Yu, Y., H. Wang, and N. Li. 2019. “Fault-tolerant control for over-actuated hypersonic reentry vehicle subject to multiple disturbances and actuator faults.” Aerosp. Sci. Technol. 87 (Apr): 230–243.
Zhao, S., J. Wang, H. Xu, and B. Wang. 2022. “Composite observer-based optimal attitude-tracking control with reinforcement learning for hypersonic vehicles.” IEEE Trans. Cybern. 53 (2): 913–926. https://doi.org/10.1109/TCYB.2022.3192871.
Zheng, C., Z. Gong, X. Wu, T. Dragičević, J. Rodriguez, and F. Blaabjerg. 2021. “Finite-set quasi-sliding mode predictive control of $ LC $-filtered voltage source inverters.” IEEE Trans. Ind. Electron. 69 (12): 11968–11978. https://doi.org/10.1109/TIE.2021.3135605.
Information & Authors
Information
Published In
Journal of Aerospace Engineering
Volume 37 • Issue 2 • March 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Dec 6, 2022
Accepted: Oct 24, 2023
Published online: Jan 11, 2024
Published in print: Mar 1, 2024
Discussion open until: Jun 11, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.