Analysis of Pilot Interaction with the Control Adapting System for UAV

The aircraft ailerons and flaps malfunction is a problem that may happen and occur in all aerial platforms, which is a situation when the aileron/flap blocks appears in various types of aircraft: civil/military/unmanned. The consequences of the failure and the sudden loss of the maneuverability can lead to a serious crash. Thus, overcoming this problem was a subject of research (Żugaj 2017). Similar systems that constantly monitor and react to various types of malfunctions are strongly developed around the world (Abdi and Labib 2003; Chen and Zhang 2012; Narkiewicz et al. 2017).

The reconfigurable system helps to retrieve the controllability of the aircraft if some of the control surfaces get stuck in a certain position. The reconfigurable system detects the control surfaces failure and adapts the available control surfaces, restoring the maneuverability of the plane (Żugaj 2017). Typically, aircraft control surfaces such as elevators, ailerons, and flaps work in pairs and are located on the opposite sides of the longitudinal plane of symmetry of an aircraft (Fig. 1) (Żugaj et al. 2016). The description and analysis of the reconfiguration system itself was not the subject of this paper. However, the detailed information regarding the principle of work may be found in Naskar et al. (2015) and Yang et al. (2012).

The main advantage of the reconfigurable system is that during a random failure event, the system restores maneuverability of the UAV as much as the algorithms will allow (Burcham et al. 1997; Żugaj 2017). However, such a solution impacts on aircraft’s dynamics significantly. Theoretically, in case of failure, the system restores the maneuverability, so the operator’s interaction should be similar, and the aircraft should be controllable as before the failure. However, the drawback of the system is that the aircraft’s dynamics change while the system is active, which may cause some difficulties for the operator. Thus, in some cases, the system may not be fully supportive for the operator considering the adaptation to the new aircraft configuration. The multiple simulation confirmed that the system is a helpful tool, but it was necessary to investigate the system not only in virtual simulations but with a group of experienced operators. In this paper, the experiment with 10 UAV operators was evaluated to assess the reconfigurable system effectiveness. The aim of the experiment was to provide clues and data, so the system could be developed more. The goal of the experiment was to register the operators’ task evaluation during the set of failures. In the experiment, both the subjective and objective assessment methods were analyzed. The aim of the investigate reconfiguration algorithm was to take advantage of the working part of the control system in the case of partial system failure to ensure the aircraft controllability. The reconfiguration was performed by a software algorithm designed to use the remaining control surfaces to compensate for the failure effects and modify the strategy of control surfaces handling.

The Fig. 2 presents the reconfiguration method. The input signals ${\mathbf{\delta }}_{\mathbf{0}}$ vector comes from the operator and contains demanded signals for the ailerons, the elevators, and the rudder in fault-free conditions. The reconfiguration algorithm distributes these signals to all working control surfaces ${\mathbf{\delta }}_{\mathbf{f}}$ in the case of a control system failure. The working control surface deflections are calculated to obtain in an ideal case the same control loads as in an undamaged system based on the failure data $\mathbf{\upsilon }$.

An experiment took place at the Department of Automation and Aeronautical Systems (DAAS) at the Warsaw University of Technology at Simulators Laboratory. For the experimental tests, a group of 10 UAV operators (male) were selected. The average subject age was 26.6 years. All of the subjects had advanced experience in flight simulator games, and most of them had experience with flying the real UAV. Seven operators had a basic unmanned aerial vehicle operator (UAVO) qualification license, and three operators had experience in piloting the various UAVs. It is worth mentioning that the aim of the experiment was not to assess the particular UAV model and its handling qualities, but to understand and give feedback about the reconfiguration system and, most importantly, how the system impacts the UAV handling qualities. Also, the individual subjects’ skills in controlling an UAV were not assessed. Every measurement was done individually, and the results were not compared between the subjects.

The hardware used for experiment was a UAV flight simulator developed at DAAS. The UAV dynamic model was identified and implemented into the virtual environment (Żugaj 2017). A model was identified and verified so it could be used in further experiments (Lichota et al. 2017). An operator used an onboard view mode on a 52-in. screen. On a second screen, the operator could see all the important information regarding the UAV status.

The joystick used in experiment was a Saitec X52 Pro (Logitech, Lausanne, Switzerland). The maneuver task used in the experiment was a slalom trajectory made of two turns. The slalom maneuver is one of the tasks defined as an element often used for assessing the handling qualities of aircraft [ADS-33D-PRF (ADS 1996)]. The tracking task is a maneuver convenient for later evaluation (Gittleman et al. 1992; Ito and Ito 1975; Takashima et al. 1980). The slalom parameters were adopted to the maneuverability of the particular UAV, so the task completion was possible with the set criteria (Freeman et al. 2000; Kopyt et al. 2017). The slalom trajectory was generated in a 3ds MAX (Autodesk, San Rafael, California) environment as a pipe with additional rings on it. The rings were added to help the operator follow the desired trajectory. An open architecture of the simulator system allowed implementation of the additional elements into the virtual environment. The slalom was added to existing scenery (Fig. 3). Since the trajectory was not treated as an obstacle, the intersections with the additional elements were turned off and there was no possibility to crash by hitting the slalom elements.

The assessment of the reconfiguration system was divided into two sections: subjective and objective. The subjective method contained three various questionnaires. The idea was to assess the reconfigurable system from two aspects: the aircraft dynamics change and the operator’s workload. To assess the aircraft handling qualities, a Cooper-Harper (C-H) scale was used (Cooper and Harper 1969). The C-H questionnaire is a 10-point scale with questions that help identify the proper aircraft handling behavior. The 10 refers to the worst handling qualities, and 1 refers to the best. The second questionnaire used was a Bedford scale (Roscoe and Ellis 1990), which is based on the C-H scale but covers the effort and workload during the tests. Finally, the Overall Workload (OW) scale was used (scale from 0 to 100) (Hwa and Hyung-Shik 2001). The C-H scale is more focused on the handling qualities of UAV, and the Bedford and OW are focused only on the operator’s workload. The UAV operator filled out the questionnaires after each task was completed. Some of the subjects have flown the real model of the aircraft, and they suggested that the dynamics differ from the reality. This was why, as the instructor strongly suggested, the point of the assessment did not cover the handling quality of UAV itself but its changes coming from reconfiguration system. That statement had to be underlined before all subjects performed the experiments. The trajectory of the UAV during the slalom was registered automatically in the system. The criteria for the objective assessment was as follows:

All information such as speed, altitude, position, and so on were displayed on the second screen. Based on these data, it was possible to define how close to the original trajectory each subject flew. Having the most important four parameters ($x$, $y$, $z$, and $V$), the operator’s performance was expressed with the error squared integral—a criterion commonly used in automation (Mazurek et al. 2002). For each parameter, the error was calculated comparing the values to the references, which were precalculated during the trajectory generation (Kopyt 2015). First, the error itself was calculated for each parameter (1–3), and finally, the $J$ parameter was obtained (4–6). The $J$ parameter provides information on how far the realized trajectory was from the original parameter. The $J$ parameter was calculated for three parameters: $y$ deviation along $x$ axis, altitude deviation, and forward velocity deviationwhere the original trajectory is derived from the equation describing the slalom trajectory. The original trajectory (in the $x$ and $y$ plane) is made of enter-straight line, cosine wave, and exit-straight line (Fig. 4).

The test scenario was made of 11 short flight runs. Each of the flights included an identical maneuver task—a slalom. The procedure of the experiments started with the getting acquainted with the UAV model. The first few minutes were intended for an operator to do a free run and feel the aircraft dynamics (5–10 min). Next, the operator did a reference run during which no failure occurred. This run was treated as a reference result for further analysis. Consequently, the operator evaluated a set of five different failures cases with the system on and off. The list of flight configurations is presented in Table 1.

Where shortcuts correspond to

Since one of the most common failures is when the mechanism failure leads to the control surface getting stuck so it cannot be moved in any direction, the following failures have been selected as crucial for the system. For the purpose of the experiment, the failure was defined as the selected control surface is blocked on a desired angle, and so

The failure scenarios were divided into two groups. The first three failures, simple failures, (AIL_R, AIL_R+RUD, AIL_R+RUD+ELEV_L) are combinations of single failures that get more and more difficult for both an operator to handle and an algorithm to reconfigure the surfaces, since there are less elements to affect. The second group of failures is when the full (left and right) control channel is not working: ELEV*2 and AIL*2. The last to configurations are definitely more difficult to handle. The full elevator failure makes aircraft especially uncontrollable, thus the group name is total failure.

The dataflow from 10 subjects resulted in collecting the following:

The analysis of the results was done in two steps. First approach was to analyze the subjective operators’ assessment using questionnaires, and second, the objective assessment based on the flight parameters.

The evaluated experiment confirmed the hypothesis that the reconfigurable system is a desired system for UAV. The results from the experiment showed that for some configurations the system is highly desirable, especially when the one control surface is failed (cases 4 and 5). The best example was during the elevator failure, when the UAV was completely unable to control, but with the system on, the aircraft has the pitch maneuverability restored. From the subjective analysis, there were no doubts that reconfigurable system is a helpful feature of the UAV. Those results were confirmed by the objective results where the performance of the task evaluation was calculated. Those results confirmed the subjective analysis providing the results on a 75% effectiveness. The experiment however pointed out one significant system drawback. Regarding first three cases (single failure), the dynamic of the UAV was strongly changed. In some cases, the operators indicated that it was easier for them to fly with the single failure rather than with reconfiguration system on. Such a conclusion might be overcome with applying new algorithms that will provide similar dynamics to the original configuration. Also, such an impression might be an effect that operators simply did not have enough time to understand and learn the UAV with the system on. Proper training and simulator tests would certainly increase the operator’s efficiency.

The paper was prepared within project MYSTERY—Synthesis methods of aircraft control system, taking into account the situation of risk, funded by National Center for Research and Development under Grant Agreement No. PBS2/B6/19/2013.