A six-degree-of-freedom proportional-derivative control strategy for bumblebee flight stabilization

Flying insects perform active flight control with flapping wings by continuously adjusting their wing kinematics in stabilizing the body posture to stay aloft under complex natural environment. While the Proportional Derivative (PD) / Proportional Integral Derivative (PID)-based algorithms have been applied to examine specific single degree of freedom (DoF) and/or 3 DoF flight control associated with insect flights, a full 6 DoF flight control strategy remains yet poorly studied. Here we propose a novel 6 DoF PD controller specified for flight stabilization in flapping flights, in which proportional and derivative gains are optimized to facilitate a fast while precise flight control by combing Laplace transformation and root locus method. The vertical position, yaw, pitch and roll are directly stabilized by tuning the wing kinematics while the forward/backward position and lateral position are indirectly stabilized by controlling the pitch and roll, respectively. Coupled with a recently developed flight dynamic model informed by high-fidelity CFD simulation (Cai et al. 2021), this methodology is proven to be effective as a versatile and efficient tool to achieve fast flight stabilization under both small and large perturbations for bumblebee hovering. The 6 DoF PD flight control strategy proposed may provide a useful bioinspired flight-controller design for flapping-wing micro air vehicles (FWMAVs).


Nomenclature
• quantity expressed in ground frame

Introduction
Flapping-wing insects achieve excellent flight control and maneuverability by continuously adjusting their wing kinematics under various unsteady environments. The unsteady external environment challenges the insects to maintain stable flight (Ravi et al. 2016), let alone the maneuvers evading from predators or stimuli (Muijres et al. 2014) and tracking interested targets (Matthews et al. 2018;Zhang et al. 2019). Moreover, the small body inertia renders the insects susceptible even to gentle unsteady currents. While it has been pointed out that flapping wing insects are dynamically unstable (Taylor et al. 2003;Sun et al. 2005;Sun et al. 2007;Xiong et al. 2008), under the unfavourable conditions, insects enable achievements of high flight stability and maneuverability.
It is reported that insects are capable of correcting the flight deviations under disturbances within a very few wing beats (Ristroph et al. 2010;Cheng et al. 2011;Beatus et al. 2015;Whitehead et al. 2015), which outperforms any manmade air vehicles. Both vision-based and mechanical sensory systems are utilized in flight stabilization: the visual system like compound eyes or ocelli can sense the position and attitude; the highly specialized organs such as the halteres (Deora et al. 2017) and the antennas (Sane et al. 2007) normally serve as vibrating structure gyroscopes capable of sensing body rotations, of which the sensory neurons are directly wired to the motor neurons (Fayyazuddin et al. 1996), guaranteeing extremely low latency and fast control. Conventional studies (Ristroph et al. 2010;Cheng et al. 2011;Beatus et al. 2015;Whitehead et al. 2015) suggest that the proportional-derivative (PD) control strategy may be widely adopted by flying insects, where a proportional feedback is made for attitudes (roll, pitch and yaw) and positions (forward/backward, lateral and vertical) of insect body while a derivative feedback is for the time derivatives of their attitudes and positions. The flight control system of flying insects is observed to be inherently intermittent as recently pointed out by Xu et al (Xu et al. 2020), however, a continuous PD controller can reasonably capture the behaviors in responds to perturbations (Ristroph et al. 2010;Beatus et al. 2015;Whitehead et al. 2015) and the PD control strategy has been reported to work well for the flapping-wing flight control in insects (Yao et al. 2020;Cai et al. 2021).
Thus, flight stabilization can be achieved with some PD controller through adjusting the control parameters including both proportional and derivative gains so as to avoid the overdamped process or large overshoot oscillation and achieving a trimmed flight. Observation of the flight stabilization in insects indicates that PD flight strategies successfully developed can not only overcome the inherent dynamical instability but also achieve rapid stabilization through optimizing the control parameters. Determination of the correlations between wing kinematics and production of aerodynamic forces and torques (Hedrick et al. 2006;Yao et al. 2019) enables the development of insect-inspired flight controllers Yao et al. 2019;Zhang et al. 2019). Zhang et al. (Zhang et al. 2019) performed a CFD-based modeling to determine the feasible control parameters through a fitting methodology. Yao & Yeo (Yao et al. 2019) applied a simplified aerodynamic model to derive the control parameters. However, how the control parameters are adjusted while optimized to achieve a flight control capable of fastest restoring remains yet poorly studied. To work out the optimized control parameters in a fast while precise way, Cai et al. (Cai et al. 2021) has recently successfully developed a novel method for the control parameter optimization by combing Laplace transformation and root locus method, which is established on the basis of a simplified flight dynamic model informed by high-fidelity CFD simulation, and thus guarantee the direct implementation of the control parameters to a CFD-based flight environment. This methodology provides a versatile and efficient tool to achieve fast and precise aerodynamical prediction for flying insects in various flight behaviors. � , left wing-fixed frame � , , � , body-fixed frame � , , � and ground frame � , , �. (b) A sketch of a single wing (either right wing or left wing) with the definitions of the stroke plane frame � , , �, stroke plane angle , positional angle , elevation angle and feathering angle . ( ) represents the velocity of a blade element on the wing with the distance to the wing base and represents the corresponding angle of attack. (c) A sketch of the body with the definitions of the yaw angle , the pitch angle and the roll angle .
represents the velocity of the body and represents the corresponding angle of attack.
It is worth noting that the disturbances in nature can induces the deviations along all the six degree of freedoms (rotation and translation). However, up to now, the PD-based algorithms have been only applied to examine specific single degree of freedom (DoF) and/or 3 DoF flight control associated with insect flights (Zhang et al. 2018;Cai et al. 2021), a full 6 DoF flight control strategy remains yet poorly studied. In addition, biological evidence (Fayyazuddin et al. 1996;Ristroph et al. 2010;Beatus et al. 2015;Whitehead et al. 2015) reveals the implementation of PD control strategy to attitudes control, while whether such strategy is applicable to 3 DOF position control remains unclear. In this study, we make an extension to study the control of flight stabilization in bumblebee by further proposing a full 6 DoF control strategy. Based on the observation that (Windsor et al. 2014) that the forward/backward and lateral motions are highly correlated with pitch and roll angle, respectively, here we propose a hierarchical 6 DoF PD controller accounting for time latency, of which the vertical, yaw, pitch and roll motion are simultaneously controlled by tuning the wing motion while the forward/backward and lateral motions are controlled by control the pitch and roll motion. Besides of mild perturbations, the PD controller is also validated against large perturbations, which insects usually experience from nature (Ristroph et al. 2010).

Methods
In this section, we first give a brief introduction to the morphology of the modelled bumblebee and the definitions of the coordinate systems, which is adopted from a prior literature (Kolomenskiy et al. 2019). Then we build up an analytical framework for the design of PD control strategy with Laplace transformation and root locus method based on the dynamic response of bumblebee to unsteady flapping aerodynamics. A dynamic simulator for free flapping wing system is developed to simulate the six-degree-of-freedom flight dynamics of the bumblebee model. The CFD data-driven aerodynamic model (CDAM) capable of fast and precise aerodynamical predictions is adopted to estimate the unsteady aerodynamic forces and torques in flapping flights (Cai et al. 2021).

Morphology and coordinate system of a bumblebee
For a bumblebee flying in air with the wings flapping back and forth relative to the body, of which the geometry is measured by Kolomenskiy et al (Kolomenskiy et al. 2019), several coordinate systems are defined such as: a ground frame, a body fixed frame, a right wing-fixed frame, a left wing-fixed frame and a corresponding stroke plane frame, as depicted in figure 1. The flapping wing is assumed to be flat and rigid, rotating around the wing base with respect to the stroke plane. Thus, the wing kinematics can be defined by three basic motions of positional, elevation and feathering angles as plotted in figure 2. The angle is the stroke plane angle with respect to the horizontal plane. The "rotation axis" in our terminology is the longitudinal axis of the wing. The positional angle is measured between the lateral direction and the projection of the rotation axis on the stroke plane; the elevation angle is the deviation angle of the rotation axis from the stroke plane; the feathering angle is the angular displacement of the wing about the rotation axis. The moving body is also assumed to be rigid, which can be defined as the body angles called yaw, pitch and roll angles. The yaw angle measures the rotation of the body around vertical axis; the pitch angle is the inclination of the body between the horizontal plane; the roll angle is the angular displacement of the body about its longitudinal axis. The body angle and stroke plane angle in hovering are 4 and 0, respectively. Detailed morphology parameters are listed in Nomenclature.

CFD data-driven aerodynamic model
Here, the CFD data-driven aerodynamic model (CDAM) is adopted to predict the aerodynamic forces, torques and power consumption of a flying bumblebee (Cai et al. 2021), which consists of two sub-models: a CFD-informed quasi steady model for flapping wings and a simplified aerodynamic model for a moving body. The flapping wing model is capable of accounting for various flight velocities and wing kinematics, which is based on a blade element method. The wings are assumed to be flat and rigid, which are divided into small elements. The quasi-steady model is applied on each element to obtain the aerodynamic force, torque and power which are consequently summed up over the entire wing. The simplified aerodynamic model for the moving body based on the quasi-steady approximation can predict the lift force , and the drag force , . More details can be found in (Cai et al. 2021). Note that the CFD-based database ensures a fast while highly accurate prediction of flapping aerodynamics with the CDAM method served as an alternative to the expensive direct numerical simulations.

Flight dynamic simulator
Insects with paired wings are usually assumed as a rigid body (Sun et al. 2005;Gao et al. 2009;Gao et al. 2011), of which the inertial force induced by reciprocating wings is neglected. This assumption is reasonable when the wing beat frequency is high or the wing is light (Sun et al. 2007;Taha et al. 2020). In addition, many dynamic equations of flapping-wing insects are only applicable to small deviations from equilibrium state (Gao et al. 2011;Taha et al. 2015;Zhang et al. 2018) by neglecting high-order terms, which is not suitable for extreme motions under large perturbations. In order to derive the full dynamic equations applicable to arbitrary pair-winged systems like flapping-wing insects and FWMAVs that take the effects of reciprocating wings into account, here we propose a dynamic simulator extended from prior work (Gebert et al. 2002;Sun et al. 2007), which is capable of simulating the interactions between insect body and insect wings. The governing dynamic equations of the insect body can be given as, where expressions of the coefficients 1 , 1 , 1 , 1 , 1 , 1 , 2 , 2 , 2 , 2 , 2 , 2 , 2 , 2 , 2 and 2 are summarized in Appendix.
are the aerodynamic forces and moments acting on the body, right wing and left wing, respectively, which can be calculated through the CDAM, therefore, the dynamic simulator accounts for the fluid-structure interaction (FSI). The dynamic equations of right and left wings can be further defined as, where expressions of the coefficients , , , , , , and are also summarized in Appendix.
2 and 2 are the torques due to the interactions between the body and two wings, which determine the wing motion. To achieve the desired wing motion determined by the control system, which is a PD controller in this study, 2 and 2 need to be precisely tuned. To link the 2 and 2 and the known wing kinematics to control the flight of the bumblebee, we need to further add the following equations: where the second derivatives of flapping angles ̈, ̈, ̈, ̈, ̈ and ̈ are the wing kinematics inputs and the motion of the bumblebee can be solved by integrating equations (2.3.1) ~ (2.3.4).

6-DOF PD controller design
As shown in figure 3(a) and (b), pitch down/up can tilt the force forward/backward and roll can tilt the force sideward. In addition, experiments of flight behaviors of hawkmoth predict that roll angle feedback is needed to stabilize the lateral dynamics, and that a combination of pitch angle and pitch rate feedback is most effective in stabilizing the longitudinal dynamics (Windsor et al. 2014), therefore in this study, we control the forward/backward position and lateral position by adjusting the pitch and roll angles. Except the forward/backward and lateral freedoms, the remaining 4 degrees of freedom point to the vertical position, yaw angle, pitch angle and roll angle, respectively, which are controlled by tuning the wing kinematics. Here, we choose the positional angle amplitude ( ), the mean positional angle ( ), the difference of positional angle between right and left wings ( ), and the difference of mean feathering angle between right and left wings ( , ) as the tunable wing kinematics (figure 3(c)~(f)). All the quantities are expressed in a nondimensional form, such as: length, velocity, time, force, torque, mass, moment of inertia are by , , ⁄ , 2 2 , 3 2 , 3 , 5 , respectively. Because the body of bumblebee is of large mass and its flapping frequency is high so that the fluctuation of inertial forces and torques during the wing beat merely slightly influence the body dynamics, we thus take into consideration the cycle-averaged body dynamics for the sake of simplicity in designing the PD controller, such as: For small perturbations at hovering or very low flight velocities, these equations may be linearized as, where and are the acceleration and angular acceleration, respectively. The aerodynamic forces and torques are functions of the body motion ( , , , , , ), the wing kinematics ( , , , , ) and the body attitude ( , ). Because the pitch angle is selected to control the forward/backward position, we assume that the forward/backward force is mainly affected by the forward/backward velocity and pitch angle, thus the dynamics of forward/backward motion can be simplified as = ( , ). (2.4.4) The roll angle is selected to control the lateral position, and thus we assume that the lateral force is mostly affected by the lateral velocity and roll angle, resulting in the body dynamics of lateral motion as, = � , �. (2.4.5) We utilize the wing kinematics ( , , , , ) to control the vertical position, yaw angle, pitch angle and roll angle, while ignoring the effects of the degrees of freedom due to forward/backward and lateral position, and thus the dynamics of vertical, yaw, pitch and roll motions can be simplified as , with a sufficiently small , which are summarized in Table 1.
We then focus on the controller design on vertical position and three body angles (yaw, pitch and roll) assuming a time delay of one stroke cycle of the bumblebee. Following the analytical framework by our previous work (Cai et al. 2021), we first decouple the dynamic system (2.4.9) by introducing a transformation matrix which diagonalizes     (2.4.14) Δ ( = 1,2,3,4) is set to be the function of only, then the system is transformed from a MIMO (multi-input-multi-output) system into 4 SISO (single-input-single-output) systems. The PD controller with a time delay of one stroke cycle is then adopted, After substituting (2.4.14) into (2.4.12), we obtain By applying the Laplace transformation and root locus method under the principle maximizing the negative poles to achieve fast control (Cai et al. 2021), we obtain the optimal control parameters and . It should be noted that for In general, the hierarchical PD controller consists of a direct controller (equation (2.4.18)) and an indirect controller (equations (2.4.21), (2.4.22)). The indirect controller determines the desired pitch and roll angles while the direct controller determines the desired wing kinematics. Figure 4 shows the closed-loop flight dynamic system of the bumblebee.

Optimization of control parameters
By applying root locus method, the control parameters can be optimized in terms of minimizing the time of stabilization by minimizing the poles of the characteristic equation (Cai et al. 2021). The values of the optimized control parameters are listed in table 2 and the root locus are illustrated in figure 5. The optimal poles are all negative, guaranteeing the convergence of the flight stabilization with the proposed PD controller. Although the bumblebee processes various types muscles that function differently to each other (Lindsay et al. 2017), they are normally activated once in one stroke cycle, so the PD controller is actually time-discrete. To this extent, there will be a gap between the desired wing kinematics for next stroke cycle and the current stroke cycle (figure 6) by explicitly implementing the proposed PD controller, which is unrealistic. Here, we employ a smooth step function to create a transition of wing kinematics between the two successive stroke cycles with a transition time of 0.1 , ( ) = 6 5 − 15 4 + 10 3 . This smooth step function ensures the continuity of the second derivative of the wing kinematics (Ebert et al. 2003). Hence, the wing kinematics during the transition period can be given as where ( ) is the current wing kinematics, ( ) is the desired wing kinematics for next stroke cycle and is the transition wing kinematics.

Stabilization control under small perturbations
Firstly, we examine the flight stabilization based on the proposed PD controller under small perturbations. Figure 7 illustrates the trimmed state of a hovering bumblebee up to 100 stroke cycles (~0.7s), which reaches a stable periodic state after several stroke cycles; even with an initial state deviating slightly from the hovering state, obviously the flight can be stabilized effectively by the PD controller. With the symmetric wing kinematics of left-and right wing (figure 2), the reciprocation motion of flapping wings induces a slight body oscillation in the longitudinal motions involving forward-and backward-motion, vertical motion and pitch motion.
We then apply velocity perturbations to the hovering state along the three directions (x, y, z), which lead to some deviations associated with body attitudes in three body positions and three rotational angles (figure 8). The velocity disturbance is about 0.03 m/s (0.025 ) and the angular velocity 5 rad/s. All the perturbation-induced deviations are successfully corrected by the PD controller. The longitudinal motions are observed to be highly coupled: the forward/backward perturbation can result in pitch-deviation while the vertical perturbation can induce forward/backward and pitch deviations. This is also observed in the sideways motions of lateral motion, yaw and roll motions, that the lateral perturbation leads to yaw and roll deviations. Particularly, the time evolutions of yaw and roll are almost synchronous (figures 8(b), 10(a), 10(c)), indicating some strong coupling between yaw and roll motions. On the other hand, the longitudinal motions and sideways motions are likely to be decoupled as also reported in previous studies (Yao et al. 2019;Yao et al. 2020) that deal with the flight control in terms of longitudinal and sideways control separately. This trend is also seen in the flight stabilization under small angular velocity perturbations (figure 10). The variations of wing motion during the stabilization are relatively small compared to their amplitudes (figure 9, 11) except for the stabilization under vertical perturbation (figure 9c). The PD controller mainly tunes the wing kinematics in the first 20 stroke cycles, consistent with the 3 DoF PD control (Cai et al. 2021). While the PD control strategy has been proposed and proven to be effective in modelling some single DoF control in concern with various insect flights . Refer to the legends in figure 7 for the meanings of the colored curves. (Cheng et al. 2011;Ristroph et al. 2013;Beatus et al. 2015;Whitehead et al. 2015), our recent studies have further examined its capability in modeling 3 DoF body attitude control in terms of yaw, pitch, and roll (Zhang et al. 2018) and longitudinal control (Cai et al. 2021). In this study, our results demonstrate that the PD control strategy is also feasible in achieving a full 6 DoF body control in insect flights under small perturbations.
Furthermore, the 6 DoF PD controller-based flight stabilization shows a feature that the stabilization associated with the vertical motion and three body rotations of yaw, pitch and roll can be achieved very fast within 10 stroke cycles (0.07s), which is also observed in our previous study of 3DoF attitude control and longitudinal control (Zhang et al. 2018;Cai et al. 2021) and supported by the previous biological evidences of several different insect species (Ristroph et al. 2010;Cheng et al. 2011;Ristroph et al. 2013;Beatus et al. 2015;Whitehead et al. 2015). This implies that the fast-stabilization observed in the single and/or 3DoF PD controls can remain the same feature in the 6DoF PD-based control. On the contrary, the flight stabilization along forward/backward and lateral motions is comparatively much slower, showing slight overshoots and turning out to be stable up to more than 50 stroke cycles (figures 8, 10). This is mainly because the forward/backward and lateral motions are here controlled in terms of pitch-and roll-control, which lead to relatively some large time delays, i.e., latencies. The forward/backward motion however can be controlled directly by adjusting the wing motions to achieve a faster stabilization, with a longitudinal PD controller as proposed by Cai et al. (Cai et al. 2021). Since it is of the most importance for a flying insect to stabilize its body attitude in terms of yaw, pitch and roll rotations under any disturbances (Jakobi et al. 2018), a relatively slower stabilization on body positions may not essentially dominate its flight stabilization. Moreover, the current 6 DoF PD controller that is proven to be effective in 6 DoF body control in insect flights requires merely 4 direct alternations of wing kinematics, which is of great potential to simplify the flight controller design and related actuator-based fabrication in flapping-wing MAVs. This will be investigated in our future work as an application in insect-inspired flapping MAVs.

Flight stabilization with large perturbations
For the 6 DoF PD controller designed for flight stabilization under small perturbations, we assumed that the fluctuations of the inertial and aerodynamic forces can be neglected with a time-averaged body dynamic model. Thus, the PD controller is time-continuous rather than discrete in terms of the time interval of a wing stroke and given the small perturbations the body dynamic model can be linearized. Albeit with the assumptions, the effectiveness of the 6 DoF PD controller on the small perturbation is validated in section 3.2. In nature, however any wind gusts even with rather slight disturbances can impact largely the tiny insect body (Ravi et al. 2013;Beatus et al. 2015). Here we further examine the capability of the PD controller proposed on whether or not it still works even for large perturbations. We investigate the variation and response of three body positions and three rotational angles associated with flight stabilization in bumblebee, simply through amplifying the perturbations by 10 times. As under which the insect even stops flapping the wings at the beginning of stabilization control.
Interestingly, the highly coupled feature associated with the longitudinal motions is also observed here under large perturbations as that under small perturbations except for the vertical velocity perturbations. Under a large initial upward velocity perturbation (figure 12(c)), a backward deviation is induced opposite to that under small perturbation, which can be explained by the pitch response of the bumblebee, which pitches up under large upward perturbation but pitches down under small perturbations. As illustrated in figure 3(a), a pitch up causes a backward motion while a pitch down a forward motion. In addition, because of the largely induced pitch deviation, the forward/backward deviation even turns out to be greater than the vertical deviation. This however never occurs in the case of small perturbations. Under large perturbations, the sideways motions can also affect the longitudinal motions, for instance, under a large angular perturbation along , the pitch motion displays a remarked variation ( figure 14(a)). Compared to small perturbations, the dynamic response of hovering bumblebee under large perturbations behaves differently and the coupling among different degrees of freedoms shows much complicated while highly nonlinear feature. Nevertheless, the PD controller proposed is very likely to be capable of effectively stabilizing the complex flight as it does under small perturbations. Figure 14 shows the effects of the modulations of wing kinematics, of which the ranges cover the variation of wing kinematics for large perturbations, on the production of aerodynamic forces and torques. The aerodynamic forces and torques present linear relations with the modulations of wing kinematics, except for the effect of positional angle amplitude on the pitch torque ( ) (figure 14(e)), of which the nonlinearity is not strong. To this extent, the linear assumption for small perturbations in terms of the effects of wing kinematics on aerodynamic forces and torque still holds for large assumptions, which explains the applicability of the PD controller tuning wing kinematics to large perturbations.
It should be noted that for extremely large perturbation such as turning several circles the angular displacement should be modified because an additional entire circle actually does not affect the flying state . For insects, the proportional feedback is normally integrated from derivative feedback which could induce integral error for long-time flight and extremely large perturbations, so it is necessary to adopt the visual system to compensate such errors (Whitehead et al. 2015).
In addition, although maintaining stable body attitudes in flight is the primary objective for insects to stay airborne, remaining still at a specific position is also an important mission such as for pollinating flowers (Matthews et al. 2018). This can be achieved by introducing a time integral of body positions, and thus a PID controller is normally employed Yao et al. 2019;Yao et al. 2020). A complicated control strategy trained by deep reinforcement learning can be an alternative effective tool . However, whether the PD control strategy for body attitude control is applicable to the position control has been poorly studied (Ma et al. 2013). In insect flights, the attitude feedback can be detected by mechanosensory organs like halteres and antenna while visual systems are necessary to capture the body-position feedback. It is found that the sustained visual depolarization is transformed into a temporally structured train of action potentials synchronized to the haltere beating movements (Huston et al. 2009), indicating that the position feedback can be fused with attitude feedback to inform the PD control strategy. Such locomotion (flight mode and flight route) control such as following a moving target while locomoting to another location is essentially of great importance for point-point flight control in both bio-flyers and flying robots, and thus whether the 6DoF PD control strategy proposed here can be a feasible flight controller will be studied in our future research.

Conclusion
In this study we have established a computational framework that couples a 6 DoF PD controller and a CFD data-driven aerodynamic model (CDAM) to study the flight stabilization in bumblebee hovering. Unlike prior work mainly focusing on the longitudinal flight control for slight perturbations (Cai et al. 2021), of which the application is very limited, the full 6 DoF PD controller is capable of handling the disturbances from any direction. The flight control strategy associated with 6 DoF body attitudes comprising three translational velocities and three rotational angles (pitch, roll and yaw) is successfully underactuated with 4 inputs and 6 outputs. The 4 input parameters sorted out in flight control are the positional angle amplitude ( ), the mean positional angle ( ), the difference of positional angle between right and left wings ( ), and the difference of mean feathering angle between right and left wings ( , ), which are proven to fulfill a fast while precise flight control in terms of full 6 DoF. Our results demonstrate that the vertical, yaw, pitch and roll motions can be stabilized rapidly within merely 10 stroke cycles whereas it takes much more cycles to stabilize the forward/backward and lateral motions. Although the PD controller is derived under the assumption of small perturbations, it also shows a capability in the flight stabilization under much larger perturbations. This further indicate that the 6DoF PD flight controller is of potential capabilities in not only unveiling the fast stabilization mechanisms in insect flight but also providing an insect-inspired control strategy design for flapping-wing micro air vehicles.