In the past three years, there has been rapid progress in the use of drones in society. Drones, which were previously used only experimentally in various industrial fields, are now being used in earnest in everyday operations. Drones are becoming indispensable tools in several industrial fields, such as surveying, inspection, and agriculture. At the same time, there has also been dramatic progress in autonomous drone technology. With the advancement of image processing, simultaneous localization and mapping (SLAM), and artificial intelligence technologies, many intelligent drones that apply these technologies are being researched. At the same time, our knowledge of multi-rotor helicopters, the main type of drones, has continued to deepen. As the strengths and weaknesses of multi-rotor helicopters have gradually become clearer, drones with alternate structures, such as flapping-wing drones, have come to attract renewed attention. In addition, the range of applications for drones, including passenger drones, has expanded greatly, and research on unprecedented drone operations, as well as research on systems and controls to ensure operational safety, is actively being conducted.
This special issue contains the latest review, research papers, and development reports on autonomous drones classified as follows from the abovementioned perspectives.
· Research on drone airframes and structures
· Research on drone navigation and recognition with a focus on image processing
· Research on advanced drone controls
· Research and development of drone applications
We hope that the readers will actively promote the use of drones in their own research and work, based on the information obtained from this special issue.
Aerial manipulation: physical interaction with the environment by using a robotic manipulator attached to the airframe of an aerial robot. In the future one can expect that aerial manipulation will greatly extend the range of possible applications for mobile robotics, especially multirotor UAVs. This can range from inspection and maintenance of previously hard to reach pieces of infrastructure, to search and rescue applications. What kind of manipulator is attached to what position of the airframe is a key point in accomplishing the aerial robot’s function and in the past, various aerial manipulation solutions have been proposed. This review paper gives an overview of the literature on aerial manipulation that have been proposed so far and classifies them by configuration of the workspace and function.
This paper describes the development of a flapping-wing unmanned aerial vehicle (UAV) named WiFly, which is equipped with a center-of-gravity (COG) shift mechanism. This mechanism allows seamless changes in the flight attitude between hovering and level flight by controlling the pitch angle. We implemented two types of feedback control systems in WiFly: PID control and reinforcement learning (shallow Q-learning) to stabilize the flight attitude. The controllability of WiFly is drastically improved by employing a double-motor drive system to independently control the flipping frequencies of the left and right wings.
In this paper, a computational fluid dynamics (CFD) analysis system based on a 3D-CAD model of a butterfly-style flapping robot using its experimental flight data is proposed. The butterfly-style flapping robot can control its attitude by changing its flapping and lead-lag angles; however, measuring the lift, thrust, and body pitch moment directly during flight is difficult. In the case of the flight motion analysis of insects, the state of flight has been photographed, and numerical analysis has been performed to obtain the flow field around the wings. However, when performing the motion analysis of hardware, it is difficult to reflect the shape of the body accurately using this method. In this study, a CFD analysis system considered the shape of the developed butterfly-style flapping robot as 3D-CAD data and analyzed the flow field around the wings using the experimental flight data of the hardware. The results of motion analysis showed that the attitude during flight differs due to the difference in lifts and body pitch moments in the flight experiment data of the hardware with different neutral angles of the flapping wings.
This paper proposes a rebound reduction mechanism for landing multicopter. Multicopter applications in the logistics industry are expected to increase owing to the aging of logistics drivers and the decline in their numbers. Currently, most multicopters land on their designated landing pads. However, these pads are not always available at the destination, and landing on rough terrain is needed in such cases. This paper describes the development of a mechanism to reduce rebound during landing, which can easily cause tipping over. The mechanism should be lightweight to ensure that battery power is conserved and that payload transport capability is increased. Simultaneously, the mechanism should be robust against environmental variations because multicopters are used outdoors and under various temperature conditions. A mechanism consisting of a spring and a magnetic damper is proposed and is modeled using multibody dynamics. It is known that the magnetic damper possesses robustness against temperature variations. Moreover, this paper presents the design parameter optimization for the proposed mechanism considering both the rebound reduction effect and weight reduction. The spring constant and viscous damping coefficient of the proposed mechanism are determined via numerical simulations with the electromagnetic simulator JMAG. The effectiveness of the proposed mechanism is verified via vertical freefall simulations. A simple experimental system is used to evaluate the mechanism, and the experimental results indicate that the mechanism is effective.
This paper describes a control method for an aerial manipulator on an unmanned aerial vehicle (UAV) by using a generalized Jacobian (GJ). Our task is to realize visual check of bridge inspection by employing a UAV with a multi-degree-of-freedom (DoF) manipulator on its top. The manipulator is controlled by using the GJ. Subsequently, by comparing the aerial manipulator control with a conventional Jacobian experimentally, we discovered that the accuracy of the control improved by applying the GJ. The manipulator has three DoFs in the X-Z plane of the UAV coordinate system. The experiment shows that the manipulator controlled with the GJ can compensate for the pose error of the body by 54.5% and 47.7% in the X- and Z-axes, respectively.
In this study, we propose an unmanned aerial vehicle (UAV) navigation system using LED panels and QR codes as markers in an indoor environment. An LED panel can display various patterns; hence, we use it as a command presentation device for UAVs, and a QR code can embed various pieces of information, which is used as a sign to estimate the location of the UAV on the way of the flight path. In this paper, we present a navigation method from departure to destination positions in which an obstacle lies between them. In addition, we investigate the effectiveness of our proposed method using an actual UAV.
This paper presents a method for the position identification of an unmanned aerial vehicle (UAV) in the Martian atmosphere in the future. It uses the image processing of craters captured via an onboard camera of the UAV and database images. The method is composed of two processes: individual crater detection using a cascade object detector and position identification using the recognition Taguchi (RT)-method. In crater detection, objects with shapes that resemble craters are detected regardless of their positions, and the positions of multiple detected craters are identified using the criterion variable D*, which is a normalized Mahalanobis distance. D* is calculated from several feature variables expressing the area ratios and relative positions of the detected craters in the RT-method.
Recently, flight control of unmanned aerial vehicles (UAVs) in non-global positioning system (GPS) environments has become increasingly important. In such an environment, visual sensors are important, and their main roles are self-localization and obstacle avoidance. In this paper, the concept of a multi-camera UAV system with multiple cameras attached to the body is proposed to realize high-precision omnidirectional visual recognition, self-localization, and obstacle avoidance simultaneously, and a two-camera UAV is developed as a prototype. The proposed flight control system can switch between visual servoing (VS) for collision avoidance and visual odometry (VO) for self-localization. The feasibility of the proposed control system was verified by conducting flight experiments with the insertion of obstacles.
As a possible extension of a drone application, transportation of a cable-suspended load is expected. The model of a drone with a suspended load is a nonlinear underactuated system that is known to be difficult to analyze and control. This paper applies the linearization method, known as hierarchical linearization, to the system. We observed that, via the hierarchical linearization scheme, the system can be linearized exactly and the controller can be designed simultaneously. There are two features of this approach. First, the controller exactly considers the system nonlinearity, and the feedback controller is based on the linear control theory. Second, it is possible to derive the analytical solution of the closed-loop system. We have demonstrated these features via numerical simulations.
This paper considers the position and attitude control of a quadcopter in the presence of stochastic disturbances. Basic quadcopter dynamics is modeled as a nonlinear stochastic system described by a stochastic differential equation. Subsequently, the position and attitude control is formulated as a nonlinear stochastic optimal control problem with input saturation constraints. To solve this problem, a continuous-time stochastic differential dynamic programming (DDP) method with input saturation constraints is newly proposed. Finally, numerical simulations demonstrate the effectiveness of the proposed method by comparing it with the linear quadratic Gaussian and the deterministic DDP with input saturation constraints.
The increased use of UAVs (Unmanned Aerial Vehicles) has heightened demands for an automated landing system intended for a variety of tasks and emergency landings. A key challenge of this system is finding a safe landing site in an unknown environment using on-board sensors. This paper proposes a method to generate a heat map for safety evaluation using images from a single on-board camera. The proposed method consists of the classification of ground surface by CNNs (Convolutional Neural Networks) and the estimation of surface flatness from optical flow. We present the results of applying this method to a video obtained from an on-board camera and discuss ways of improving the method.
In recent years, the number of bedridden patients, including amyotrophic lateral sclerosis (ALS) patients, has been increasing with the aging of the population, owing to advances in medical and long-term care technology. Eye movements are physical functions that are relatively difficult to be affected, even if the symptoms of ALS progress. Focusing on this point, in this paper, in order to improve the quality of life (QOL) of bedridden patients, including ALS patients, we propose a drone system connected to the Internet that can be remotely controlled using only their eyes. In order to control the drone by using only their eyes, a control screen and an eye-tracking device were used in this system. By using this system, for example, the patients in New York can operate the drone in Kyoto using only their eyes, enjoy the scenery, and talk with people in Kyoto. In this drone system, since a time delay could occur depending on the Internet usage environment, agile operation is required for remotely controlling the drone. Therefore, we introduce the design of the control screen focused on remote control operability and human eye movements (microsaccades). Furthermore, considering the widespread future use of this system, it is desirable to use a commercial drone. Accordingly, we describe the design of a joystick control device to physically control the joysticks of various drone controllers. Finally, we present experimental results to verify the effectiveness of this system, including the control screen and the joystick control device.
In this paper, we describe a semi-automatic viewpoint moving system that employs a drone to provide visual assistance to the operator of a teleoperated robot. The objective of this system is to improve the operational efficiency and reduce the mental load of the operator. The operator changes the position of the drone through an interface to acquire the optimal assist image for teleoperation. We confirmed through an evaluation experiment that, in comparison with our previous study in which the final positions of the drone were determined in advance, the proposed method improves the operational accuracy and reduces the mental load of the operator.
In drone photography of vast natural terrain, it is difficult to know in advance the exact location and shape of an object. In addition, there are many time and location constraints in such environments; therefore, it is desirable to capture images quickly by automatic flight. The authors had previously proposed a method to determine the safety of such an automatic flight photography plan in advance and capture photographs quickly. The method was designed to model an object from the safety zone using multiple drones and determine a safe path in advance. However, further improvement in the efficiency was necessary when photographing the object over a large area. In contrast, in this study, to improve the efficiency of the safe flight path determination method for large-scale subjects, we developed a new method to model each number of photographs using structure from motion (SfM) and verify the accuracy of the model obtained for each number of photographs in advance. In addition, by determining the appropriate number of shots based on the results obtained and reducing the loss of time and battery during shooting, we verified the extent to which the total flight time could be reduced for a flight path of shooting a large-scale object in the Esan Prefectural Natural Park. In the case of the Esan Prefectural Natural Park, we demonstrate that the difference in the small-object shooting time was not a problem, but the difference was significant for shooting large objects. The effectiveness of determining and applying, in advance, the number of shots that provides appropriate accuracy is demonstrated.
This paper presents an alternative approach to identify and classify the group of small flying objects especially drones from others, notably birds and kites (inclusive of kiteflying), in near field, by examining the pattern of their flight paths and trajectories. The trajectories of the drones and other flying objects were extracted from multiple clips of videos including various natural and synthetic database. Four trajectories characteristics are observed and extracted from the object’s flight paths, i.e., heading or turning angle, curvature, pace velocity, and pace acceleration. Subsequently, principal component analyses were applied to reduce the number of these trajectory features from 4 to 2 parameters. Multi-class classification by support vector machine (SVM) with non-linear transformation kernel was used. Multiple classification models were developed by several algorithms with various transformation kernels. The hyperparameters were optimized using Bayesian optimization. The performances of the different models are compared through the prediction accuracy of the test data.
A variable-pitch-controlled quadrotor drone was simulated in the ground effect using a high-fidelity CFD solver. In contrast to a single rotor in the ground effect, which has been extensively studied for conventional helicopters, the flow fields around multiple rotors are complex. In this study, the rotating speed of the rotors was maintained constant, and the blade pitch angles were adjusted so that the total thrust of the multicopter was the same regardless of the rotor height from the ground. It was observed that the power required for the quadrotors, which generate the same thrust, decreases when the rotors are approaching the ground from the height where they can be considered to be out of the ground effect, but increases locally when the rotor height is approximately the rotor radius, owing to flow recirculation into the rotor, and then decreases abruptly when the rotors further approach the ground. The outwash from the quadrotors depends heavily on the direction relative to the quadrotor layout. Along the plane crossing the diagonal rotor centers, the outwash velocity profiles resemble those of a single rotor; however, the outwash from the rotor gaps is stronger and extends to a much higher altitude.
An unmanned aircraft system traffic management (UTM) system to support flights beyond visual line-of-sight is considered necessary for the promotion of commercial drone use. In the research and development of UTM systems, cost and time constraints make it difficult to actually fly a large number of drones in the same airspace, so research is mainly conducted using a simulator. This paper presents details of a UTM simulator named the “scalable simulator for knowledge of low-altitude environment” (SKALE) developed by the Japan Aerospace Exploration Agency (JAXA), with respect to the construction of a model case of drone delivery model set in 2030 in Japan. Moreover, UTM concepts for airspace safety and efficient airspace utilization (parcel transport) are proposed and evaluated using JAXA’s UTM simulator and drone delivery model cases. Simulation results are discussed, and the knowledge gained for the improvement of airspace safety and airspace utilization (parcel transport) efficiency is documented.
The 169 MHz band was assigned for unmanned vehicles such as robots and drones by the Ministry of Internal Affairs and Communications (MIC) in 2016 in Japan, according to the reallocation of the TV band from VHF (analog) to UHF (digital). This band is expected to have a long range and good diffraction characteristics during drone operation over a long range with obstacles such as buildings, trees, or terrains. In this paper, we present the prototype system and its propagation experiments of command and telemetry communications at 169 MHz, which is added to a multi-hop communication unit in the 920 MHz band developed previously to enhance the beyond line-of-sight (BLOS) operation performance for drones.
An RTK-GNSS was introduced to realize the precision flight of a drone system for transporting harvested loquats and spraying pesticides to suppress the rotting of fruits. Transportation and spraying experiments were conducted. Precision automatic navigation flights were realized in transportation experiments. In addition, precision landing was performed within approximately 10 cm of the target position. Sufficient spraying flights were performed during the spraying experiment.
This study evaluates the effect of swing support during walking using a wireless pneumatic artificial muscle (PAM) driver on hip and knee flexion angles. This driver can control two contraction parameters of the PAM: delay of contraction from the trigger and duration of contraction through a smartphone. Eleven healthy young individuals participated in this study. We asked the participants to walk with two PAMs attached to the left hip joint and a pressure sensor placed under the right heel to trigger the contraction. During the experiment, the contraction parameters were randomly changed: 0, 100, or 200 ms for the delay and 0, 100, 200, or 300 ms for the duration. The experimental results revealed significant differences in the hip and knee flexion angles, hip joint angular excursion, and stride length among the conditions. In addition, the optimal parameter differed among the subjects. It was confirmed that this individual variation was related to the walking speed of the subject, without PAM assistance.
In this study, with the aim of installing an object recognition algorithm on the hardware device of a service robot, we propose a Binarized Dual Stream VGG-16 (BDS-VGG16) network model to realize high-speed computations and low power consumption. The BDS-VGG16 model has improved in terms of the object recognition accuracy by using not only RGB images but also depth images. It achieved a 99.3% accuracy in tests using an RGB-D Object Dataset. We have also confirmed that the proposed model can be installed in a field-programmable gate array (FPGA). We have further installed BDS-VGG16 Tiny, a small BDS-VGG16 model in XCZU9EG, a system on a chip with a CPU and a middle-scale FPGA on a single chip that can be installed in robots. We have also integrated the BDS-VGG16 Tiny with a robot operating system. As a result, the BDS-VGG16 Tiny installed in the XCZU9EG FPGA realizes approximately 1.9-times more computations than the one installed in the graphics processing unit (GPU) with a power efficiency approximately 8-times higher than that installed in the GPU.
This study focuses on the high maneuverability of fish in water to design a fish-like robot via snap-through buckling. The aim of this study is to improve swimming speed by increasing the frequency at which snap-through buckling occurs. Here, we propose a novel drive mechanism using a triangular cam that can continuously generate snap-through buckling at a high frequency. In addition, we developed a fish-like robot via the proposed mechanism and analyzed the influence of the frequency of snap-through buckling on swimming speed. The results obtained indicate that swimming speed is improved and that the relationship between frequency and swimming speed exhibits a single peak. In other words, the swimming speed is reduced when the frequency is significantly increased. We also determined that swimming speed was improved using a wide elastic thin plate as the driving mechanism.
The McKibben Pneumatic Actuator (MPA) is well-known as a type of soft actuator. As MPA generates tension only in the direction of compression, it is necessary to construct an antagonistic structure to drive a joint by MPAs and to coordinate antagonized MPAs. Similar to MPA, muscles in animals also generate tension only in the direction of contraction. Some studies have reported that animals utilize tension information to coordinate muscles for various autonomous movements. The purpose of this study is to realize autonomous cooperation between antagonized MPAs by applying tension feedback control and analyzing the mechanism of coordination. For this purpose, we verify the effect of tension feedback control on the 1-DOF pendulum model with antagonized MPAs. First, through numerical simulations, it is confirmed that the tension feedback generates various coordinated movements of antagonized MPAs, and the pendulum exhibits a bifurcation phenomenon based on the phase difference of the inputs of MPAs. Thereafter, we develop an actual experimental machine based on the model and confirm the autonomous cooperation between actual MPAs through verification experiments similar to the numerical simulations.
Stainless steel railway car bodies are assembled by joining the outer plates and the pillar materials using resistance spot welding. In recent years, more and more car bodies are being assembled using laser welding in addition to the resistance spot welding. For this laser welding system, we developed a condition monitoring system considering the processes before and after laser welding as a single system, and obtained and put into practical use an appropriate condition that suppresses spatter generation during laser welding. On the other hand, in resistance spot welding, the current, weld time, electrode load, and electrode tip shape are the main factors that determine the welding quality. Therefore, the configuration of the equipment is less complicated than that of laser welding system, and the condition monitoring is easier than that of the laser welding. In this study, by transferring the concept of the condition monitoring system developed for laser welding to resistance spot welding, we achieved a reduction of more than 60% in terms of electricity consumption and improved the appearance of the car body by optimizing the indentation shape. In addition to this technical achievement, we also present in this paper a case study showing the opportunity for innovation by restructuring the technological paradigm of the resistance spot welding in the production of stainless steel car body shells.