Autonomous mobility, as exemplified by self-driving cars, autonomous mobile robots, drones, etc., is essential to the acceleration and practical application of transportation services and the automation of delivery, guidance, security, and inspection. Therefore, in recent years, expectations have been building for autonomous mobility to grow as a technology that not only improves the convenience and comfort of transportation and the efficiency of logistics but also leads to solutions to various social problems. Various technological elements are required to ensure the safety and quality of autonomous mobility. For example, technology is needed to create environmental maps and automatically determine obstacles based on data acquired by cameras and sensors such as LiDAR. Technologies for planning appropriate routes and controlling robots safely and comfortably are also essential.

This special issue highlights 24 exciting papers, including 20 research papers, three letters, and one development report. They are related to “recognition,” “decision and planning,” and “control” technologies for autonomous mobile robots, such as self-driving cars and drones. The papers’ keywords are as follows:

• Collision avoidance, path planning, path tracking control

• Motion control, attitude control

• Measurement, position and posture estimation, modeling

• Point cloud processing

We would like to express our gratitude to all authors and reviewers, and we hope that this special issue contributes to future research and development in autonomous mobility.

During the autonomous navigation of personal mobility vehicles (PMVs) in pedestrian spaces, avoiding collisions with pedestrians walking nearby is necessary. The avoidance paths of PMVs are affected by the behaviors of the pedestrians, which may also affect passenger comfort. Herein, a local path-planning method that considers passenger comfort to realize comfortable pedestrian avoidance during the autonomous navigation of PMVs in pedestrian spaces is proposed. First, the avoidance path and pedestrian behavior parameters affecting passenger comfort are investigated by evaluating passenger comfort in scenarios with different avoidance paths and pedestrian behaviors. Next, the requirements of a pedestrian avoidance method that considers passenger comfort are set based on the parameters affecting passenger comfort. Finally, a novel path-planning method that satisfies the requirements is proposed. The method is shown to generate comfortable paths in pedestrian spaces via a numerical simulation and participant experiment.

In typical traffic scenarios where there are no clear separations between the traffic participants, such as mixed traffic or shared space, vehicles and pedestrians are usually moving in the same time so that ego vehicle may need to face with multiple pedestrians in a relatively short interaction distance. Considering the stochastic motion of pedestrians and to balance the time consumption and safety during passing process, this paper proposes two strategies of collision avoidance (CA) for ego vehicle, which are based on model predictive control (MPC) and social force model (SFM). Besides, a modified SFM-based pedestrian model that considers the stochastic motion is given to evaluate the effectiveness of the proposed strategies. For MPC-based CA strategy, considering the unpredictable motion of the pedestrians, a novel speed re-planning layer combined with collision probability estimation, which is used to calculate an acceptable maximum safe speed for ego vehicle, is proposed. On the other hand, parameters associated with the SFM-based vehicle model are re-calibrated by particle swarm optimization (PSO) and the calibration process has been analyzed physically in details. The recommended values based on different initial interaction speed and distance of vehicle and pedestrians are also determined for further reference as useful findings from the analysis.

In demonstration experiments of automated driving vehicles, lane departures and collisions with roadside structures due to poor vehicle positioning and self-localization have been reported. In this study, we propose a promising method to prevent such departures and collisions, and then validate the proposed method by applying it to an actual automated driving vehicle. The proposed method monitors the target steering angles computed by the automated driving control and limits them before commanded the actuator when there is a risk of colliding with obstacles. As the above-mentioned control is lower-level, it can prevent an automated driving vehicle from colliding with obstacles without complicating upper-level controls. Experiments on an actual automated driving vehicle showed that the steering control structure of the proposed method could prevent an automated driving vehicle from colliding with obstacles by limiting its target steering angle. In addition, the method does not impose excessive limits on the steering angle when the automated driving vehicle follows a normal path and no risk of collision exists.

This study investigates the application of obstacle avoidance trajectory generation considering the vehicle shape of a micro ground vehicle by successive convexification and state-triggered constraints. The avoidance trajectory is generated by numerical computation and path-following experiments are conducted to assess the generated trajectory. The numerical computation results indicate that the trajectory obtained by the algorithm successfully avoids obstacles considering the vehicle shape and satisfies the constraints. The experiment includes the model predictive control to follow the generated trajectory. Numerical computations and experiments confirm the usefulness of the trajectory generation algorithm.

In this study, we investigate the problem of finding energy-efficient routes for multiple drones conducting a surface-wave seismic survey. The survey utilizes one seismic source and multiple measurement points spread over a designated area. Each drone carries a seismometer, and is tasked with visiting pre-specified points to take measurements of seismic signals by resting idle on the ground for a set time. Due to this mandatory idling time, their energy consumption is not proportional to the flight distance, nor it is possible to apply standard path minimization algorithms. To address this issue, we establish an energy consumption model for each drone and propose algorithms to optimally allocate points to each drone and generate routes that minimize total energy consumption. The validity of these algorithms is discussed using numerical simulations.

We have been participating in the Tsukuba Challenge, an open experiment involving autonomous robots, since 2014. The technology of our robot has stabilized, and our robot has continued to win the Tsukuba Mayor Prize from 2018 to 2021 without changing the basic configuration of the body and navigation software. Here, we report the robot’s structure as the project’s current completed form. Our robot is designed with the policy of selecting the most rational technology (appropriate technology) to achieve the purpose, even if it is not the latest. For example, we used image-like two-dimensional data instead of a three-dimensional point cloud in map matching for robot positioning. For pedestrian signal recognition, which was required to perform an optional task, we did not use deep learning but rather conventional color image processing. These techniques are advantageous for balancing the execution time and accuracy required in the challenge.

This paper describes a gesture interface for a factory transfer robot. Our proposed interface used gesture recognition to recognize the pointing direction, instead of estimating the point as in conventional pointing gesture estimation. When the autonomous mobile robot (AMR) recognized the pointing direction, it performed position control based on the object recognition. The AMR traveled along our unique path to ensure that its camera detected the object to be referenced for position control. The experimental results confirmed that the position and angular errors of the AMR controlled with our interface were 0.058 m and 4.7° averaged over five subjects and two conditions, which were sufficiently accurate for transportation. A questionnaire showed that our interface was user-friendly compared with manual operation with a commercially available controller.

In vehicle control, state estimation is essential even as the sensor accuracy improves with technological development. One of the vehicle estimation methods is receding-horizon estimation (RHE), which uses a past series of the measured state and input of the plant, and determines the estimated states based on linear or quadratic programming. It is known that RHE can estimate the vehicular state to which the extended Kalman filter cannot be applied owing to modeling errors. This study proposes a new computational form of the RHE based on primal-dual dynamics. The proposed form is expressed by a dynamic system; therefore, we can consider the computational stability based on the dynamic system theory. In this study, we propose a continuous-time representation of the RHE algorithm and redundant filters to improve the convergence performance of the estimation and demonstrate its effectiveness through a vehicle path-following control problem.

Vehicle yaw-rate control is important for realizing autonomous driving. If the desired yaw-rate response is realized, good autonomous driving can be realized. The gain-scheduled controller should be designed because vehicle has time-variant properties. However, it is difficult to design gain-scheduled controller in the case where vehicle parameters are unknown. To solve this problem, we expand virtual internal model tuning (VIMT) so as to realize desired yaw-rate responses. VIMT can tune the feedback controller by using one-shot experiment data. The processing cost is extremely low because the controller parameter can be obtained by using least square methods. In this study, we verify the validity of the proposed method through vehicle simulator of TruckMaker.

A quad rotor helicopter (QRH), a type of unmanned aerial vehicle (UAV), uses a tilted attitude to generate a horizontal thrust component in the flying direction. In the case of autonomous control, the attitude control system is used to tilt the airframe against disturbances such as crosswinds. Consequently, the flying attitude of a QRH is always inclined. In this study, a tilting mechanism for rotors (TMR) was mounted on a QRH to maintain a horizontal attitude. The TMRs were tilted to generate thrust against disturbances without inclining the airframe. The system was constructed using a QRH and TMRs tilted around only one axis and allocated every 90°. Because the airframe is always horizontal, this system can be used for the precise measurement of landforms and buildings. This paper reports the dynamic modeling of QRH equipped with TMRs and discusses the horizontal constant attitude flight using the proposed system.

This study presents the positioning method and autonomous flight of a quadrotor drone using ultra-wideband (UWB) communication and an optical flow sensor. UWB communication obtains the distance between multiple ground stations and a mobile station on a robot, and the position is calculated based on a multilateration method similar to global positioning system (GPS). The update rate of positioning using only UWB communication devices is slow; hence, we improved the update rate by combining the UWB and inertial measurement unit (IMU) sensor in the prior study. This study demonstrates the improvement of the positioning method and accuracy by sensor fusion of the UWB device, an IMU, and an optical flow sensor using the extended Kalman filter. The proposed method is validated by hovering and position control experiments and also realizes a sufficient rate and accuracy for autonomous flight.

Convolutional neural networks (CNNs) are often used for image recognition in automatic driving applications. Object recognition by a CNN generally uses a confidence score, which expresses the degree of recognition certainty. In most cases, the focus is on binary information, such as the size of the confidence score compared with a threshold value. However, in some cases the same recognition result has different scores. Even at intermediate confidence scores, the degree of recognition can be reflected in the control using the confidence score itself. Motivated by this idea, the aim of this study is to develop a method to control a mobile robot on the basis of the continuous confidence score itself, not on a binary judgment result from the score. Specifically, we designed a reference shaper that adjusts the reference speed according to the confidence score in which an obstacle exists. In the proposed controller, a higher score results in a smaller reference speed, which slows the robot. In a control experiment, we confirmed that the robot decelerated according to the confidence score, and demonstrated the effectiveness of the proposed controller.

Speed bumps and rumble strips have been introduced into the traffic infrastructure to improve traffic safety. When a vehicle travels on a road where speed bumps and rumble strips are installed, vibration stimuli are transmitted to the driver to encourage control of the speed and position of the vehicle. In this letter, speed bumps are applied to an automated driving system. More precisely, this letter considers the speed control of a mobile robot using vibration stimuli from bumpy road surfaces. We formulated a design problem for a speed control law for a mobile robot and proposed a controller that can adjust the speed according to road surface geometry. The performance of the proposed method was verified via simulation using Unity.

We implemented a novel visual navigation method for autonomous mobile robots, which is based on the results of semantic segmentation. The novelty of this method lies in its control strategy used for a robot during road-following: the robot moves toward a target point determined through semantic information. Previous implementations of the method sometimes failed to turn at an intersection owing to a fixed value of the turning angle. To address this issue, this study proposes a novel method for turning at an intersection using a control method based on a target point, which was originally developed for road-following. Here, an intersection is modeled as consisting of multiple straight roads. Evaluation using the CARLA simulator showed that the proposed method could accurately estimate the parameters representing a virtual road composing an intersection. In addition, run experiments conducted at the Ikuta Campus of Meiji University using an actual robot confirmed that the proposed method could appropriately make turns at intersections.

It is necessary for the robot to use interactions from the environment through the body in order to adaptively move through various environments. When the robot is faced with a narrow path or a space with many pillars, it should be able to use its interaction with the environment to thin its own shape, i.e., it should have a flexible body. In contrast, in the case where we want the robot to move forward powerfully on a slope or uneven terrain (small steps), it is preferable for the robot to rigidify its own body and exert a strong propulsive force in response to interactions from the environment. In this paper, we present an idea of a mobile robot that can adjust its body flexibility (stiffness) to realize such adaptive behavior, and furthermore, we demonstrate its validity through experiments. Specifically, we propose a closed-link deformable mobile robot whose stiffness can be adjusted by indirectly driving joints. We design a function that increases the stiffness of the body by controlling the joints to follow the target angle quickly, and a function that decreases the stiffness of the body by controlling the joints to follow the angle slowly. The effectiveness of a robot that can adjust its stiffness is demonstrated through experiments of traversing narrow paths and steps. We also discuss propulsion control that takes advantage of the deformable mobile robot and its applicability to uneven slopes due to the flexibility of the links.

In recent years, autonomous robots have been practically used outdoors for transportation, delivery, and other applications. To ensure safety and reliability, it is necessary to measure and evaluate the accuracy of a robot’s movement from the outside. In this study, we develop a device to externally measure the coordinates and azimuth angle of a robot by combining image processing and distance measurement. In this study, an azimuth angle detection method that combines coarse-angle detection using AKAZE feature detection and small-angle detection using optical flow is proposed. The experimental results show that the azimuth angle can be detected with a standard deviation of σ=0.70° indoors and σ=0.99° outdoors.

This study proposes a method for estimating the initial position of a mobile robot during a mobile robot experiment using expansion resetting. Depending on the type of sensor attached to the robot and the robot position and orientation estimation method, many operations may be required to estimate the initial position of the robot during an experimental run. The proposed method reduces the time and manual operations required to estimate the initial position and orientation of a mobile robot. The implementation of the method and its experimental results demonstrated the feasibility and effectiveness of the procedure.

In this paper, we propose a GNSS/IMU localization system for mobile robots when wheel speed sensors cannot be attached. Highly accurate location information is required for autonomous navigation of mobile robots. A typical method of acquiring location information is to use a Kalman filter for position estimation. The Kalman filter is a maximum-likelihood estimation method that assumes normally distributed noise. However, non-normally distributed GNSS multipath noise that frequently occurs in urban environments causes the Kalman filter to break down, and degrades the estimation performance. Other GNSS/IMU localization methods capable of lane-level estimation in urban environments use wheel speed sensors, which are unsuitable for the present situation. In this study, we aim to improve the performance of lane-level localization by adding a vehicle speed estimation function to adapt the method to those requiring wheel speed sensors. The proposed method optimizes time-series data to accurately compensate for accelerometer bias errors and reduce GNSS multipath noise. The evaluation confirmed the effectiveness of the proposed method, with improved velocity and position estimation performance compared with the Kalman filter method.

An autonomous vision-based landing system with a passively driven perching mechanism for a UAV is described in this paper. In our previous research, we developed a passively driven perching mechanism that can land on various shapes. The goal of this study was to achieve automatic perching on a thin vertical board using the passive mechanism. For autonomous perching, an RGB-D camera was used to detect and track a perching target and automatically control the flight of the UAV to the target position. The combination of RGB-D tracking system for thin vertical board and self-position estimation from another tracking camera mounted on the UAV enabled automatic perching. The results of the experiment using the passively driven perching mechanism and autonomous system verified that it is possible to land on objects such as road signs at heights, by using the integrated system for object detection and UAV control. In the experiment, the UAV was controlled to fly autonomously to the vicinity of the target and then perched on a 2 mm thick board.

The transportation of goods by drones using cable towing has recently attracted considerable attention. When flying a suspended load drone, any discrepancy between the mathematical model and the actual drone deteriorates control performance. However, because some physical parameters are difficult to measure, creating an accurate mathematical model is extremely difficult. Therefore, we propose a parameter estimation method using differential flatness that can be extended for application to suspended load drones. This method overcomes the problem of dealing with higher-order derivatives of flat outputs and enables the estimation of physical parameters. In this study, we experimentally show that the proposed method improves trajectory tracking performance.

This paper examines the aerodynamic drag force acting on a tilt-rotor UAV that has three mini fans and a main rotor with a tilt mechanism. The mini fans are embedded in the nose and the left and right wings. The main rotor is located near the center of the vehicle, and its front half is surrounded by the trailing edge of the nose in rotary-wing mode. The downward airflow from the fans and the main rotor generates an aerodynamic drag force called momentum drag, which is linearly proportional to the airspeed of UAV. To verify the existence of momentum drag, parameter identification of drag coefficients is performed from experimental data where the UAV flies forward in rotary-wing mode. The drag force is also investigated using computational fluid dynamics simulations. These experimental and numerical results are consistent with theoretical results based on momentum theory.

Light detection and ranging (LiDAR) has been the primary sensor for autonomous mobility and navigation system owing to its stability. Although multiple-channel LiDAR (3D-LiDAR) can obtain dense point clouds that provide optimal performance for several tasks, the application scope is limited by its high-cost. When employing single channel LiDAR (2D-LiDAR) as a low-cost alternative, the quantity and quality of the point cloud cause conventional methods to perform poorly in clustering and tracking tasks. In particular, when handling multiple pedestrian scenarios, the point cloud is not distinguished and clustering is unable to succeed. Hence, we propose an optimized clustering method combined with a Kalman filter (KF) for simultaneous clustering and tracking applicable to 2D-LiDAR.

To realize autonomous vehicle safety, it is important to accurately estimate the vehicle’s pose. As one of the localization techniques, 3D point cloud registration is commonly used. However, pose errors are likely to occur when there are few features in the surrounding environment. Although many studies have been conducted on estimating error distribution of 3D point cloud registration, the real environment is not reflected. This paper presents real-time error covariance estimation in 3D point cloud registration according to the surrounding environment. The proposed method provides multiple initial poses for iterative optimization in the registration method. Using converged poses in multiple searches, the error covariance reflecting the real environment is obtained. However, the initial poses were limited to directions in which the pose error was likely to occur. Hence, the limited search efficiently determined local optima of the registration. In addition, the process was conducted within 10 Hz, which is laser imaging detection and ranging (LiDAR) period; however, the execution time exceeded 100 ms in some places. Therefore, further improvement is necessary.

The three-dimensional (3D) normal distributions transform (NDT) is a popular scan registration method for 3D point cloud datasets. It has been widely used in sensor-based localization and mapping applications. However, the NDT cannot entirely utilize the computing power of modern many-core processors, such as graphics processing units (GPUs), because of the NDT’s linear nature. In this study, we investigated the use of NVIDIA’s GPUs and their programming platform called compute unified device architecture (CUDA) to accelerate the NDT algorithm. We proposed a design and implementation of our GPU-accelerated 3D NDT (GPU NDT). Our methods can achieve a speedup rate of up to 34 times, compared with the NDT implemented in the point cloud library (PCL).

In this study, a polyarylate fiber rope, which is a high-strength synthetic fiber rope, is used in the wire drive mechanism of a multi-degree of freedom (DOF) robotic forceps to evaluate its mechanical practicability. Using a nonconducting material for the drive wire, different from typical use of metallic wires made of stainless steel and tungsten, a technology is developed to simplify the insulation structure significantly, decrease the diameter of the robotic surgical instrument, and lower its cost. In this study, first, a prototype of the multi-DOF robotic forceps equipped with a polyetheretherketone (PEEK) resin flexible wrist joint part with an external diameter of 5 mm is manufactured. The prototype is used to evaluate the assembling of a polyarylate fiber rope with a diameter of 0.34 mm in a multi-DOF mechanism and examine the endurance of the rope to mechanical motions under a single-use assumption. As fastening structures to assemble the rope – a crimp terminal using a hollow pipe and a thread knot – are examined individually by assembling them in the prototype robotic forceps and conducting strength tests of the tension generated by the drive. The test results show that the thread knot method exerts a stabler fastening strength than the hollow pipe method. However, a problem of the former is that the wire may break because of its strong contact with the edge of the hole of the wire guide. Subsequently, to evaluate the endurance of the rope to single-use operation motion, operation tests are conducted by implementing reciprocating bending motions of the flexible wrist joint part of the robotic forceps 1,000 times. The assembled rope endures the sliding within the flexible wrist joint part and the contact loading with the guide part and the fixed structure within the cartridge repeatedly. The endurance operation test results confirm that the drive transmission of the polyarylate fiber rope has sufficient mechanical endurance to 1,000 reciprocating bending motions of the PEEK flexible wrist joint part.

This paper proposes a method for the point cloud data (PCD) map creation for the 3D LiDAR localization. The features of the method include the creation of a PCD map from a drawing of the buildings and partial scan of the not-existing object of the map by the tablet computer with the LiDAR. In the former, a map creation procedure, including the up- and down-sampling, as well as the processing, with voxel grid filter is established. In the latter, automatic position correction of the tablet scan data is introduced when they are placed to the current PCD map. Experiments are conducted to determine the size of the voxel grid filter and prove the effect of the tablet scan data in enhancing the matching level and the localization accuracy. Finally, the experiment with an autonomous mobile robot demonstrates that a map created using the proposed method is sufficient for autonomous driving without losing the localization.

Herein, we propose a spherical shell robot that can roll and move on its legs, and develop a prototype of the robot. Recently, there has been a growing demand for robots that can move freely and gather information on rough terrains, such as disaster sites, which are not accessible to humans. The robot developed here has two types of mobilities: rolling movement using a spherical shape and walking movement using its legs. Because the morphological transformation does not require recombination of parts, it can be reversibly performed via remote control. Therefore, the robot can select the movement method according to the environment, and reach the target point reliably even on uneven terrains, such as a disaster site. We designed a mechanism that enabled the transformation of the form and devised an operation method. Accordingly, a prototype was developed and tested. A rolling test on flat ground confirmed that the robot can roll over 5.0 m and its speed could be controlled using a gyro sensor. The leg locomotion test confirmed that the robot can turn and move straight ahead without turning over. In addition, we also conducted experiments, such as sudden stops and remote morphological deformation, to confirm the operation of the robot during rolling.

A method for welding line detection using point cloud data is proposed to automate welding operations combined with a contact sensor. The proposed system targets a fillet weld, in which the joint line between two metal plates attached vertically is welded. In the proposed method, after detecting the position and orientation of two flat plates regarding a single viewpoint as a rough measurement, the flat plates are measured from the optimal shooting position in each plane in detail to detect a precise weld line. When measuring a flat plate from an angle, the 3D point cloud obtained by a depth camera contains measurement errors. For example, a point cloud measuring a plane has a wavy shape or void owing to light reflection. However, by shooting the plane vertically, the point cloud has fewer errors. Using these characteristics, a two-step measurement algorithm for determining weld lines was proposed. The weld line detection results show an improvement of 5 mm compared with the rough and precise measurements. Furthermore, the average measurement error was less than 2.5 mm, and it is possible to narrow the range of the search object contact sensor for welding automation.

Approximately half of the patients with spinal cord injuries in Japan have a cervical spinal cord injury. Owing to the trunk dysfunction, patients with high-level spinal cord injuries have particular difficulty when searching for or picking up objects from the floor. Recently, welfare robot arms have been developed to help such individuals increase self-reliance. In this study, we propose an operating system that includes an eye-in-hand system with a touchscreen interface for grasping objects from the floor and delivering them to the individual. In the proposed method, the visual information of the target object is shown on a touchscreen interface. The patient specifies the target position for the robot arm by drawing a line on the target object on the interface. We conducted an experiment to compare the proposed interface with an on-screen joystick to demonstrate the proposed system’s efficiency and its ability to reduce physical burden. The results show that the proposed method is both quicker to use and effectively reduces the physical burden on the user compared to the conventional method.

Home-care manipulation robot requires exploring and performing the navigation task safely to reach the grasping target and ensure human safety in the home environment. An indoor home environment has complex obstacles such as chairs, tables, and sports equipment, which make it difficult for robots that rely on 2D laser rangefinders to detect. On the other hand, the conventional approaches overcome the problem by using 3D LiDAR, RGB-D camera, or fusing sensor data. The convolutional neural network has shown promising results in dealing with unseen obstacles in navigation by predicting the unseen obstacle from 2D grid maps to perform collision avoidance using 2D laser rangefinders only. Thus, this paper investigated the predicted grid map from the obstacle prediction network result for improving indoor navigation performance using only 2D LiDAR measurement. This work was evaluated by combining the configuration of the various local planners, type of static obstacles, raw map, and predicted map. Our investigation demonstrated that using the predicted grid map enabled all the local planners to achieve a better collision-free path by using the 2D laser rangefinders only rather than the RGB-D camera with 2D laser rangefinders with a raw map. This advanced investigation considers that the predicted map is potentially helpful for future work in the learning-based local navigation system.