The Tsukuba Challenge and the Nakanoshima Robot Challenge are both technical challenges in which mobile robots run autonomously in real outdoor environments. They have been held almost every year since 2007 and 2018, respectively, and many robots have participated in these public experiments. The autonomous navigation of a mobile robot in the real world, that is, not on test tracks but in environments normally used by everyday people, poses a great many challenges. The robots have to deal with environments that change with the time of day, the weather, the season, etc., and they have to deal with unexpected stationary obstacles as well as moving ones, including people, bicycles, etc. These days, demonstration tests of delivery robots are being conducted in many places, but there are still many problems that need to be solved. In this special issue, we have gathered papers detailing the insights gained from running mobile robots in the two outdoor experiments, the Tsukuba Challenge and the Nakanoshima Robot Challenge. To run a robot with a high success rate in a real environment, it is very important to devise the robot configuration, the sensor data processing, and the behavior control based on the knowledge gained from many experiences. We hope that sharing the successes and failures in the papers in this special issue will lead to further technological improvements in the future.

Semantic segmentation, which provides pixel-wise class labels for an input image, is expected to improve the movement performance of autonomous robots significantly. However, it is difficult to train a good classifier for target applications; public large-scale datasets are often unsuitable. Actually, a classifier trained using Cityscapes is not enough accurate for the Tsukuba Challenge. To generate an appropriate dataset for the target environment, we attempt to construct a semi-automatic method using a colored point cloud obtained with a 3D scanner. Although some degree of accuracy is achieved, it is not practical. Hence, we propose a novel method that creates images with shadows by rendering them in the 3D space to improve the classification accuracy of actual images with shadows, for which existing methods do not output appropriate results. Experimental results using datasets captured around the Tsukuba City Hall demonstrate that the proposed method was superior when appropriate constraints were applied for shadow generation; the mIoU was improved from 0.358 to 0.491 when testing images were obtained at different locations.

This study focuses on visual navigation methods for autonomous mobile robots based on semantic segmentation results. The challenge is to perform the expected actions without being affected by the presence of pedestrians. Therefore, we implemented a semantics-based localization method that is not affected by dynamic obstacles and a direction change method at intersections that functions even with coarse-grain localization results. The proposed method was evaluated through driving experiments in the Tsukuba Challenge 2022, where a 290 m run including 10 intersections was achieved in the confirmation run section.

Self-localization in probabilistic robotics requires detailed, geographically consistent environmental maps, which increases the computational cost. In this study, we propose a simple self-localization method that does not require such maps. In the proposed method, the order structure, such as the mobile robot’s navigation route, is embedded as trajectory attractors in the state space of a nonmonotone neural network, and self-position estimation is performed by processing based on the autonomous dynamics of the network. From experiments, we demonstrated the basic performance of the proposed method, including robust self-localization in complex outdoor environments. Furthermore, self-localization is possible on multiple courses with overlapping paths by suitably varying the network dynamics based on environmental information. While issues remain, this study points to the great potential of neurodynamics-based robotic self-localization.

We are exploring the use of semantic scene understanding in autonomous navigation for the Tsukuba Challenge. However, manually creating a comprehensive dataset that covers various outdoor scenes with time and weather variations to ensure high accuracy in semantic segmentation is onerous. Therefore, we propose modifications to the model and backbone of semantic segmentation, along with data augmentation techniques. The data augmentation techniques, including the addition of virtual shadows, histogram matching, and style transformations, aim to improve the representation of variations in shadow presence and color tones. In our evaluation using images from the Tsukuba Challenge course, we achieved the highest accuracy by switching the model to PSPNet and changing the backbone to ResNeXt. Furthermore, the adaptation of shadow and histogram proved effective for critical classes in robot navigation, such as road, sidewalk, and terrain. In particular, the combination of histogram matching and shadow application demonstrated effectiveness for data not included in the base training dataset.

This paper outlines a path planning method for autonomous rovers navigating urban environments without prior mapping, with a particular focus on addressing the Tsukuba Challenge. Our approach utilizes observations of pedestrian and robot movement trajectories to construct path graphs for global path planning. We provide a detailed overview of the autonomous rover’s hardware and software system, as well as a comprehensive description of the path planning algorithm. Our methodology entails extracting and continuously tracking dynamic objects from LiDAR data, resulting in the creation of a path graph based on their observed trajectories. Subsequently, a path aligned with the desired direction is selected. Notably, in indoor experimental settings, our approach proves effective, as the rover successfully generates a path to the goal by closely monitoring and tracking pedestrian movements. In conclusion, this paper introduces a promising path planning methodology and suggests potential areas for further research in autonomous mobility within uncharted environments.

In recent years, high-precision 3D environmental maps have attracted the attention of researchers in various fields and have been put to practical use. For the autonomous movement of mobile robots, it is common to create an environmental map in advance and use it for localization. In this study, to investigate the usefulness of 3D environmental maps, we scanned physical environments using two different simultaneous localization and mapping (SLAM) approaches, specifically a wearable 3D scanner and a 3D LiDAR mounted on a robot. We used the scan data to create 3D environmental maps consisting of 3D point clouds. Wearable 3D scanners can be used to generate high-density and high-precision 3D point-cloud maps. The application of high-precision maps to the field of autonomous navigation is expected to improve the accuracy of self-localization. Navigation experiments were conducted using a robot, which was equipped with the maps obtained from the two approaches described. Autonomous navigation was achieved in this manner, and the performance of the robot using each type of map was assessed by requiring it to halt at specific landmarks set along the route. The high-density colored environmental map generated from the wearable 3D scanner’s data enabled the robot to perform autonomous navigation easily with a high degree of accuracy, showing potential for usage in digital twin applications.

This paper proposes a robust posegraph optimization (PGO) method for posegraphs with keypoints. In the conventional PGO formulation, a loop constraint is defined between a pair of nodes, whereas in the proposed method, it is defined between a pair of keypoints. In this manner, robust PGO based on switch variables can be realized in a more fine-grained manner. Loop constraint is defined based on the unique geometric property of proximity point, and implemented as a new edge type of the g²o solver. The proposed method is compared with other robust PGO methods using real world data recorded in Nakanoshima Robot Challenge 2021.

We applied a brute-force value iteration algorithm to mobile robot navigation. Value iteration is computationally more expensive than search methods used for navigation. However, it can perfectly calculate the expected cost-to-go from any point in a state space. From this cost data, a robot can know not only the optimal behavior at any position and orientation but also the appropriate detour path against suddenly appearing obstacles. This study implemented value iteration and investigated its properties through experiments with simulated and actual robots. Although its computational cost remained high, our implementation could operate a robot in an actual outdoor environment with 3,700 m² free space. We also verified that our implementation calculates long detour paths toward closures composed of obstacles.

This paper proposes a method for the recognition of the opened/closed states of automatic sliding glass doors to allow for automatic robot-controlled movement from outdoors to indoors and vice versa by a robot. The proposed method uses an RGB-D camera as a sensor for extraction of the automatic sliding glass doors region and image recognition to determine whether the door is opened or closed. The RGB-D camera measures the distance between the opened or moving door frames, thereby facilitating outdoor to indoor movement and vice versa. Several automatic sliding glass doors under different experimental conditions are experimentally investigated to demonstrate the effectiveness of the proposed method.

A self-localization method that can seamlessly switch positions and attitudes estimated using normal distributions transform (NDT) scan matching and a real-time kinematic global navigation satellite system (GNSS) is successfully developed. One of the issues encountered in this method is the sharing of global coordinates among the different estimation methods. Therefore, the three-dimensional environmental maps utilized in the NDT scan matching are created based on the planar Cartesian coordinate system used in the GNSS to obtain accurate information regarding the location, shape, and size of the actual terrain and geographic features. Consequently, seamlessly switching between different methods enables mobile robots to stably obtain accurate estimated positions and attitudes. An autonomous driving experiment is conducted using this self-localization method in the Tsukuba Challenge 2022, and the mobile robot completed a designated course involving more than 2 km in an urban area.

In the trash-collection challenge of the Nakanoshima Robot Challenge, an autonomous robot must collect trash (bottles, cans, and bentos) scattered in a defined area within a time limit. A method for collecting the trash is to use machine learning to recognize the objects, move to the target location, and grasp the objects. An autonomous robot can achieve the target position and posture by rotating on the spot at the starting point, moving in a straight line, and rotating on the spot at the destination, but the rotation requires stopping and starting. To achieve faster movement, we implemented a smooth movement approach without sequential stops using a spline curve. When using the training data previously generated by the authors in their laboratory for object recognition, the robot could not correctly recognize objects in the environment of the robot competition, where strong sunlight shines through glass, because of the varying brightness and darkness. To solve this problem, we added our newly generated training data to YOLO, an image-recognition algorithm based on deep learning, and performed machine learning to achieve object recognition under various conditions.

This paper describes a study of the simple system integration of a mobile robot in the Nakanoshima Robot Challenge 2022. To improve the operability of the robot at the start of its journey, we studied the solution to the problem of initial localization during the experimental run and the setting of virtual obstacles on the map to be used by the mobile robot. This method reduces the amount of time and manual operation required to estimate the initial position and orientation of a mobile robot in mobile robot experiments. In this study, a mobile robot is implemented using open-source products such as robot operating system (ROS) and i-Cart mini. Experimental runs in the Extra Challenge of the Nakanoshima Robot Challenge 2022 demonstrate the feasibility of the method.

Recently, the legislation regarding autonomous mobile robots for outdoor pedestrian areas have been advancing, leading to increased expectations for task automation such as transportation and cleaning. Outdoor environments like parks, where vehicles cannot enter, present many three-dimensional terrains such as stairs and inclined surfaces, causing difficulty in achieving accurate environment recognition and autonomous movement. Furthermore, robots that navigate pedestrian walkways must be smaller and lighter than cars and also have a robust system capable of traversing steps and uneven surfaces and withstanding rainy weather. Currently, robots designed for paved roads are commercially available; however, robots capable of navigating park walkways are still in the research and development stage. Therefore, to accelerate the research and development of outdoor autonomous mobile robots, this study proposes the Navit(oo)n platform, designed for use in outdoor environments. This robot can be manufactured using easily obtainable parts, and all CAD data, circuit design data, and autonomous movement software are provided as open source. This paper introduces an overview of Navit(oo)n that successfully completed the course and achieved all tasks in the recent Nakanoshima Robot Challenge.

The ceiling serves as an ideal location for robots to handle transportation tasks, as it ensures minimal interference between automated guided vehicles (AGV) and human activities. A previous study developed a ceiling mobile robot called HanGrawler 2. It can travel at a high speed of 1.0 m/s to compete with ground vehicles. However, it occasionally fails during high-speed travel. This study aims to improve the reliability of starting, accelerating, and traveling at high speed. Optical motion capture is used to observe the crawler behavior of HanGrawler 2. The observation of the crawler behavior revealed that the crawler moves on an inflated trajectory during the high-speed movement. In addition, the experimental results show that the collision is not caused by the inflation, but by the push-in timing. The reliability of high-speed travel was improved by installing an encoder and optimizing the push-in timing in accordance with speed fluctuations.

In recent years, personal mobility vehicles have been required to operate autonomously in places with numerous pedestrians. A navigation method using a single human-following scheme is used to avoid collision with pedestrians. However, in many cases, a single human-following method cannot be successfully used for guidance. In crowded places, pedestrians do not always keep walking in the desired direction a user wants to go, and the vehicle must change the target pedestrian frequently. Instead of following a single pedestrian, we propose a method for the vehicle to follow a cluster of pedestrians for stable and robust following. First, the pedestrians around the vehicle are detected by multiple RGB-D cameras, and the pedestrians are tracked using YOLO and Deep Sort. Pedestrians are classified according to their walking direction, and the cluster of pedestrians walking toward the goal is selected and followed. However, the position of pedestrian is sometimes lost in occlusions and the accuracy of the walking direction depends on the distance and pose detected by the sensors. A notable problem is that the cluster of pedestrians is unstable in the cluster following; therefore, a median of candidate vectors (MCV) observer is used to remove outliers caused by observation errors. The proposed method is applied to a scenario involving pedestrians walking toward an elevator hall in a building, and its effectiveness is verified through experiments.

This study achieved four door operations, namely push-opening, push-closing, pull-opening, and pull-closing movements, using a mobile manipulator consisting of a commercially available arm robot and a mobile robot. We assumed that the arm robot is controlled by position commands at intervals of a few milliseconds, and that the mobile robot is guided by a simple straightforward linear trajectory. Ott, Borst, Bäuml, and Hirzinger proposed a push-opening method using impedance control in a cylindrical coordinate system for the arm robot. With this control, when the mobile robot advances toward and through the door, the arm robot moves passively and properly pushes the door open. However, their method is unsuitable for the above type of robot. Thus, we propose a method with two modifications: the use of admittance control and the improvement of force relaxation by considering a force obtained through a novel force decomposition. Furthermore, the proposed method was demonstrated not only in the push-opening movement but also in the push-closing, pull-opening, and pull-closing movements.

In cooperative transport systems, multiple robots work together to transport objects that are difficult to transport with a single robot. In recent years, multi-robot systems that cooperate to transport objects have been researched. However, during the transfer of objects, misalignment occurs between the ideal and actual grasp positions. In an automatic transport system, a grasping error can cause an error in the trajectory of the object, significantly reducing the transport efficiency. In this paper, a control system that allows robust cooperative transport control using a model error compensator is proposed for a leader–follower system in which the transported object is the virtual leader and the followers are ideally arranged. This system adds robustness to the operation of a conventional cooperative transport system by using the ideal formation of robots. The effectiveness of the proposed method was evaluated through cooperative transport experiments using two ideal formations for passing through a narrow entrance. The cooperative transport system could not pass through the narrow entrance using the conventional method; however, the system using the compensator passed through the narrow entrance smoothly.

Human-robot collaboration has garnered significant attention in the manufacturing industry due to its potential for optimizing the strengths of both human operators and robots. In this study, we present a novel variable admittance control method based on iterative learning for collaborative manipulation, aiming to enhance operational performance. This proposed method enables the adjustment of admittance to meet task requirements without the need for heuristic designs of admittance modulation strategies. Furthermore, the incorporation of dynamic time warping in human operational detection assists in mitigating the learning performance decline caused by fluctuations in human operations. To validate the effectiveness of our approach, we conducted extensive experiments. The results of these experiments highlight that the proposed method enhances human-robot collaborative manipulation performance compared to conventional methods. This approach also exhibits the potential for addressing complex tasks that are typically influenced by diverse human factors, including skill level and intention.

Many teams participating in robotic competitions achieve localization using a 2D map plotted using adaptive Monte Carlo localization, a robot operating system (ROS) open-source software program. However, outdoor environments often include nonlevel terrain such as slopes. In the indoor environment of multilevel structures, the data representing different levels overlap on the map. These factors can lead to localization failures. To resolve this problem, we develop a software by combining HDL localization, which is an ROS open-source software, with our own program, and use it to achieve localization based on a 3D map. Furthermore, the authors observe the erroneous recognition of a slope as a forward obstacle during a competition event. To resolve this, we propose a method to correct erroneous recognition of obstacles using a 2D laser range finder and 3D map and confirm its validity in an experiment carried out on a slope on a university campus.

In this paper, we propose an inchworm-type soft robot using a self-healing gel as its body and shape-memory alloy (SMA) wires as its actuators. To realize inchworm-like locomotion, two coiled SMA wires are placed in parallel in the gel-fabricated body. The bottom-side wire and the upper-side wire reciprocally bend by applying electric current to the actuators. To realize the self-restoration automatically, the robot consists of a self-healing body equipped with magnets. The paper introduces the structure of the inchworm-shaped robot with its inchworm-like locomotion performance, together with the self-healing function.

Visual place recognition from a 3D laser LiDAR is one of the most active research areas in robotics. Especially, learning and recognition of scene descriptors, such as scan context descriptors that map 3D point clouds to 2D point clouds, is one of the promising research directions. Although the scan-context descriptor has a sufficiently high recognition performance, it is still expensive image data and cannot be handled with low-capacity non-deep models. In this paper, we explore the task of compressing the scan context descriptor model while maintaining its recognition performance. To this end, the proposed approach slightly modifies the off-the-shelf classifier model of convolutional neural networks (CNN) from its basis, by replacing the SoftMax part with a support vector machine (SVM). Experiments with publicly available NCLT dataset validate the effectiveness of the proposed approach.

In fruit cultivation, viticulture requires the longest working hours in extended arm postures, much of which is carried out in standing postures to accumulate fatigue on arms, shoulders, and legs: a tough working environment. In this study, we propose a power assist system to assist its users in their extended arm work while they move in vineyards. The proposed system largely consists of a mobile robot, a power assist device for work, and a control system. The mobile robot is structured with a tracked vehicle for rough terrain arranged on its left and right sides so that the users can sit between the two vehicles and be assisted by the power assist device for work installed on it. The power assist device for work with a single linear actuator utilizing a linkage mechanism has the function to retain users’ hand attitude angles while assisting the flexion and extension movements of their shoulder, elbow, and carpometacarpal joints. Then, we verify by simulations the effects that the arrangement and lengths of links will have on the carpometacarpal joints’ trajectories as well as on the hand attitude angles. Finally, in order to check the effectiveness of the proposed power assist device for work, we conducted the evaluation experiments for assumed grape-harvesting work and gibberellin treatments. As a result, we proved its work assisting effects from the muscle activity states as well as its applicability to other kinds of work by altering its linkage structure and hand support part.

In recent years, underwater robotics has become very important because it can be applied to various fields such as underwater exploration, underwater inspection, marine industry, and environmental monitoring. Fin plays an essential role in the movement of underwater robots, providing operation, control, and efficient propulsion. This research aims to design and develop a unique robotic fin for underwater robots to improve their handling and propulsion efficiency. The goal is to improve the power density and propulsion efficiency of underwater survey robots. The study is based on a comprehensive analysis of experience and a performance evaluation. Five types of tail fin models were used in the study. The experimental results showed that the performance of the fin design can be compared with existing configurations under different conditions. The best design parameters will be determined by analyzing the experimental results. The results of this study will contribute to underwater robotics by providing a concept of the principles of fin design and its impact on the performance of robotics.

This paper presents a multi-box interpolation method to estimate point clouds during aerial-aquatic transition. Our proposed method is developed based on an investigation of noise characteristics of aerial point clouds and aquatic point clouds. To evaluate the performance of realistic point cloud estimation, we compare the interpolation method with the Gaussian mixture method. We also investigate how single-box and multi-box approaches deal with noise in point cloud estimation. The simulation and the experimental results show that the estimated point cloud is accurate even when the aerial and aquatic point clouds contain noise. Also, the multi-box concept helps the algorithm to avoid taking unwanted noise into consideration when predicting point clouds.

In this study, we propose a method for generating highly accurate high-density point clouds of piping facilities using an unmanned aerial vehicle (UAV) laser scanner and a handheld laser scanner. The point cloud for each scanline measured by the UAV scanner is repositioned on the piping axis, and the handheld scanner’s 3D point cloud is subsequently registered so that the center axis of the piping coincides with the UAV point cloud as a reference. The method proposed in this study was used to accurately reconstruct linear piping measured in high winds, which can easily deteriorate measurement accuracy. Whereas the conventional method incurred a deviation of 44.3 mm between the predicted and true values at altitudes of 15 m, the proposed method reduced this deviation to 19.4 mm. An application of the registration method demonstrated that the combined use of the two laser scanners enabled the creation of a high-density point cloud.

The number of patients with swallowing disorders is increasing according to the aging of society, although swallowing plays a significant role in the dietary process. The process of swallowing includes a very fast reflexive motion; there are difficulties in analyzing its mechanism even with the latest medical imaging technologies. In recent years, a simulator, named “Swallow Vision®,” has been developed from medical images such as MRI and CT to clearly visualize swallowing motion. It enables us to understand the kinesiology and analyze the motion of organs in swallowing. By using kinematic data obtained from this simulator and referring to medical knowledge, we develop a robotic simulator that has the potential to mimic human swallowing motion. The robot is able to perform tongue depressor and pharynx contraction to swallow food bolus. A performance evaluation is conducted to determine whether it is possible to swallow food bolus properly or where the bolus remains when failing.

Musculoskeletal disorders are common occupational diseases that have become a major social problem. Mechanization has been promoted as a solution to this problem. However, several tasks still require manual labor, such as fruit harvesting in orchards, making the introduction of machinery difficult in many cases. Recently, from the viewpoint of worker protection and ergonomics, various wearable robots for work support have attracted attention. In Europe and the US, there has been much development of arm-lifting assistive devices that support upward work while holding tools in the hands for industrial applications. However, most of the devices currently on the market are expensive compared to their assistive capabilities. Against this background, we developed three types of arm-lifting assistive devices with different concepts (an exoskeleton arm-lifting assistive device utilizing a gas spring, an exoskeleton arm-lifting assistive device utilizing McKibben-type artificial muscles, and an arm-lifting assistive suit utilizing rubber) to develop inexpensive, high-power devices. Furthermore, comparative verification of the assist effectiveness of each device was conducted.