The World Robot Challenge was organized to stimulate and gather findings in robotics and their applications to three important fields: Industrial Robotics, Service Robotics, and Disaster Robotics, which constituted three categories for the competition. Researchers and professionals from all over the world in the field of robotics were provided a unique opportunity to showcase their work by solving unprecedented problems, on the spot. Computer vision, robust control, navigation, and manipulation are some of the key elements to create robots that could solve the challenges proposed during the competition.

This special issue provides some examples of the work developed and performed in that aim, that could serve for the future of robotics applications in the three designated fields but not only.

The three first papers are the results of the convenience store challenge for Service Robotics. In this challenge, identifying objects and pose estimation of object are crucial for the acquisition of knowledge and robot manipulation, shelfing and logistics, as detailed in the first two papers. Robot’s navigation is also paramount for cleaning and operating in the human environment, such is the topic of the third paper.

The Industrial Robotics challenge focuses on parts pick and place and assembly of complex elements. The fourth and 5th papers of this special issue deal with this topic.

Finally the Disaster Robotics challenge presents the more versatile components as a disaster area may be of multiform and of very complex nature, depending of the nature of the disaster, whether it is indoor or outdoor etc. The last three papers of the special issue deal with such a various nature for plant inspection, for the control of robots in disaster area, or the rescue robot in confined spaces such as tunnels.

We hope this collection of work carefully selected, will provide the reader with insights for the future developments of robots.

The issues of estimating the poses of simple-shaped objects, such as retail store goods, have been addresses to ease the grasping of objects by robots. Conventional methods to estimate poses with an RGBD camera mounted on robots have difficulty estimating the three-dimensional poses of simple-shaped objects with few shape features. Therefore, in this study, we propose a new class called “poseclass” to indicate the grounding face of an object. The poseclass is of discrete value and solvable as a classification problem; it can be estimated with high accuracy; in addition, the three-dimensional pose estimation problems can be simplified into one-dimensional pose-estimation problem to estimate the yaw angles on the grounding face. We have developed a new neural network (PYNet) to estimate the poseclass and yaw angle, and compared it with conventional methods to determine its ratio of estimating unknown simple-shaped object poses with an angle error of 30° or less. The ratio of PYNet (68.9%) is an 18.1 pt higher than that of the conventional methods (50.8%). Additionally, a PYNet-implemented robot successfully grasped convenience store goods.

Product management using a multifunctional shelf, and manipulation using an electromagnet hand and a magnetic marker, are proposed for stock and disposal tasks. The multifunctional shelf manages the type, position, and number of products on the shelf, and plans display and disposal operations. The shelf provides directions to a mobile manipulator for moving products on the shelf according to the display and disposal plan. The proposed multifunctional shelf has a camera on each level that helps the mobile manipulator recognize the product. By optimizing the movement of products, the display and disposal work can be performed much more efficiently. To quickly grasp the product, a new manipulation strategy using a magnetic marker and an electromagnet hand is proposed. The electromagnet hand has two electromagnets and can quickly grasp and release the magnet marker by changing the S/N pole pair. Experiments using the proposed multifunctional shelf and electromagnet hand were conducted to demonstrate the effectiveness of the proposed system.

Research on the automated control of robots for wiping curved surfaces includes studies on cleaning contaminated areas and controlling the end-effector posture to apply a constant force along curved surfaces. However, such robots should also clean difficult-to-find contaminants such as dust and dirt. One of previous studies has considered motion generation as a generalized traveling salesman problem. In particular, a point cloud is acquired from an RGBD camera, and a large graph is created to represent the surface based on the point cloud. The system then finds an appropriate end-effector posture for each node and sets up multiple coordinate systems. Consequently, an efficient cleaning motion can be generated; however, path generation using the previous study is a highly time-consuming process. Therefore, in this study, a 3D model is used to generate cleaning actions more efficiently. A point cloud is then generated from the mesh data of the 3D model, based on which the surface is represented as a simple graph that can be solved as a traveling salesman problem, thereby reducing the computational time. The optimal end-effector posture is determined based on the shape of the surface and is set as the coordinate system for each node. Finally, experiments are conducted to compare the cleaning results with the previous study results, thereby verifying that results can be obtained in less than one-tenth the computational time required for the previous study results.

The World Robot Summit (WRS) 2020 Assembly Challenge aims to realize the future of industrial robotics by building agile and lean production systems that can (1) respond to ever-changing manufacturing requirements for high-mix low-volume production, (2) be configured easily without paying a high cost for system integration, (3)be reconfigured in an agile and lean manner, and (4)be implemented as agile one-off manufacturing for the industry. One of the technical challenges of assembly is the detection, identification, and classification of a small object in an unstructured environment to grasp, manipulate, and insert it into the mating part to assemble the mechanical system. We presented an end-to-end system solution based on flexibility, modularity, and robustness with integrated machine vision and robust planning and execution. To achieve this, we utilized the behavior tree paradigm and a hierarchy of recovery strategies to ensure effective operation. We selected the behavior tree approach because of its design efficiency in creating complex systems that are modular, hierarchical, and reactive, and offer graphical representations with semantic meaning. These properties are essential for any part assembly task. Our system was selected for the WRS 2020 Assembly Challenge finals based on its successful performance in qualifying rounds.

The assembly challenge of the World Robot Challenge (WRC) 2020, which was a part of the World Robot Summit (WRS) 2020, aimed to complete rapidly changing tasks in high mix/low volume production through building agile and lean production systems that can respond to one-off products. The authors of this paper participated in the challenge with the team PneuBot from the Industrial Robotics Facility of the Italian Institute of Technology by developing a flexible assembly system. The purpose of this work was to develop an assembly system able to handle variations of parts and tasks with a minimal changeover in hardware and software. In particular, assembly tasks were carried out, such as the assembly of a DC motor, pulleys, and a flexible belt on a plate, starting from pieces of unknown positions and orientations on a tray. The proposed work cell is light-weighted and can be fast deployed and replicated. It is composed of two Universal Robots; an RGB-D camera mounted on the wrist of the robot, able to detect both the position and orientation of the different objects to manage; a custom gripping system composed of 3D printed fingers for manipulation purposes and miniature force sensors for the grasping detection.

In this article, an approach used for the inspection tasks in the WRS2020 Plant Disaster Prevention Challenge is explained. The tasks were categorized into three categories: reading pressure gauges, inspecting rust on a tank, and inspecting cracks in a tank. For reading pressure gauges, the “you only look once” algorithm was used to focus on a specific pressure gauge and check the pressure gauge range strings on the gauge using optical character recognition algorithm. Finally, a previously learned classifier was used to read the values shown in the gauge. For rust inspection, image processes were used to focus on a target plate that may be rusted for rust detection. In particular, it was necessary to report the rust area and distribution type. Thus, the pixel ratio and grouping of rust were used to count the rust. The approach for crack inspection was similar to that for rust. The target plate was focused on first, and then the length of the crack was measured using image processing. Its width was not measured but was calculated using the crack area and length. For each system developed to approach each task, the results of the preliminary experiment and those of WRS2020 are shown. Finally, the approaches are summarized, and planned future work is discussed.

To participate in the Standard Disaster Robotics Challenge in the Disaster Robotics Category at the World Robot Challenge in the World Robot Summit 2020 Fukushima Competition, we have developed a main-replica type controller using which anyone can easily control the disaster robot. The two student operators finished second in the competition despite having only five hours of practice time. In this study, we present the results of surveying the essential elements of a controller for disaster response robots, and introduce the controller that is designed and implemented based on the survey.

Aging social infrastructure needs maintenance and inspection for which robot technology is highly effective. It is also effective for disaster rescue and recovery operations. Tunnel disaster rescue is risky for human workers. Robot technology can perform this work easily and accelerate rescue and recovery operations. This paper introduces a framework for the rescue robot development, control, and deployment steps. This framework also proposes automated and hybrid robot control systems in addition to conventional manual control. A multi-robot collaboration system causes the proposed control system to be convenient for deployment, reliable, and accelerate rescue work. In the development phase, controllers and different additional assistance tools for the robot have been developed in addition to the basic robot. As the proposed framework has been cultivated through our development and team collaboration training for the World Robot Summit (WRS) and the actual competition of the WRS, we will also present what we have achieved at the WRS.

This study developed an electric-powered manual wheelchair with a differential-drive active caster mounted at the rear of the wheelchair. The active caster has two axes: a caster wheel axis and a steering axis, and the coordinated control of these axes enables the wheel-mounted part to move in all directions. Conventional active casters have separate motors for the steering and wheel axes, and only the motor on the wheel axis operates when the wheelchair moves straight ahead, resulting in a low drive efficiency. Therefore, we developed a differential-drive-type active caster that uses a differential-drive mechanism. The differential-drive-type active caster uses the sum of the rotations of the two motors to move the steering axis, and the difference to move the wheel axis. Therefore, the two motors always operate, thus the drive efficiency is higher than that of a conventional type. However, we found that the two motors interfere with each other owing to their structure, making it difficult to stabilize the steering axis when speed increases. A control method was developed to solve this problem. Through verification experiments, we demonstrated the ability of the wheelchair to move at speeds over 6 km/h, the legal speed limit in Japan, the improved stability of the steering axis, and the accuracy of its omnidirectional movement. Additionally, to reduce the wobble that occurs when the electric-powered manual wheelchair moves straight ahead, we introduced a yaw-rate feedback to the joystick input system. The feedback was then verified in a lane-change experiment, confirming the wobble reduction.

We have developed a pneumatic artificial rubber muscle using two shape-memory polymer (SMP) sheets. We attached the SMP sheets to a linear pneumatic artificial rubber muscle. Utilizing the large difference in the elastic modulus below and above the glass transition temperature, the shape fixity and shape recovery of SMPs, the bending direction and the initial shape can be changed. In this study, in order to increase the bending motion range, we developed a segmented SMP sheet with embedded electrical heating wires, and evaluated its mechanical properties using bending and tensile tests. Moreover, we attached such sheets to an artificial muscle and evaluated the bending motion. Bending tests showed that segmenting the SMP sheets greatly reduced their bending stiffness. In tensile tests at temperatures below the glass transition temperature, it was found that the artificial muscle with the attached sheet withstood high elongating loads without failure. Attachment of such segmented SMP sheets to an artificial muscle resulted in an increased bending angle. Through isotonic and isometric tests, we showed that the prototype actuator could bend in two directions.

In this study, we proposed an electronic gimmick picture book system based on projection mapping for early childhood education in ordinary households. In our system, the visual effects of pop-up gimmicks in books are electronically reproduced by combining projection mapping, three-dimensional expression, and visual illusion effects. In addition, the proposed projector-camera system can provide children effective interaction experiences, which will have positive effects on their early childhood education. The performance of the proposed system was validated by projection experiments.

In this study, an autonomous traveling control system for a mobile robot was developed using the results for calculation of the relative positions and angles between the mobile robot and markers, based on the image information obtained from a camera mounted on the mobile robot. The mobile robot runs autonomously based on the path of the marker. However, as the conventional method uses OpenCV to identify the shape of the marker using the color information of the marker, the marker may be misrecognized owing to the influence of light. Furthermore, the specifications of the camera limit the detection distance of the marker placed opposite it, which inevitably limits the straight traveling distance of the mobile robot in the proposed method. The proposed method improves the accuracy of marker recognition by using deep learning, and also devises the method of placing markers that allows the user to move straight ahead over a longer distance. It can also easily achieve autonomous path travel control, including long-distance straight lines, for a mobile robot in an environment where global positioning systems (GPS) cannot be received. In addition, the system can be easily operated by an actual user, who need not have any knowledge of programming, because the travel path of the mobile robot can be set up simply by placing markers. The effectiveness of the proposed system was demonstrated through several experiments.

Numerous training simulators have been developed using virtual reality (VR) owing to their various advantages. Systems for training machine operations with physical movements face differences in the operational feel between actual and virtual machines. Moreover, virtual training is problematic in safety education because trainees in safe virtual environments can exhibit unsafe behavior in reality. To solve these problems, a previous study developed a virtual reality (VR) system to train a lathe operation with mixed reality using a motion capture system. This study included a function to teach the procedure and safety precautions for straight turning operations using a lathe. To evaluate the training effectiveness of this system, an experiment was conducted to compare learning using a video. Testees were divided into a simulator group, who learned with the system, and a video group, who learned with the video material. Work on the actual lathe by each testee after learning, was evaluated. Consequently, the actual work by the testees who used this system had fewer errors and shorter standstill times in which they attempted to recollect the next phase task. Although the number of trainees was small, this relationship had a statistical advantage. In the actual work, all the testees in the video group entered the danger area; however, only half of the testees in the simulator group entered the danger area. Therefore, a trainee using a simulator can remember the work process more reliably and accurately and perform it safely. Moreover, trainees who have undergone training several times should be able to perform actual work without making operational errors or engaging in unsafe behaviors.

Safety must be guaranteed for the widespread use of automated vehicles. Accurate estimation of the automated vehicle’s self-position is crucial to guarantee the safety of the automated vehicle. In this study, the performance of an image registration method using brightness for the self-position estimation of automated vehicles using 2D map images was evaluated. Moreover, the effect of the difference between the two map images on the image registration was evaluated. Consequently, if a two-dimensional Fourier transform is applied to a map image and the brightness gradient feature is present in only one direction, image registration can be performed within a 15 pixels offset in that direction. In addition, image differences in the direction of no brightness gradient were difficult to align. If the brightness gradient is in more than two directions, image registration can be performed within a radius of 10 pixels. Furthermore, failure to align the images in the rotational direction significantly affected the alignment of the images. If a map image is transformed using a two-dimensional Fourier transform and there are multiple brightness gradient features, image registration using the brightness gradient is effective for the map image.

Leg/wheel mobile robots, which have articulated legs ending in a wheel, can walk on legs as well as drive on wheels by switching between those two motive mechanisms in response to the terrain. However, effective control of the redundant degrees of freedom of leg/wheel mobile robots is complex. In this study, we propose a model predictive controller for leg configuration control that achieves both driving along the ground surface and climbing over a step. The proposed method simultaneously optimizes the robot pose, wheel positions, and joint angles. To consider the kinematic configuration of the legs explicitly, we formulate constraints on the relative position between the body and wheels. The ground contact condition of the wheels is approximately expressed as a continuous function with respect to each wheel’s relative position to the ground. This formulation induces smooth lifting of the wheels when the ground level abruptly changes, as when climbing a step. To prevent overturning, we evaluate the load distribution between each grounded wheel and constrain the body position to form a support polygon consisting of the grounded wheels. We conducted numerical simulations to verify that the proposed method achieves both driving on wheels and climbing over a step.

We propose a step-detection system to improve the safety of electric wheelchairs for the elderly. In automobile driving support systems, roads and other environments are being improved, and collision detection and avoidance for vehicles and other pedestrians are being studied and prioritized. However, in the driving environment of electric wheelchairs, it is important to deal with bumps and one-way slopes on the road. In this study, we conducted a survey on elderly people’s awareness of risk factors related to wheelchairs and categorized these risk factors. In addition, we confirmed the usefulness of the step-detecting device by conducting a driving experiment.

In a virtual reality (VR) space, wearing a head-mounted display can help with the visualization of objects; however, users cannot experience realistic tactile sensations. Recently, several force feedback devices have been developed, including wearable devices that use straight-fiber-type pneumatic muscles and magnetorheological fluids. This allows the devices to render elastic, frictional, and viscous forces during spatially unrestricted movement. Nevertheless, a problem remains in that previous devices could handle many bilateral upper limb movement tasks. Therefore, this study aims to develop a device that can handle movements that interact with both arms. Based on experiments concerning the pushing motion in a VR space, the influence of the pseudo force sense was determined to not be small. In addition, we confirmed that the force sensation presented by this system was more realistic when the robot was operated with both arms than when operated with the right arm.

A structure is proposed for high-speed goal recognition by SSD MobileNet using the Coral USB Accelerator. It was confirmed that the goal recognition rate from a long distance is equal to or better than that of conventional methods. Using this method, two CanSat contests were won. The aim of the CanSat competitions is to guide the CanSat to a distance of 0 m from the goal. It is initially guided by GPS but must eventually employ an image recognition model to identify the nearby goal. Zero-meter guidance to the goal was achieved in the Tanegashima Rocket Contest 2018 using a method that recognized the color of the goal by the color of the image, but it was vulnerable to changes in lighting conditions, such as weather changes. Therefore, a deep-learning method for CanSat goal recognition was applied for the first time at ARLISS 2019, and the zero-distance goal was achieved, winning the competition. However, it took more than 10 s for recognition owing to the CPU calculations, making it time consuming to reach the goal. The conventional method uses image classification to recognize the location of a goal by preparing multiple regions of interest (ROIs) in the image and repeating the recognition operations for each ROI. However, this method has a complex algorithm and requires the recognition of more ROIs to recognize goals over long distances, which is computationally time consuming. Object detection is an effective method to identify the location of the target object in an image. However, even if the lightest SSD MobileNet V1 and V2 and a hardware accelerator are used, the computation time may not be short enough because the computer is a Raspberry Pi Zero, the weakest class of Linux computers. In addition, if SSD MobileNet V1 and V2 do not have a sufficiently high recognition rate at long distances from the goal compared with conventional methods, it will be difficult to adapt them to a CanSat. To clarify this, SSD MobileNet V1 and V2 were applied to a Raspberry Pi Zero connected to a Coral USB Accelerator, and the recognition rate and recognition time were investigated at long distances from the goal. It was found that the recognition rate was equivalent to or better than that of the conventional method, even at long distances from the goal, and that the recognition time was sufficiently short (approximately 0.2 s). The effectiveness of the proposed method was evaluated at the Noshiro Space Event 2021 and Asagiri CanSat Drop Test (ACTS) 2021, and the 0-m goal was achieved at both events.

In this letter, we propose a monitoring system that employs a ceiling camera and a mobile robot. Ceiling camera is used to estimate the position of a fallen person, which is used by the robot to navigate to the person’s vicinity to examine his or her condition. This is done by transmitting the height information of obstacles and the fallen person from the ceiling camera to the robot so that it can accurately localize itself and plan a safe route. An actual system was constructed and verified that the required movements could be executed.

This letter presents an identification method for motor drive systems with flexible shafts and couplings using frequency response measurement. The drive system can be approximated as a two-mass non-rigid mechanical system to model the lowest resonant frequency with three parameters to be identified: the motor side inertia, the load side inertia, and the shaft stiffness. The proposed method does not require knowledge of the total inertia as many other techniques require. However, additional known inertia is added or removed from the load side assembly. The frequency response measurement is carried out with and without additional inertia to identify the resonant and anti-resonant frequencies. It is shown that this procedure directly identifies the two inertias and shaft stiffness and can be utilized to assist in the commissioning of electrical drives.