“Multi-agent systems (MAS)” have been extensively studied across various fields, including robotics, economics, biology, and computer science. A distinctive feature of these systems is the ability of multiple agents, each with different characteristics, to perform system-wide tasks through local bottom-up interactions. Furthermore, design and control methods for system networks based on graph theory are being developed. Recent applications of these methods include autonomous driving technology, smart grids, and understanding social systems.

This special issue aims to deepen the understanding of MAS, focusing on their control and applications. It features 16 papers, including one review paper. The accepted papers cover a wide range of topics, including reinforcement learning, autonomous mobility systems, and machine learning, presenting the latest research findings on MAS.

These studies provide valuable insights into various aspects and potential applications of MAS. We hope that this issue will be beneficial to our readers and contribute to the advancement of future research.

Multiagent reinforcement learning performs well in multiple situations such as social simulation and data mining. It particularly stands out in robot control. In this approach, artificial agents behave in a system and learn their policies for their own satisfaction and that of others. Robots encode policies to simulate the performance. Therefore, learning should maintain and improve system performance. Previous studies have attempted various approaches to outperform control robots. This paper provides an overview of multiagent reinforcement learning work, primarily on navigation. Specifically, we discuss current achievements and limitations, followed by future challenges.

A multi-agent system (MAS) is a system whose overall behavior is determined by local interactions among multiple autonomous agents. Recently, research has been conducted on the application of MASs in real-world environments, in which the agents are assumed to be robots that drive on the ground, i.e., autonomous mobile robots, and acquire external environmental information using cameras. In such cases, the information that can be obtained by the agent is limited to the field of view (FOV) of the respective camera, and the overall graph structure is dynamic and time varying. In addition, because the FOV may be obstructed by obstacles during camera measurements, obstacle avoidance must be considered. In this study, we examined the MAS consensus problem considering the effects of a limited FOV obstructed by obstacles. Specifically, we propose a control method using virtual agents that considers obstacle avoidance based on funnel control. In addition, simulation study was performed to demonstrate the effectiveness of the proposed method for solving the MAS consensus problem in an environment with obstacles.

Patrolling represents a potential application area for multi-robot systems, as it can enable efficient surveillance. A key aspect in facilitating the real-world applications of such missions is the enhancement of situation awareness of the base station (BS), in addition to ensuring well-coordinated patrol behavior. This paper addresses this requirement by proposing a layered patrol algorithm designed to maintain network connectivity with the BS. The novelty of this research lies in the distributed nature of the algorithm, despite the presence of the BS. Each robot independently determines its behavior based on local information while concurrently preserving connectivity to the BS. Additionally, this study introduces a novel performance metric to assess the situation awareness of the BS, focusing on the algorithm’s ability to provide prompt information about mission progress. Simulated missions revealed that the proposed algorithm outperformed existing algorithms, visited locations of interest more frequently and comprehensively, and provided the BS with improved situation awareness. Enhancing situation awareness may enable human operators to quickly gain insights into the system’s behavior based on mission progress, allowing for timely interventions if necessary. This capability contributes to improving human trust in autonomous systems.

Swarm robotics is the study of how a large number of relatively simple physically embodied robots can be designed such that a desired collective behavior emerges from local interactions. Furthermore, reinforcement learning (RL) is a promising approach for training robotic swarm controllers. However, the conventional RL approach suffers from the sparse reward problem in some complex tasks, such as key-to-door tasks. In this study, we applied hierarchical imitation learning to train a robotic swarm to address a key-to-door transport task with sparse rewards. The results demonstrate that the proposed approach outperforms the conventional RL method. Moreover, the proposed method outperforms the conventional hierarchical RL method in its ability to adapt to changes in the training environment.

Altruism is a key concept in the design of decentralized systems with high survivability. We focus on a community of vampire bats to reveal how intra-group altruism produces group-wide survivability. Although these bats die within three days if food is unavailable, they can survive for over 10 years by developing a highly sophisticated social community in which they share food. This food-sharing behavior occurs not only among blood relatives, but also among unrelated individuals through self-organizing social relationships based on grooming behavior. We propose a simple network model that focuses on the relationship between food sharing and grooming. We performed simulations under periodic, stationary, and irregular feeding environments, and found that suitable update rules for social relationships depend on the type of environment. Our findings provide insights into how decentralized systems with high survivability can be designed based on altruism.

As online resources such as social media are increasingly used in disaster situations, confusion caused by the spread of false information, misinformation, and hoaxes has become an issue. Although a large amount of research has been conducted on how to suppress disinformation, i.e., the widespread dissemination of such false information, most of the research from a revenue perspective has been based on prisoner’s dilemma experiments, and there has been no analysis of measures to deal with the actual occurrence of disinformation on disaster SNSs. In this paper, we focus on the fact that one of the characteristics of disaster SNS information is that it allows citizens to confirm the reality of a disaster. Hereafter, we refer to this as collective debunking, and we propose a profit-agent model for it and conduct an analysis using an evolutionary game. As a result, we experimentally found that deception in the confirmation of disaster information uploaded to SNS is likely to lead to the occurrence of disinformation. We also found that if this deception can be detected and punished, for example by patrols, it tends to suppress the occurrence of disinformation.

In search and rescue operations, the primary, urgent, and critical task is to locate people trapped under debris in collapsed buildings. Mobile robots are expected to facilitate the acquisition of information about a disaster site and the detection of victims. In this paper, we investigate the strategy for exploring the space under debris using multiple mobile robots. To establish a baseline, we first evaluate the performance of simple and primitive mobility algorithms, such as random walk and depth-first search, across various scenarios with different debris densities. We then consider combinations of these primitive algorithms, which allow mobile robots to adapt to the local surrounding conditions. Through simulation evaluation, we find that a stochastic algorithm contributes to fast exploration by multiple mobile robots, regardless of debris density, while a deterministic algorithm is effective when used by a single agent.

The Angry Birds AI Competition engages artificial intelligence agents in a contest based on the game Angry Birds. This tournament has been conducted annually since 2012, with participants competing for high scores. The organizers of this competition provide a basic agent, termed “Naive Agent,” as a baseline indicator. This study enhanced the Naive Agent by integrating a profit-sharing approach known as exploitation-oriented learning, which is a type of experience-enhanced learning. The effectiveness of this method was substantiated through numerical experiments. Additionally, this study explored the use of level selection learning within a multi-agent environment and validated the utility of the rationality theorem concerning the indirect rewards in this environment.

In this study, we introduce a deep Q-network agent utilizing a dueling architecture to refine the valuation of actions through separate estimations of the state-value and action-value functions, adapted to facilitate concurrent multi-agent operations within a shared environment. Inspired by the self-organized, decentralized cooperation observed in natural swarms, this study uniquely integrates a centralized mechanism, or a centralized critic. This enhances performance and coherence in decision-making within the multi-agent system. This hybrid approach enables agents to execute informed and optimized decisions by considering the actions of their counterparts while maintaining an element of collective and flexible task-information sharing, thereby presenting a groundbreaking framework for cooperation and information sharing in swarm robot systems. To augment the communication capabilities, we employ low-power wide-area networks, or Long Range (LoRa), which are characterized by their low power consumption and long-range communication abilities, facilitating the sharing of task information and reducing the load on individual robots. The aim is to leverage LoRa as a communication platform to construct a cooperative algorithm that enables efficient task-information sharing among groups. This can provide innovative solutions and promote effective cooperation and communication within multi-agent systems, with significant implications for industrial and exploratory robots. In conclusion, by integrating a centralized system into the proposed model, this approach successfully enhances the performance of multi-agent systems in real-world applications, offering a balanced synergy between decentralized flexibility and centralized control.

Studying collective behavior in opposing multi-agent teams is crucial across game theory, robotics, and sports analytics. In sports, especially football, team tactics involve intricate strategic spatial and action behaviors displayed as event sequences during possession. Understanding and analyzing these tactics is essential for successful training, strategic planning, and on-field success. While traditional approaches, such as notational and statistical analyses, offer valuable insights into team tactics, they often lack a comprehensive consideration of contextual information, thereby limiting the holistic evaluation of teams’ performances. To bridge this gap and capture the nuanced intricacies of team tactics, we employed advanced methodologies. The sequential pattern mining algorithm PrefixSpan was utilized to extract tactical patterns from possession sequences, enabling a deeper understanding of how teams strategize and adapt during play. Additionally, the neural marked spatio temporal point process (NMSTPP) model was leveraged to model and predict team behaviors, facilitating a fair comparison among teams. The evaluation of team possessions was further enhanced through the innovative holistic possession utilization score metrics, providing a more nuanced assessment of performance. In our experimental exploration, we identified and classified five distinct team tactics, validated the efficacy of the NMSTPP model when integrating StatsBomb 360 data, and conducted a comprehensive analysis of English Premier League teams during the 2022/2023 season. The results were visualized using radar plots and scatter plots with mean shift clustering. Lastly, the potential applications to RoboCup were discussed.

Although individuals of species engaging in cooperative foraging behaviors are often modeled as swarm intelligence optimization algorithms, there are also several species whose individuals take noncooperative foraging behaviors. Some such species exhibit common behaviors, which we call scale-free behaviors in this study. A type of scale-free behavior is spatial scale-free behavior, in which the moving distance of an individual from the present food source follows a power-law distribution. Second, the staying duration of an individual at the current food source follows a power-law distribution, and this behavior is called temporal scale-free behavior. We propose two types of noncooperative population-based search methods, based on the two types of scale-free behaviors. We also conducted simulations to compare the two methods, assuming static and dynamic environments in which the position of the food source did not change and changed, respectively. The simulation results showed that temporal scale-free behavior is suitable for specific problems in which individuals around the global optimum can be eliminated probabilistically, and spatial scale-free behavior is suitable for problems in which such elimination never occurs. In other words, the two types of scale-free behaviors are complementary. Next, we first assume problems for which we cannot know if the probabilistic elimination of individuals occurs in advance, and then propose a search method that selects an appropriate type of scale-free behavior for individuals during the search. The simulation results showed that this method demonstrates a good search performance, on average, for such problems.

With the development of autonomous driving technology utilizing machine learning, AI, and sensors, research on autonomous driving control has become more active, and a large number of innovative studies are underway. In the near future, all autonomous vehicle fleets will be able to communicate with each other for sharing information and overall optimal traffic control will be achieved. One of the vehicle control systems that are based on the premise of such a fully automated society is the “signal-less intersection.” There is an intersection traffic control method that achieves safe and rational route selection by using virtual walls (VWs), which are virtual obstacles, but there are issues in terms of total route length and reduction of computation time. To address the issues, we propose a method that (1) prunes unneeded paths and (2) arranges VWs in a stepwise manner. The effectiveness of the proposed method was evaluated by simulation, and the results showed that the total route length and execution time were reduced.

In recent years, extensive research has been conducted on the practical applications of autonomous driving. Much of this research relies on existing road infrastructure and aims to replace and automate human drivers. Concurrently, studies on zero-based control optimization focus on the effective use of road resources without assuming the presence of car lanes. These studies often overlook the physical constraints of vehicles in their control optimization based on reinforcement learning, leading to the learning of unrealistic control behaviors while simplifying the implementation of ranging sensors and vehicle-to-vehicle communication. Additionally, these studies do not use map information, which is widely employed in autonomous driving research. To address these issues, we constructed a simulation environment that incorporates physics simulations, realistically implements ranging sensors and vehicle-to-vehicle communication, and actively employs map information. Using this environment, we evaluated the effect of vehicle-to-vehicle communication and map information on vehicle control learning. Our experimental results show that vehicle-to-vehicle communication reduces collisions, while the use of map information improves the average vehicle speed and reduces the average lap time.

Although communication plays a pivotal role in achieving coordinated activities in multi-agent systems, conventional approaches often involve complicated high-dimensional messages generated by deep networks. These messages are typically indecipherable to humans, are relatively costly to transmit, and require intricate encoding and decoding networks. This can pose a design limitation for the agents such as autonomous (mobile) robots. This lack of interpretability can lead to systemic issues with security and reliability. In this study, inspired by common human communication about likely actions in collaborative endeavors, we propose a novel approach in which each agent’s action probabilities are transmitted to other agents as messages, drawing inspiration from the common human practice of sharing likely actions in collaborative endeavors. Our proposed framework is referred to as communication based on action probabilities (CAP), and focuses on generating straightforward, low-dimensional, interpretable messages to support multiple agents in coordinating their activities to achieve specified cooperative goals. CAP streamlines our comprehension of the agents’ learned coordinated and cooperative behaviors and eliminates the need to use additional network models to generate messages. CAP’s network architecture is simpler than that of state-of-the-art methods, and our experimental results show that it nonetheless performed comparably, converged faster, and exhibited a lower volume of communication with better interpretability.

Among automated driving technologies, adaptive cruise control (ACC), which controls the distance between vehicles and their relative speed, is becoming popular. Cooperative ACC (CACC) uses vehicle-to-vehicle communication and sensors to control the distance between vehicles. Recently, cyber attacks against automated driving systems have been a focus, in which information on pedestrians or preceding vehicles obtained from LiDAR and sensors is disguised. Such cyber attacks enable attackers to obtain accurate information on preceding vehicles and pedestrians. This can result in accidents. The purpose of this study is to develop a filtering function to reduce the impact of cyber attacks using available information. The contribution lies in proposing a filtering function for situations in which sensor offsets and communication delays arise owing to cyber attacks.

This study investigates the problem of distributed state estimation. A distributed Kalman filter algorithm is proposed, in which sensors exchange their innovations. A detailed analysis is conducted for the case of two sensor networks, demonstrating that the proposed algorithm outperforms the case where each sensor runs a conventional Kalman filter without communication. The upper bounds of error covariance matrices are also derived in the case of packet loss. Numerical examples verify the effectiveness of the proposed algorithm.

The Fundamental Mechatronics Research Institute at Osaka Electro-Communication University (MERI) has a research structure that includes five divisions. The institute conducts research on a wide variety of themes which cover a broad range of subjects. This paper introduces the institute’s research activities.

Recently, we have observed that the digital potential function defined by the difference between the real and virtual organ depth images is globally stable where the real and virtual livers coincide. This globality is then used to overlay the real and virtual livers. In this study, we consider the installation of a robotic mechanical system for measuring the depth images of real organs in the surgical bed. In general, virtual organs measured by CT or MRI show the position and posture of blood vessel groups and malignant tumors, and if these can be presented to the physician during surgery, he or she can operate while confirming their positions in real time. Although this robotic mechanical system is designed such that the camera can be raised or lowered as necessary to avoid interfering with the movement of the doctor, assistant, or nurse during surgery, it may still shift owing to contact with the hands or head of the doctor or nurse. In this study, an experiment was conducted in which a surgical measurement robotic mechanical system was constructed in a VR environment, and an actual robot was installed using this as a model. In the experiment, a video image of a virtual object was superimposed on that of a real object to confirm whether the surgical robotic mechanical system was able to accurately measure the surgical site.

Landing is a frequently executed motion in athletic activities, and injuries have been reported to occur often during landing, suggesting that legs are subjected to large loads during landing. Many studies, based on posture and floor reaction, have been conducted on this subject; however, few have been conducted from the muscular-activity perspective. Moreover, we found no studies investigating the functions of biarticular muscles (muscles that cross two joints). Therefore, we studied landing motion in which feet make ground contact after falling from a suspended position. Our objective was to clarify activity patterns of biarticular thigh muscles in two different trunk postures. Our results showed that the activity patterns of the antagonistic biarticular muscle pair at the thigh anterior and posterior were affected by trunk posture.

As the elderly population increases, independent living for the aged has become essential for extending a healthy life expectancy, and this requires the maintenance of mobility for daily activities, such as standing up and walking. Among these, standing up, a series of movements involved in getting out of a chair or bed, is the starting point of daily activities. Therefore, it is important to clarify the factors that contribute to accomplishing the transition from sitting to standing up. This study attempted to clarify the relationship between motor characteristics and the mechanism of muscle activity involved in the change from sitting to standing up. Specifically, we focused on the activity of the bi-articular muscles involved in the two joints simultaneously rather than joint torque that is the conventional standard for evaluating motion. We probed the mechanistic characteristics of bi-articular muscle activity as well as the main muscles that function during normal standing up motion, namely a natural standing up movement where the trunk is not vertically restricted, using electromyographic analysis, theoretical analysis using a link model based on the muscle arrangement of the lower limb, and experimental analysis using an actual model that reproduces the functions of these muscles to define the muscular activities of the thigh muscles.

Weed control has a significant impact on crop yield during cultivation. In this research, semantic segmentation is used to detect morning glories in soybean fields. By removing morning glory earlier in the growing season, the decrease in soybean crop yield can be minimized. However, it is difficult to create annotated images necessary for semantic segmentation at the growing season because soybeans and morning glories are both green and similar in color, making it difficult to distinguish them. This research assumes that morning glory colonies, once located at the growing season, remain stationary during the harvest season. The colonies of the morning glory at the growing season are identified by aligning the orthomosaic image from the growing season with the orthomosaic image from the harvest season because the leaves of the soybeans wither and turn brown during the harvest season. The proposed method trains a model of morning glory at the growing season based on its location at the harvest season and estimates the colonies of morning glory on the harvest season orthomosaic image. In this research, we investigated the accuracy of a deep learning-based morning glory detection model and discovered that the performance of the model varied depending on the proportion of morning glory areas on each image in the training dataset. The model demonstrated an optimal performance when only 3.5% of the proportion of the morning glory areas achieved an F₂ score of 0.753.

Remote-controlled mobile robots are expected to be used in difficult- or impossible-to-access environments for inspection workers and responders, such as in investigations and search activities at accident/disaster sites and inspection/investigation work at plants/infrastructure. Among ground mobile robots, crawler mobile robots with sub-crawlers (also known as ground-adaptive crawler robots) excel at in-ground adaptability and stack escape; however, their operators require advanced remote-control technology and experience. Therefore, the introduction of semi-autonomous control to assist the operator is required. In this study, the principle of the pushing-up sequence and the possibility of mobiligence emerging from interaction with obstacles caused by the robot movement were described. In addition, the sub-crawler rotary joint’s compliance, which significantly contributes to ground adaptability, was hypothesized, and a compliance control system design method that uses the sub-crawler constraint angle as a design condition was proposed. It was confirmed that the model robot for the evaluation, which used the proposed method, could adapt to unknown obstacles without measuring their height and shape and traverse them based on experimental results. In addition, based on the numerical calculation results, it was determined that the optimum solution for the restriction angle of the sub-crawler was approximately 35°–50° from the perspective of propulsive force and tumble stability.

In this study, a stable and high-speed vision-based self-position estimation method was proposed that improves upon the existing method of lane detection by recognizing the lighting facilities that are installed in tunnels on Japanese expressways where GNSS cannot be used. In addition, we proposed a method for inspecting multiple cracks at once by estimating the self-position with the successful rate 75% in the traveling direction by counting the lighting with the successful rate 99.85%. The effectiveness of the method was verified by capturing images of cracks in an actual tunnel. The proposed method will enable more frequent inspections for tunnel cracks that lead to flaking while maintaining infrastructure safety, reducing costs, and improving tunnel visual inspection flow efficiency.

Throwing manipulation utilizes virtual driving forces, such as gravity, centrifugal force, and Coriolis force, to extend the workspace and increase the degrees of freedom available for manipulation. However, it is highly sensitive to various uncertainties such as wind disturbance and measurement noise, thereby deteriorating the throwing performance. This study proposes a motion planning method that uses Bayesian optimization to obtain an optimal arm trajectory for throwing manipulation through repeated trials. Bayesian optimization can explicitly account for stochastic uncertainties and obtain an optimal solution with a small number of trials. The key contributions of the proposed method are the explicit modeling of stochastic uncertainties using a Gaussian distribution and the ability to reduce the number of retraining attempts. The efficacy of the proposed motion planning method was validated through extensive experiments. Specifically, in environments with randomly changing wind directions and wind speeds, experiments demonstrated that the proposed method generated throwing motions that were more robust against wind disturbances than conventional methods based on iterative learning methods. Furthermore, even when the throwing target point is changed, the experiments demonstrate that the proposed learning method can learn with fewer trials than the conventional method by utilizing past observation data.

The authors previously developed a flexible and multi-degree-of-freedom pneumatic artificial rubber muscle that uses shape-memory polymer (SMP) sheets with an embedded electrical heating wire. The bending direction and initial shape of the muscle can be changed by utilizing the large difference in the elastic modulus below and above the glass transition temperature, shape fixity, and shape recovery of SMPs. In this study, to improve performance, we propose a shape-memory composite (SMC) sheet that consists of SMP sheets with an embedded shape-memory alloy (SMA) wire used as an electric heating wire. The mechanical properties of the previously developed SMP sheets and the proposed SMC sheets are compared through shape recovery, bending, and tensile tests using prototypes. The motion of the artificial muscle with various samples attached is evaluated through an isometric test and bending angle measurements. The experimental results confirm that the use of the SMA wire improves the production reproducibility and shape recovery of the SMP sheets without degrading other mechanical properties or actuator performance.

This study proposes an autonomous lane-changing system for congested merging areas. Manual and autonomous vehicles are expected to coexist until all vehicles are substituted by autonomous vehicles. Therefore, interactions between humans and autonomous driving systems should be discussed. This study assumed a scenario in which an autonomous vehicle performed a lane change to a congested main lane, where all vehicles were manual. The proposed system estimated the possibility of changing lanes without collisions. A driving simulator was used to measure the lane-changing operations of human drivers in a congested merging area, and the proposed method was developed based on the experimental results. Simulations demonstrated that the proposed method could safely change lanes.