In the studies done to date on the swarm behaviors of animals, many different observational techniques have been developed, indicating the importance of such detailed observations. The techniques of researchers aiming to capture the swarm behavior of animals, which is normally visually unobservable, have included attaching microsensors to honey bees or ants and data loggers (micro recorders) to birds or mammals. Such techniques, collectively known as “bio-logging,” can go far in clarifying why we feel animals that exhibit swarm behaviors seem to have a sort of collective intelligence, or “swarm intelligence.” Furthermore, studies on the swarm behaviors of animals may provide important clues to researchers in the field of swarm robotics. It is in this context that this special issue presents papers on bio-logging technologies, the collective behaviors of animals, and various advanced measurement technologies related to them.

This special issue consists of one review article and 14 research papers. The subjects cover a wide range of areas, including control engineering, data science, and ecology. Thus, bio-logging is an interdisciplinary area that can expect to see much growth in the near future.

The editors are confident that this issue will greatly contribute to further progress in the field of bio-logging.

The bio-logging technique is extensively used in the fields of ecology and ethology, wherein a data logger, such as a sensor or camera, is attached to the target animal’s body to collect the required data. In this method, the efficiency of recovery of the data logger is not ideal. In this study, we proposed a new recovery method, with the aim of addressing the aforementioned problem in bio-logging. The authors previously fabricated a data-logger separator, which weighed approximately 10 g, and was targeted at small seabirds. Because there were some problems associated with the circuit board and the separation performance of this device, we modified the device to overcome the previous drawbacks. We fabricated a flexible printed circuit to improve the operation of the mounted actuator and wireless microcomputer, and improve the efficiency of the fabrication process. We conducted an experiment to determine the proper length and position at which the actuator is attached, in order to achieve a stable motion. We thus fabricated a new prototype with these improvements and performed an operational test at low temperatures from a particular distance, simulating actual usage in a natural environment. The results demonstrated that separation occurred without failure, thus indicating that the separator can be efficiently used in practical environment.

A typical honeybee colony contains more than 15,000 individuals, each with its own task related to supporting the hive and maintaining the colony. In previous studies on honeybees, observing individual animals’ behaviors has been a difficult and time-consuming task to understand the relationship between in-hive communication and environmental changes outside the hive, therefore it is necessary in any attempt to develop applying a remote sensing technology. To allow researchers to pass much of this tracking work on to computers, we have developed the lifelog monitoring system for honeybees, which uses RFID and Raspberry Pi camera recordings. Our preliminary experiments consisted of several tests aimed at identifying the optimal conditions for this system. First, two commercial RFID readers with antennas were compared in terms of their sensitivity to signals from RFID tags placed at various distances. We found that the UP16-1000-J2 reader was much more sensitive and had a longer effective range compared to the UP4-200-J2. The most sensitive region in the RFID antenna on the UP16-1000-J2 reader was 30 mm long and 5 mm wide at its center. Based on this preliminary information, we designed and built a passage from the interior of the observation hive to the outside so that all RFID-tagged bees could be detected individually by the RFID reader as they walked through the passage. Moreover, to detect the direction of either departure or arrival of each bee, we placed two RFID antennas under the passage between the observation hive and the outside, one near each end of the passage. All departure and arrival times of RFID-tagged bees were detected with their ID numbers. Using recorded data from these two RFID readers, we could measure how much time each tagged bee spent outside the hive. In addition to RFID recording on the passage, we also tracked all in-hive movements of numbered RFID-tagged honeybees. In-hive movements were simultaneously, comprehensively and automatically recorded via six Raspberry Pi camera modules arranged on the two sides of the observation hive. The cameras were set to record from 6:30 to 19:30 every day for one month, once or twice each year from 2015 to 2018. The in-hive behaviors of these bees were analyzed according to a simultaneous tracking algorithm that we developed for this purpose. Data from the monitoring system revealed that time spent outside the hive increased markedly after following the waggle dance. In addition to its findings on bee behavior, this study also confirms the effectiveness of our recording system combining RFID and Raspberry Pi cameras for honeybee lifelog monitoring.

This paper presents a seabird biologging system with a compact waterproof airflow sensor. Although biologging methods have attracted attention in the evaluation of seabird flight performance, a direct measurement method of airflow velocity has not yet been established. When an airflow sensor is added to a biologging system, a more accurate assessment of the flight performance can be obtained. We developed a compact Pitot tube-type airflow sensor that is specialized for seabird biologging systems. Here, we integrated micro electro mechanical system (MEMS) sensor chips and a sensing circuit into the Pitot tube housing. Then, we conducted a wind tunnel experiment using a stuffed seabird and the fabricated sensor. The results confirmed that the sensor responds to the wind speed even when attached to the dorsal surface of the seabird. Based on the above, we believe that the proposed sensor can be applied to practical seabird biologging systems.

The biologging approach of attaching a logger to the body of an animal provides information that cannot be obtained by conventional direct visual observation. Marine zoologists have used this technique for observing sperm whales preying on giant squids in the deep sea. However, it is almost impossible to capture a sperm whale to attach a logger, because of its large size. Therefore, researchers have used a long pole to attach a logger from a ship to the back of sperm whales. Unfortunately, this method is risky and requires a skilled team. In this paper, we propose a logger attaching system using a drone to solve this problem. The proposed method can be trained on land; thus, it is relatively easy to train a team, and the mobility of the drone can shorten the installation time. Several pieces of equipment developed for the proposed method are described in detail. Furthermore, field experiments were performed with sperm whales to confirm the feasibility of the system. A suction cup of the seventh prototype of the whale rover was adsorbed onto the back of a sperm whale. Although a complete installation was not possible, it was demonstrated that operation was possible in a short time using the proposed method.

A machine learning approach is investigated in this study to detect a finger tapping on a handheld surface, where the movement of the surface is observed visually; however, the tapping finger is not directly visible. A feature vector extracted from consecutive frames captured by a high-speed camera that observes a surface patch is input to a convolutional neural network to provide a prediction label indicating whether the surface is tapped within the sequence of consecutive frames (“tap”), the surface is still (“still”), or the surface is moved by hand (“move”). Receiver operating characteristics analysis on a binary discrimination of “tap” from the other two labels shows that true positive rates exceeding 97% are achieved when the false positive rate is fixed at 3%, although the generalization performance against different tapped objects or different ways of tapping is not satisfactory. An informal test where a heuristic post-processing filter is introduced suggests that the use of temporal history information should be considered for further improvements.

Insects have a sophisticated ability to navigate real environments. Virtual reality (VR) is a powerful tool for analyzing animal navigation in laboratory studies and is the most successful when used in the study of visually guided behaviors. However, the use of VR with non-visual sensory information, such as sound, on which nocturnal insects rely, for analyzing animal navigation has not been fully studied. We developed an auditory VR for the study of auditory navigation in crickets, Gryllus bimaculatus. The system consisted of a spherical treadmill on which a tethered female cricket walked. Sixteen speakers were placed around the cricket for auditory stimuli. The two optical mice attached to the treadmill measured the cricket’s locomotion, and the sound pressure and direction of the auditory stimuli were controlled at 100 Hz based on the position and heading of the cricket relative to a sound source in a virtual arena. We demonstrated that tethered female crickets selectively responded to the conspecific male calling song and localized the sound source in a virtual arena, which was similar to the behavior of freely walking crickets. Further combinations of our system with neurophysiological techniques will help understand the neural mechanisms for insect auditory navigation.

Understanding the principles of real-world biological multi-agent behaviors is a current challenge in various scientific and engineering fields. The rules regarding the real-world biological multi-agent behaviors such as those in team sports are often largely unknown due to their inherently higher-order interactions, cognition, and body dynamics. Estimation of the rules from data, i.e., via data-driven approaches such as machine learning, provides an effective way to analyze such behaviors. Although most data-driven models have non-linear structures and high predictive performances, it is sometimes hard to interpret them. This survey focuses on data-driven analysis for quantitative understanding of behaviors in invasion team sports such as basketball and football, and introduces two main approaches for understanding such multi-agent behaviors: (1) extracting easily interpretable features or rules from data and (2) generating and controlling behaviors in visually-understandable ways. The first approach involves the visualization of learned representations and the extraction of mathematical structures behind the behaviors. The second approach can be used to test hypotheses by simulating and controlling future and counterfactual behaviors. Lastly, the potential practical applications of extracted rules, features, and generated behaviors are discussed. These approaches can contribute to a better understanding of multi-agent behaviors in the real world.

We propose a method that uses ultrasound audio signals from a multichannel microphone array to estimate the positions of flying bats. The proposed model uses a deep convolutional neural network that takes multichannel signals as input and outputs the probability maps of the locations of bats. We present experimental results using two ultrasound audio clips of different bat species and show numerical simulations with synthetically generated sounds.

The study demonstrates the versatility of integration of inertial navigation and global navigation satellite system (GNSS) with its unique application to seabird biologging. Integrated navigation was originally developed in the field of aerospace engineering, which requires accurate and reliable position, velocity, and attitude information for the guidance and control of aircraft and spacecraft. Due to its high performance and recent progress of sensor development, integrated navigation has been widely used not only in aerospace but also in many fields represented by land and marine vehicles. One of its ultimate applications under the constraint on the size and power consumption of devices is this study. Seabird biologging involves attaching a logging device onto a seabird for scientific purposes to understand its biomechanics, behavior, and so on. Design restrictions for the device include several tens of grams mass, several tens of millimeters in length, and several tens of milliamperes of power consumption. It is more difficult to maintain the accuracy of such a device than applications to an artificial vehicle. This study has shown that integrated navigation is a feasible solution for such extreme applications with two examples: biologging for wandering albatrosses and great frigatebirds. Furthermore, it should be stressed that the navigation captured the world’s first data of their detailed trajectories and attitudes in their dynamic and thermal soarings. For completeness, the navigation algorithm, simulation results to show the effectiveness of the algorithm, and the logging devices attached to bird are also described.

Humans observe the actions of others and predict their movements slightly ahead of time in everyday life. Many studies have been conducted to automate such a prediction ability computationally using neural networks; however, they implicitly assumed that preliminary motions occurred before significant movements. In this study, we quantitatively investigate when and how long a preliminary motion appears in motions from static states and what kinds of motion can be predicted in principle. We consider this knowledge fundamental for movement prediction in interaction techniques. We examined preliminary motions of basic movements such as kicking and jumping, and confirmed the presence of preliminary motions by using them as inputs to a neural network. As a result, although we did not find preliminary motion for a hand-moving task, a left-right jumping task had the most preliminary motion, up to 0.4 s before the main movement.

We propose a pose estimation method using a National Advisory Committee for Aeronautics (NACA) airfoil model for fish schools. This method allows one to understand the state in which fish are swimming based on their posture and dynamic variations. Moreover, their collective behavior can be understood based on their posture changes. Therefore, fish pose is a crucial indicator for collective behavior analysis. We use the NACA model to represent the fish posture; this enables more accurate tracking and movement prediction owing to the capability of the model in describing posture dynamics. To fit the model to video data, we first adopt the DeepLabCut toolbox to detect body parts (i.e., head, center, and tail fin) in an image sequence. Subsequently, we apply a particle filter to fit a set of parameters from the NACA model. The results from DeepLabCut, i.e., three points on a fish body, are used to adjust the components of the state vector. This enables more reliable estimation results to be obtained when the speed and direction of the fish change abruptly. Experimental results using both simulation data and real video data demonstrate that the proposed method provides good results, including when rapid changes occur in the swimming direction.

Echolocating bats perceive the surrounding environment by processing echoes of their ultrasound emissions. Echolocation enables bats to avoid colliding with external objects in complete darkness. In this study, we sought to develop a method for measuring the collective behavior of echolocating bats (Miniopterus fuliginosus) emerging from their roost cave using high-sensitivity stereo-camera recording. First, we developed an experimental system to reconstruct the three-dimensional (3D) flight trajectories of bats emerging from the roost for nightly foraging. Next, we developed a method to automatically track the 3D flight paths of individual bats so that quantitative estimation of the population in proportion to the behavioral classification could be conducted. Because the classification of behavior and the estimation of population size are ecologically important indices, the method established in this study will enable quantitative investigation of how individual bats efficiently leave the roost while avoiding colliding with each other during group movement and how the group behavior of bats changes according to weather and environmental conditions. Such high-precision detection and tracking will contribute to the elucidation of the algorithm of group behavior control in creatures that move in groups together in three dimensions, such as birds.

Echolocating bats perceive their surroundings by listening to the echoes of self-generated ultrasound pulses. When multiple conspecifics fly in close proximity to each other, sounds emitted from nearby individuals could mutually interfere with echo reception. Many studies suggest that bats employ frequency shifts to avoid spectral overlap of pulses with other bats. Technical constraints in recording technology have made it challenging to capture subtle changes in the pulse characteristics of bat calls. Therefore, how bats change their behavior to extract their own echoes in the context of acoustic interference remains unclear. Also, to our best knowledge, no studies have investigated whether individual flight paths change when other bats are present, although movements likely reduce acoustic masking. Here, we recorded the echolocation pulses of bats flying alone or in pairs using telemetry microphones. Flight trajectories were also reconstructed using stereo camera recordings. We found no clear tendency to broaden individual differences in the acoustic characteristics of pulses emitted by pairs of bats compared to bats flying alone. However, some bats showed changes in pulse characteristics when in pairs, which suggests that bats can recognize their own calls based on the initial differences in call characteristics between individuals. In addition, we found that the paired bats spend more time flying in the same directions than in the opposite directions. Besides, we found that the flight paths of bats were more similar in “paired flight trials” than in virtual pairs of paired flight trials. Our results suggest that the bats tend to follow the other bat in paired flight. For the following bat, acoustic interference may be reduced, while the opportunity to eavesdrop on other bats’ calls may be increased.

Dogs are the oldest domesticated animals. The process of domestication of dogs is still unclear; however, they have established themselves as human partners and are sometimes more cooperative with humans than their conspecifics. In this study, to determine the effect of affiliative human presence on group behavior in dogs, we conducted short-time trials analyzing dog group movements. There was a hierarchical relationship in which juvenile dogs were aware of adult dogs, and adult dogs were aware of human movements. We also found that the age of the juvenile dog and the characteristics of their mothers may affect the movement behavior of juvenile dogs.

In this study, we investigated the relationship between the activity rhythms of Camponotus japonicus worker ants and their interactions. Specifically, one or two workers collected from either inside or outside the nest in a breeding colony were placed in a measurement system under a constant dark condition, and their activity rhythms were measured for 14 days. We thereby examined the relationship between the activity rhythm in the system and the experimental conditions, which consisted of four different combinations of working locations during breeding (in/outside the nest) and single/double workers (one ant / two ants) in the measurement system, over a total of 96 samples. A large number of the sampled ants (about 90% of the total) showed circadian activity rhythms. The proportion of circadian activity rhythm was lower and the dispersion of the period was larger in the circadian activity rhythm observed in single workers collected from within the nest than in the other three experimental conditions. In all four experimental conditions, the amplitude of the circadian activity rhythm decayed on an approximate 5-day scale. These results provide quantitative evidence that the activity rhythm of ants is determined by the location of labor and individual interactions during breeding.

Osaka University and Komatsu Ltd. started a joint research project in 2006 as an industry-academia collaboration activity, and Komatsu established a cooperative research center on Osaka University campus in 2015. The center has conducted joint research to solve the problems of companies as well as independent research on remote and autonomous construction machinery. Since 2017, it has been working on “HENNA” construction equipment (“HENNA” in Japanese means “novel and innovative with unconventional thinking”) as a new research idea that combines the academic nature of Osaka University with the corporate nature of Komatsu. These research initiatives and the concept of the Cooperative Research Center (2017–present) are presented here.

A major challenge in remote control is the reduction in work efficiency compared with on-board operation. The factors of reduction in work efficiency include a lack of information (information such as perspective, realistic sensation, vibration, and sound) compared to on-board operations. One of the factors is the lack of vestibular/somatosensory information regarding rotation. To clarify the effect of the presence of input of vestibular/somatosensory information regarding rotation on the worker’s operation, we conducted a basic laboratory experiment of a horizontal turning operation. The experimental results indicate that a response appropriate for the input of information regarding rotation can be made only with visual information; however, the reaction is delayed in the case without the input of information regarding rotation in comparison with a case with the input of information regarding rotation.

The teleoperation of construction machinery has been introduced to mines and disaster sites. However, the work efficiency of teleoperations is lower than that of onboard operations owing to limitations in the viewing angle and insufficient depth information. To solve these problems and realize effective teleoperations, the Komatsu MIRAI Construction Equipment Cooperative Research Center is developing the next-generation teleoperation cockpit. In this study, we develop a display for teleoperations with a wide field-of-view, a portable projection screen, and a system that reproduces motion parallax, which is suitable for depth perception in the operating range of construction machinery.

This paper describes FST-Convoy, a leader tracking control system for vehicles using the shape sensor flexible sensor tube (FST). Among many methods of autonomous driving, follow-driving is one of them. Some of these have been put into practical use in a limited environment. Unfortunately, there are situations in which such sensors do not work well. One of these is underground. In the underground, GNSS signals do not reach vehicles, so they cannot obtain their positions. Therefore, we propose a new way to achieve follow-driving in such environments. We used the shape sensor, FST. The FST is a shape sensor with a serial link structure and many joints. It can measure its shape by solving its kinematics and determine the relative position of the start link to the end link. Therefore, we can measure the relative positions of vehicles that connected a leader and a follower using FST. We call this system FST-Convoy. We developed and verified the system using a platooning-driving experiment.

Muscle contractions (or equivalent mechanical elements) are responsible for joint movement in systems with musculoskeletal structure. Because muscles can only transmit force in the tensile direction in such systems, the internal force exists between the muscles. By utilizing the potential field generated by the internal force, the musculoskeletal potential method makes it possible to control the position without complex real-time calculations or sensory feedback by entering step-inputs of the balanced internal force at the target posture. However, the conditions of convergence to the target posture strongly depend on muscular arrangement. Previous studies have elucidated the mathematical conditions of the muscular arrangement; however, they provide sufficient conditions that must be satisfied by the muscular arrangement to converge to the target posture, which do not necessarily lead to optimal muscular arrangement conditions. This study proposes a method to determine the optimal muscular arrangement of a two-joint six-muscle system, wherein muscle viscosity is considered, that uses a genetic algorithm and an evaluation function considering the motion response time. The effect of the obtained muscular arrangement is verified in a simulation.

Predictive maintenance, which means detection of failure ahead of time, is one of the pillars of Industry 4.0. An effective method for this technique is to track early signs of degradation before failure occurs. This paper presents an innovative failure predictive scheme for machines. The proposed scheme combines the use of the full spectrum of vibration data from the machines and a data visualization technology. This scheme requires no training data and can be started quickly after installation. First, we proposed to use the full spectrum (as high-dimensional data vectors) with no cropping and no complex feature extraction and to visualize the data behavior by mapping the high-dimensional vectors into a two-dimensional (2D) map. This ensures simplicity of the process and less possibility of overlooking important information as well as provide a human-friendly and human-understandable output. Second, we developed a real-time data tracker that can predict failure at an appropriate time with sufficient allowance for maintenance by plotting real-time frequency spectrum data of the target machine on a 2D map created from normal data. Finally, we verified our proposal using vibration data of bearings from real-world test-to-failure measurements obtained from the IMS dataset.

In addition to the declining birthrate and aging population in Japan, there has been a recent decrease in its total population. This threatens to exacerbate a shortage in labor force, which could trigger an increase in the luggage transport costs of transportation companies or the service industry. The demand for inverted two-wheeled luggage transport vehicles has been increasing steadily owing to their high mobility, compactness, affordability, and pivotal turnability. However, owing to their statical instability, these vehicles are limited. Accordingly, stability can be improved in these systems by configuring a spreading system and applying a disturbance observer based on a Kalman filter. The application of a Kalman filter enables us to estimate the disturbance in which the error between the true and estimated values is the least mean square. Furthermore, we validated the efficiency of the proposed method via its translational movement, turning angle control, and load-loading/unloading experiments using various loads.

We have developed a pneumatic artificial rubber muscle having a bending direction that can be changed using two shape-memory polymer (SMP) sheets, the stiffness of which depends on the temperature. In the present study, we attached two SMP sheets with embedded electrical heating wires to both sides of a pneumatic artificial rubber muscle in order to realize multidirectional actuation and evaluated the basic characteristics of the artificial muscle. The actuator is based on the design of a conventional curved-type artificial rubber muscle. Since only a heated SMP sheet becomes soft, the rigid SMP sheet inhibits the extension of the side of the actuator. Therefore, bending motion can be induced when air is supplied to the internal bladder. By controlling the temperature of the SMP sheets, the bending direction of the prototype actuator could be changed. Namely, three kinds of motions, such as two-directional bending and axial extension, became possible. Moreover, we improved the manufacturing method and the structure of the artificial muscle, such as the stitching method and the SMP sheet thickness, and evaluated the characteristics of the two-directional bending and the axial extension motions of the prototype actuator. We also calculated the theoretical values and compared these values with the experimental results. Furthermore, we examined the application of the actuators to a robot hand. Using the two-directional motion of the actuator, the proposed robot hand can grasp either small or large objects. The experimental results conducted using this prototype confirm the feasibility of the newly proposed actuator.

The authors developed a hexapod tracked mobile robot: a tracked mobile robot which is equipped with six legs attached to the robot’s body. In a transportation task, this robot can traverse a wide gap by supporting track driving with four front and rear legs while holding the target object with its two middle legs. To realize autonomous actions with this robot, we developed a two-dimensional distance measurement system using an infrared sensor. This system is very simple, with the sensor attached to a servomotor, such that it does not require high computing power for measurement. In addition, the system can be equipped at a lower cost than laser range finders and depth cameras. This paper describes the selection of the gap traversing mode according to gap width detected by the system. In this study, we conducted a gap width detection experiment and an autonomous gap traversing experiment using the hexapod tracked mobile robot with the proposed system. The obtained results confirm the effectiveness of the proposed system and autonomous traversing, which corresponds with the gap width detection.

To achieve good rehabilitation in a person, the amount of walking by the person must be increased. Herein, a compact wheeled gait-training walker with dual-assist arms for assisting pelvic motion is proposed. The training walker is constructed by modifying a commercial wheeled walker with armrests. Therefore, it can be used easily by patients to perform their daily activities at rehabilitation sites. The hardware system and controller of the proposed assisting arms are designed based on gait-assist motions conducted by a physical therapist. The dual arms can achieve a pelvis-assisting motion with five degrees of freedom. A trajectory-following control with virtual compliance is implemented for the arms. Gait-assisting experiments are conducted, in which the dual arms allow a pelvic-like plate to follow the trajectory of a reference pose while reducing the upper body’s weight resting on the armrests. A 20 N force on the armrests, which represents the upper-limb load, is reduced while the plate follows the trajectory, and the proposed gait-assisting controller is validated.

The perception of the surrounding circumstances is an essential task for fully autonomous driving systems, but its high computational and network loads typically impede a single host machine from taking charge of the systems. Decentralized processing is a candidate to decrease such loads; however, it has not been clear that this approach fulfills the requirements of onboard systems, including low latency and low power consumption. Embedded oriented graphics processing units (GPUs) are attracting great interest because they provide massively parallel computation capacity with lower power consumption compared to traditional GPUs. This study explored the effects of decentralized processing on autonomous driving using embedded oriented GPUs as decentralized units. We implemented a prototype system that off-loaded image-based object detection tasks onto embedded oriented GPUs to clarify the effects of decentralized processing. The results of experimental evaluation demonstrated that decentralized processing and network quantization achieved approximately 27 ms delay between the feeding of an image and the arrival of detection results to the host as well as approximately 7 W power consumption on each GPU and network load degradation in orders of magnitude. Judging from these results, we concluded that decentralized processing could be a promising approach to decrease processing latency, network load, and power consumption toward the deployment of autonomous driving systems.