The goal of this research is to realize an efficient collaborative object search by a human a robot. For efficient collaborative object search, it is important for the robot to know where the human has searched or will search and share the search areas among them. Human search behavior is estimated by matching the travel time calculated by FMM with the actual elapsed time. In the simulation, it is confirmed that the developed method reduces the amount of duplicated search and enables efficient search.

In order that people are willing to accept life-support robots in their life, we would like to understand the reason why they refuse such robots. We hypothesize that the refusal is caused by the fear of the robots. To prove this hypothesis, we conduct a questionnaire, lecture, and group discussion to home-economics, science, and robotics students. In the questionnaire, we ask students to estimate the applicability of the robots to various occupations. We guess the over-estimation or under-estimation of the robots’ performances leads to fear. Following the questionnaire, we arrange the group discussion. In this paper, we will report five elements related to the fear of robots that are obtained by the aforementioned activity. These are robots’ appearance, communication ability, human-robot relationship, knowledge of robots, and unfamiliarity with robots.

We are developing an anthropomorphic robot that can play the violin that can communicate with humans through its expressive performance. Violin performance can be divided into two tasks: bowing and fingering. This report introduces a new right hand for bowing. Previously, the bow was rigidly fixed to the right hand, which causes the vibration of the bow during the bowing. The vibration alters the bow force, which degrades the quality of the produced sounds. Thus, the new right hand that can reduce the vibration must be developed to produce sounds with good quality. We fabricated a new right hand with a soft material and confirmed that it can reduce the alternation of bow force.

This report presents the design and software of the left hand for our violin-playing robot, and its experimental results. The hand has four fingers with two active joints actuated by RC servo motors. We redesigned the hand so that it can be placed under the neck of the violin. The distance between the adjacent fingers can be adjusted so that each finger can push the appropriate positions on the strings to produce sounds with the correct pitch. We conducted frequency analysis on the produced sound and confirmed that the hand can produce sounds with the correct pitch. We then investigated the delay time of the finger motion and found that the average delay time is approximately 0.0587 s.

Rehabilitation and training need to be performed in an appropriate posture, and if performed in an inappropriate posture, it may not be possible to effectively stimulate the target muscles. Furthermore, depending on the posture, there is a risk of applying unnecessary load to specific muscles and tendons. For this reason, in order to effectively stimulate only the target muscles, it is necessary to control the system in consideration of the posture during training. However, in general, it is conceivable to use guidance from a specialized trainer as a method of correcting posture during training, but the cost is high, and it will be particularly remarkable in the future declining birthrate and aging society. Therefore, by examining the feedback method of the evaluation result, the trainee can correct it in real time and train himself in an appropriate posture.

In these days, the number of users of handle type electric wheelchairs called senior car has increased. However, senior car has a limited range of mobility due to physical obstacles like steps. Further, a step cause psychological stress such as fear and anxiety to user. This study target on senior car motion surmounting step and the user. First, this study examines senior car motion surmounting step by impact force measurement experiment and estimates psychological stress of senior car user. Then, we propose mechanism for surmounting step include results of the experiment. The mechanism enables to provide time lags in the impact force. Finally, we measure the impact force using proposal mechanism and estimate less psychological stress.

The purpose of this research is to detect ‘off phenomenon’ that occurs when a medicine for Parkinson's disease is no longer effective. We focus on the periodicity of gait in patients with Parkinson's disease. A patient is requested to attach a simple device equipped with an accelerometer, and acceleration time series during walking will be collected. We employ LSTM to detect walking abnormalities. In our preliminary experiments, we attempted to detect anomalies by LSTM in eight healthy young people, where the persons are requested to mimic typical performance of off phenomenon. As a result, Mahalanobis distance, which indicates the degree of abnormality, changed, so that the anomaly could be detected. The next problem is how to detect walking phases from accelerometer’s time series observed from patients. To investigate whether usual walking phase detection algorithms are able to be applied, we collected walking date for 60 seconds from two Parkinson's disease patients. As a result, the patients with Hoehn & Yahr severity III had a disordered periodicity, which indicates we have to make a new walking pattern identification algorithm for Parkinson’s patients.

In this study, the driving force of a cycling wheelchair user is evaluated by multiple seat position. The cycling wheelchair is a device that combines both rehabilitation and mobility for people with lower limb disabilities. However, the cycling wheelchair has some structural problems. For example, the center of gravity is not optimized and the crank hinders the user from getting in and out of the wheelchair. Therefore, we focus on the position and posture of the seat to solve this problem. At first, we confirm the method of estimating the joint torque to be used for evaluation.Secondly, we design the seat position changing device and decide the experimental condition. Finally, we conduct the experiment that measure the joint torque of three participant under the experimal condition, and the results were discussed.

Currently, the number of elderly people in Japan is on the rise, and it is expected that the demand for welfare equipment will increase. Among them, electric wheelchairs are especially widely used. However, many of the electric wheelchairs in use are blocked and it is difficult to work with them. In addition, since the wheelchair user basically does not move his / her own body, there are drawbacks such as a decrease in physical strength and aggravation of dementia. In this study, we developed a device that indicates the direction of travel by the pressure difference of the air cushion placed in the seat of the wheelchair according to the body movement of the passenger and confirmed its effectiveness by attaching it to an electric wheelchair.

Hearing impaired people have a handicap for blind spots such as behind, and as an example, they often encounter danger because they cannot recognize a car approaching from behind while honking their horn. In this situation, the driver is alerted and the hearing impaired cannot hear it, and the gap between consciousness and cognition increases the risk. In this study, we report the development of a device that senses warning sounds, recognizes the surrounding environment by mechanical vibration transmission, and supports hearing handicap as support for people with hearing impairments.

We have developed an intraoral input device using tongue. This device contains three photo-reflectors in a balloon of silicone rubber and it can measure the quantity and direction of the pushing force of silicone rubber film. By using this device, we can make a controller that generates information by the tongue. We have applied this device to control the robotic arm for the third arm for the soldering operator. We also applied it for the wheelchair controller. The device has very good operability for both applications. This forward pushing type intraoral input device, however, has the disadvantage that the tongue tends to get tired during use. In this paper, we developed a upward pushing type intraoral input device to reduce the fatigue. The first prototype had the drawback of very poor controllability because we push the whole silicone balloon and the detection of the pushing direction is very difficult. In the improved prototype device, the area of the reflector attached to the inner wall is reduced so that the movement of the balloon-shaped silicone rubber in the front-back and left-right directions by the movement of the tongue can be detected. This upward pushing type input device has same controllability to the forward pushing type with less fatigue.

For the starting of Programming education in elementary school from FY2020, it is necessary to keep the people who has skills of technical programming, education and volunteer for children. These kinds of human resources should be ideally organized from the people who lives in the local area around the school. Authors have been conducted cultivation this kind of volunteers from 2018, almost 20 senior members were joined and they are learning for several techniques of programming and its thinking method. Under the COVID-19 disease, the courses of cultivation of volunteers for programming have been also forced to change the learning method to online. The activities of our courses have been reported, then the problems and its solutions have been discussed.

In this research, we developed a walking assist robot for supporting user’s body weight and pelvis rotation simultaneously during overground walking training. We utilized load cells to instrument a pair of instrumented shoes to measure ground reaction force signals and utilized IMUs to instrument a climbing harness to measure transverse pelvic rotation. We purpose Static Body Weight Support (SBWS) method to support user’s body weight and Static Lateral Pelvis Trajectory Support (SLPTS) method to support user’s lateral pelvic rotation. We conducted the body weight support walking experiment to validate the basic design of our system. The result shows that user’s ground reaction force signals decreased accordingly under both two methods. Especially, under the SBWS+SLPTS method, lateral pelvis rotation shows a larger range of motion, which is more similar to natural walking.

In this paper, in order to evaluate the plane detection algorithm for detecting the edge of a station platform, we verified the algorithm by running it on the 3D distance data measured at the actual station platform. As the result, we were able to detect the plane with the error that is small enough to distinguish it from steps such as stairs. However, the correct answer rate and the true negative rate were greatly reduced by the distant point clouds and the movement of the sensor itself, so it is necessary to remove the distant point clouds and to reduce the error caused by the movement of the sensor itself in the current algorithm. In the future, we will verify the algorithm in other situations, and implement and verify additional processing.

It is severe problem that population ratio of elderly people is increasing in Japan. The walker is useful for walking assistance and rehabilitation, however, conventional walkers can’t move its handle so that movement of trainee’s spine is restricted and trainee’s trunk becomes falling forward. T herefore, we have been developing a walker which has novel mechanism. Handle of this new walker is movable so that training with the walker can facilitate trainee’s natural spinal rotation. In this paper, we describe mechanical improvements of the walker to secure sufficient handle-ability in rehabilitation.

Recently, many types of Functional Electrical Stimulation (FES) devices have been developed for rehabilitation and neural prosthesis. The effect on human motion by changing the waveform pattern of electrical stimulation was evaluated for suitable parameters for the precise control of MP joint of index finger, In this study, we use a combination of square wave waveforms and then we focused on the stimulation period, stimulation amplitude, and pulse width. The results showed that the stimulus amplitude and pulse width had a strong linear correlation with MP joint angle increase. However, the stimulation period may have nonlinearity. These results suggest that the voltage and pulse width are possible parameters for the precious human hand motion control with FES.

In Japan, elderly people certified as requiring long-term care can rent 13 categories of the welfare devices by using the long-term care insurance system. In addition, some robotic devices for nursing care have been covered by the insurance. The advantage of this system is that users of this system can rent welfare devices on a semi-monthly or monthly basis instead of purchasing welfare devices. Therefore, it is easy for users to change devices. The number of walker users under this system is increasing rapidly. In considering the spread of robotic devices for nursing care, it is important to understand the usage status of walker users. In this paper, we utilize the nation-wide receipt data of long-term care insurance and analyzed the status of walker types and model changes.

We propose a new type of walking support using "light touch effect" with a drone. The purpose of this research is to stabilize the user's gait by generating a "light touch effect" using a small drone. As a first step, we verified that the light touch effect is generated by floating objects. Using balloons, we measured whether the sway of the center of gravity during walking was reduced. As a result, it was confirmed that the stability of the center of gravity sway was improved. In the next stage of the research, we will verify whether the gravity center oscillation is stabilized when walking with a small drone and climbing up and down small stairs.

We have been discussing a disaster information gathering system using mobile robot. There is a problem that an accident occurs on the robot in case of disconnecting the wireless LAN between the operator and robot. Then, we proposed wireless mobile robot tele-operation system using LoRaWAN as sub-communication infrastructure to solve the accident. LoRaWAN is superior to wireless LAN in communication distance, but inferior to bit rate. So, it takes a lot of time to transmit a large amount of data. In this paper, we describe the operation method using LoRaWAN as sub-communication infrastructure of ad-hoc network model without using large amount of data to solve the accident.

We have been discussing a disaster information gathering system using mobile robot. There is a problem that an accident occurs on the robot in case of disconnecting the wireless LAN between the operator and robot. Then, we proposed wireless mobile robot tele-operation system using LoRaWAN as sub-communication infrastructure to solve the accident. LoRaWAN is superior to wireless LAN in communication distance, but inferior to bit rate. So, the image transmission time increases. In this paper, we describe a method for shorting the image transmission time by multi path communication system using LoRaWAN in order to improve the operability of mobile robot.

In this study, we developed the function of robot teleoperation by applying the RSNP unit developed in our previous study, and conducted experiments using 12 robots. The teleoperation can be performed on any robot from a common screen, and we believe that we were able to introduce RSNP communication by connecting to the unit in various ways. However, the operation method was not good because there was no feedback of the operation amount on the operation screen. We will continue to improve the remote control system to make it easier for the operator to operate the system.

We study the automatic tracking function for the optical axis of a beam. This system can be used for movable nodes for sensor network with Free Space Optics. However, there is a problem that it doesn’t follow stably when the beam is about to deviate from the angle of view. We propose a method of recognizing a beam by adjusting the target color when camera gain changes. But this automatic tracking system doesn’t have a camera gain acquisition function, it was estimated from the edge data of the image that it is related to the camera gain. In this paper, we discuss the detail of the above study.

This paper describes an information gathering system comprising multiple mobile robots and a wireless sensor network (WSN). We have been discussing communication quality between an operator and the multi-robot for tele-operation of mobile robot in the information gathering activity in the closed space. In operation multi-robot, a mobile robot carries wireless relay nodes and deploy them into the environment. After construction the network, multi-robot connect to the network and explore. An operator then controls the multi-robot remotely while monitoring end-to-end communication quality with all mobile robot. In this paper, we propose a method that measuring end-to-end throughput with multi-robot. The validity of the proposal method was inspected by doing an evaluation experiment of multi-robot tele-operation.

Service robots are expected to be used in many situations, such as homes and factories, instead of people. However, it takes a lot of effort to teach the robots how to perform behaviors which human have been doing. To expand the use of service robots, it is necessary to develop the support system for teaching robots; the system should easily transfer the behaviors from human to robots. In this paper, we propose a teaching system by extracting hand-object interaction from first-person view videos acquired by a camera attached to a human’s head. The proposed system generates a sequence of hand motions by extracting three types of simple motion elements (Translate, Rotate, and Grasp) from first-person view video as teaching information to a robot. In the experiment, we confirm the usefulness of this system by actually having a person perform the pouring task as an example, extracting the motion element sequence from the first-person view video, and reconstructing the robot’s pouring motion using it.

In this study, we analyzed experimenters’motional characteristics that lead to uniform cell seeding. The motion to seed cell was measured using a motion capture system for three novices who were taught to execute stirring by ”a rotational movement” of a multi-well plate and four experients who were asked to seed as usual. Skill indices were defined as a feature of the seeding motion. Cell uniformity was evaluated as an averaged intensity variation from binarized microscope cell images. The correlation analysis between each skill index and uniformity index showed that the stirring movement of the plate was strongly related to uniformity of the seeded cell. Significant difference between the novice group and the experient group was observed in uniformity index and several skill indices in stirring process. These results indicated that teaching rotational stirring to a novice worked to guarantee uniform cell seeding to a certain extent.

Considering the growing market size for universal design product, it is helpful if we can evaluate its usability virtually by synthesizing feasible grasp postures for a digital hand model. In this paper, we focus on the elderly people. We generate grasp postures by using a digital hand that emulates elderly hands. Generally, range of motion (ROM), muscular force, and frictional force are decreased for elderly hands. So we determine the proper rate of decrease in these features and implement them in the digital hand for grasp synthesis.

A single-camera based gait analysis system without markers, large space, and preparation can provide useful information for determining effective treatment in clinical sites. However, little is known about the effectiveness in gait analysis. This study investigated the accuracy of single-camera(the right side located) based pose estimation system using OpenPose (OP) measuring the lower limb joint angle and spatio-temporal parameters in both lower limbs during the normal gait and large foot progression angle (FPA) gait. Nine healthy adult males walked under two different foot progression angle FPA conditions (normal and large FPA). To validate the accuracy of the system in both conditions, Mean Absolute error (MAE) and Mean Absolute Percentage Error (MAPE) was calculated. Although the MAE of ankle joint angle is larger than 10 degrees and an additional device is necessary to evaluate for the large FPA gait, the MAE of all joint angle in normal conditions and MAPE of spatio-temporal parameters in both conditions are acceptable.

The paper describes a method for predicting 3D pose of a person in real-time with RGB-D camera. Three dimensional pose of a person is measured by extracted 2D skeleton on the image and corresponding depth. Motion predictor of RNN is trained by measured 3D pose of a person. 3D pose of a person after several frames is predicted in real-time by inputting time-sequential 3D pose of a person to RNN. Predicted error is evaluated in the experiment.

In recent years, shortage of skill successors for traditional crafts being serious problem, preservation of their skills as inheritable knowledge is highly demanded. In this study, our objective is to clear relationship between appearance quality of work and craftsman’s motion as skill. In particular, we focus on knitting work with needles, which is a common craft work. We propose a index of appearance quality, and analyzed quantitively the knitting pattern by using the index. Furthermore, we surveyed relationship between the knitting pattern and reaction force applied to index finger of left-hand during pull motion.

In this paper，we present a video motion capture system for sport athletes．Although marker-less video motion capture is suitable for sports motion analysis，the 2D human pose estimation used in the system often fails in estimating postures in sports．This is because those postures are rarely observed in daily life．In the proposed method，we integrate the 2D human pose estimation with an end-joint estimation，which is robust to postures in sports．We validate the proposed method measuring a kick motion of a taekwondo player．

The leading cause of child death is unintentional injuries due to accidents. Currently, to prevent accidents, parents need to understand the injuries associated with each child's developmental stage, however, the way of precise grasping of the developmental stage is not yet established. The purpose of this study is to develop a system that automatically and directly estimates the developmental stage of children and indicates danger based on it. In this paper, as its basis, we propose a new technology that estimates the developmental stage in three steps: (1) Estimation of the developmental stage from children behavior observation, (2) recording of the numbers of the behavior occurrences, and (3) statistical analysis of behavior distributions. We demonstrated the feasibility of the proposed method to verify accuracy of behavior recognition, and investigate changes in behavior distribution with the development of children. The results showed that the behavior estimation had an accuracy of 89% and the behavior distribution changed clearly, indicating that the three-step estimation is useful.

This paper proposes motion planning in low-dimensional space, so-called latent state space, embedded by variational autoencoder. We hypothesize that the path passing latent states with high likelihood works as motion planning. Assuming latent states follow the normal standard distribution, finding the path can be seen as a two-point boundary value problem of an ordinary differential equation. Then, the proposed method makes an agent follow the path by a policy pre-trained by reinforcement learning. We evaluated the proposed method in simple maze environments and found that the proposed method works well for long-horizon tasks and has a low computational cost.

The goal of this research is to acquire dribbling behavior of a soccer robot using deep reinforcement learning. We constructed a simulated soccer field environment, in which a wheeled robot successfully learned to dribble by itself. We also tackle the issue of observation uncertainty, which is one of the challenges in applying reinforcement learning to real environments. In this paper two settings are considered: 1) the robot needs to observe landmarks and estimate self-position, 2) the field of view of the ball observation is limited. We observed significant performance drop under the condition that the robot’s field of view is limited.

In this paper, we explain the effectiveness of the dynamic gesture recognition method using static images with Motion History Image (MHI) and an object detection algorithm. As the object detection algorithm, we use YOLOv3 in this study. For efficient learning and recognition of the dynamic gesture with reciprocating motion, the time-domain division of the motion is used. Two experiments were conducted to evaluate the effectiveness of the proposed method. In the first experiment, four dynamic gestures ware detected using proposed method, and the confidence score was 97.95%. In the second experiment, we compared the performance of proposed method with other methods. Finally, we confirmed the detection accuracy as good as other methods.

We propose a ROS (Robot Operating System) based reinforcement learning system for mobile robots. Previous ROS based reinforcement learning systems have a problem that a mobile robot state does not change by the mobile robot action. For example, if the mobile robot state is an image, learning the mobile robot action is difficult because the image does not change though the mobile robot action changes. To address the problem, the proposed system returns a reward and a next state when a mobile robot action, for example linear and angular velocity, is done by observing the wheel odometry of the mobile robot. We evaluate the proposed system in the simple visual navigation task that is implemented in ROS and Gazebo. Experiment results show that the proposed system works well and improves sample efficiency.

We consider how to control behaviors of the sensor’s perceptual policy during the training in our earlier work that simultaneously learns the robot’s movement policy and the sensor’s perceptual policy. In our algorithm, we simultaneously train two agents which control the robot and its sensor on the robot to achieve a task. Since the exploration space changes greatly depending on the action of the sensor, we aim to achieve efficient exploration behavior by introducing a new parameter to control the sensor action in a training process. We conducted experiments on navigation tasks in a 2D and a 3D environment. The experimental results show that our algorithm can achieve higher success rates than conventional reinforcement learning algorithms and we found that the introduced parameters worked effectively in controlling the behavior of the sensor.

Imitation learning is one of the techniques for visual navigation in robotics. This technique enables robot to visual navigation by human demonstrations. However, in a different environment from demonstrations, evaluation of visual navigation based on a learned model often fail because domain that is distribution of dataset between demonstration data and test data is difference. In meta-learning, the goal of the trained model is to learn a new task from a few shot data. We apply a new task as a new domain in meta-learning to imitation learning of visual navigation. We evaluate not only meta-learning but also domain adaptation to visual navigation by imitation learning and clarify the problem.

In the future, it is expected that robots will collaborate with humans. Such robots for collaborative works need to perform a variety of movements. Therefore, it is necessary to develop a system that can efficiently generate various movements from limited training datasets. To develop such a system, we focus on latent variables that represent features of the movements. This study proposes an encoder-decoder model, a kind of neural network, that generates a walking motion for a legged robot from the center-of-mass trajectory through latent variables. By directly changing the latent variable, we were able to let the legged robot walk at a different velocity. The experimental results suggest that the proposed system can generate walking motions with various velocities from learning a limited variety of movements.

In a conventional method of constructing a 3D-map from RGB-D images, we find a set of corresponding points in every two images, and find a frame translation between them, and merge them. However, in actual robot workspaces such as plants and office buildings, we frequently find an almost uniform workspace with less textured objects such as corridors and halls, where we cannot find the corresponding points from images. On the other hand, because it is an artificial environment, we can put artificial makers such as QR codes, for making a robot friendly environment. In this paper, we present an RGB-D image merging method by finding the corresponding points from visual features of artificial markers and investigate a position accuracy of merged 3D-maps.

In this paper, we present a learning-based method for estimating the state of a cloth product on basis of its optical flow during a dressing process. Using this method, we realize the following capabilities: (1) handling of many different kinds of fabric products, (2) detection of fabric manipulation failures (e.g. cloth snagging), (3) fabric manipulation with complex procedures, and (4) online state estimation. We evaluate state estimation accuracy in two dressing experiments: pulling a plain cylindrical cloth onto an L-shaped stand, and pulling a long-sleeved dress onto the arm of a doll. The experimental results confirm that our method can estimate the state of the fabric during the dressing process more accurately than conventional methods.

Snake-like robots are expected to be utilized for disaster rescue because they can locomote through gaps that humans cannot go through by appropriately coordinating the large degrees of freedom (DoFs) in their bodies. However, when learning locomotion using reinforcement learning (RL), an increase in learning time due to the large degrees of actions based on the bodily DoFs is a serious issue. To solve this, inspired by biomechanics on snake locomotion, we propose a reduction method for actions on model-based RL. In this paper, we verify the validity of the proposed method using an actual snake-like robot.

In order for the water purification system to operate efficiently in aquariums according to the real-time water conditions, it is necessary to automatically judge the transparency of water. We trained a convolutional neural network (CNN) model to judge the water transparency from camera images of an aquarium. The images of the aquarium are classified into three categories with respect to the water transparency: clear, little turbid, and turbid. The CNN model was trained to judge the water transparency from the grayscale images converted from the original RGB images. Cross validation within the collected dataset of 48 days was performed to evaluate the CNN's performance, and it was shown that, although the accuracy for the images at transition periods of transparency was slightly low, the trained CNN was able to judge the water transparency with high accuracy.

In recent years, breeding programs for the endangered Tsushima leopard cat are being conducted in various parts of Japan. However, there is a risk of fighting if males and females are allowed to live together during the nonestrus period. Therefore, it is necessary to detect the signs of estrus in advance in order to prevent injuries caused by fighting in the Tsushima leopard cat. However, currently, this is done manually by zookeepers, which requires a great deal of labor. In this study, we aim to automatically recognize the signs of estrus in the Tsushima cat with the cooperation of zoos. We extend the two-stream I3D to detect the estrus behavior from surveillance videos. Our results show that the estrus sign of Tsushima leopard cat can be effectively detected by using I3D.

Recently, autonomous mobile robots based on action models trained by deep reinforcement learning have been studied. Generally, it is desirable to train action models in the environment simulating the actual environment. However, it takes many steps to complete training and acquire an action model adaptive to the target environment. A purpose of this study is to acquire an action model adaptive to each environment in short steps by transfer learning from generic action models. The generic action models are acquired by training in environments containing various artificial shapes. An adaptive action models to each target environment can be obtained in shorter steps based on fine tuning of the generic action models.

The multi-degree-of-freedom disaster response robot ‘OCTOPUS’ requires an ‘environment-adaptive motion generation function’ to fully utilize its advanced ‘body.’ The disaster site is unknown and complex, so there is a limit to the amount of control rules that can be described in advance by humans. To address this issue, we have developed a basic framework for an adaptive reinforcement system that learns behaviors in a virtual space generated based on environmental information acquired in a real space. In this study, we develop a motion learning system based on task classification and reuse of learned control rules. As a result of experiments on rough terrain, we confirmed that the proposed system could accomplish a moving task on any unknown terrain in a shorter time than the conventional system by the task classification and reusing control laws.

The goal-conditioned reinforcement learning approach is useful when an agent wants to adaptively achieve arbitrary goals in various environments. Furthermore, it is desirable to be able to learn the behavior of an agent from observations from on-board sensors such as RGB cameras. Since it is difficult for agents to learn directly from images, previous research has shown that the learning process can be accelerated by learning latent representations using VAEs and performing goal-conditioned reinforcement learning in the latent space. However, latent representations learned using VAEs with image reconstruction loss contain information irrelevant to the task to be accomplished. In goal-conditioned reinforcement learning, the reward function is defined as the distance to the goal state, so latent representations that capture the distance between states are appropriate. In this study, we propose a method for maintaining the similarity between states in the latent space using contrastive learning and performing goal-conditioned reinforcement learning in the latent space given image observations. We compared the proposed method with that using latent representations obtained from VAEs, and show that our method outperformed the other.

We propose a reinforcement learning platform to learn to perform various tasks with a robotic hand to acquire a synergy. The model of deep reinforcement learning is trained to grasp an object with a multi-fingered hand. The synergy space is calculated by principal component analysis of hand postures when the task is successfully executed. The reward system is designed to minimize the distance of orthogonal projection between the posture and the synergy space, and the synergy space is acquired simultaneously with reinforcement learning.

In this paper, we propose a policy search reinforcement learning method with a non-parametric policy model and self-triggered control. We formulate a self-triggered policy search that employs a control policy and an execution length policy to reduce the number of action decisions in a trial. Our method employs sparse Gaussian process as a policy model with a self-triggered control framework, and its update law for maximizing return is derived based on variational Bayesian learning. We conducted simulations for a reaching task in a two-dimensional environment and confirmed the effectiveness of our proposed method.

In this paper, we propose a curiosity algorithm for sample efficient data collection in robotic task learning. Reinforcement learning (RL) is one of the most commonly used methods for robotic task learning. However, when the RL agent explores the environment randomly, it results in low exploration efficiency, especially in learning robotic object manipulation tasks with high-dimensional and continuous state and action space. To improve the sample efficiency, it is necessary to urge the robot to pursue unseen states where exploration is lacking. By making the robot curious, the robot would become interested in the unknown region and explore that region. In order to make the robot curious, we utilize the free energy principle, which has been attracting attention in the field of neuroscience. We propose a method that combines curiosity based on the free energy principle with Soft Actor-Critic, a state-of-the-art reinforcement learning method, and verify the effectiveness of the proposed method in simulation and real-world experiments of robotic object manipulation task learning.

We explore a control method of a robot hand where the CAVS mechanism is introduced on the fingers to change the contact friction. A vision-based tactile FingerVision is introduced under CAVS, with which the hand can sense slippage and orientation of grasped objects. We use this robot hand in controlling the orientation of a grasped object by slipping it on purpose with the gravity. This paper reports the preliminary results of comparison of control with and without CAVS.