The ability of robots to listen to several things at once with their own “ears”, i.e., robot audition, is critical in improving the performance of search and rescue activities under severe conditions. This paper introduces “HARK” robot audition open-source software and its capabilities of suppressing ego-noise that is caused by robot’s own movements such as motor, propeller and/or flying noise. Then it describes three main applications of robot audition: 1) Unmanned Aerial Vehicle (UAV) with a microphone array to capture sounds can localize a sound source by suppressing ego-noise with either hovering, slow gliding or fast gliding. It can also recognize a sound source by CNN. 2) A serpentine robot with a microphone array can estimate its posture by sound. It can also enhance a voice by Online Robust PCA. 3) A robot with a LiDAR and 32-channel microphone can visualize a sound map by superimposing sound source directions on point clouds.

This paper shows a method to estimate external load of a multiple DOF hydraulically-driven robot from oil pressure of each hydraulic cylinder. Usually external load can be measured by a force sensor installed at the end effector of the robot. However, hydraulically-driven robots, which can be used in a disaster area, may damage the force sensor due to the unexpected impact. In this paper, it is shown that the estimation accuracy has been improved by considering the friction forces at the cylinder. The estimated force will be used for force control or force feedback to the operator of tele-operated rescue robots.

This paper presents an online method for localizing the positions of multiple sound sources and stationary robots and synchronizing microphone arrays attached to those robots. Since each robot can estimate only the directions of sound sources, the two-dimensional source positions can be estimated from the source direction estimated by each robot using a triangulation. In addition, mixture signals can be separated accurately by regarding multiple microphone arrays as one big array. To perform these tasks, some methods have been proposed for localizing and synchronizing microphone arrays. These methods, however, assume only a single sound source exists. To overcome this limitation, we estimate the directions of arrival (DOAs) and separate observed signals to estimate the time differences of arrival (TDOAs) by using microphone array techniques, and integrate the DOAs and TDOAs by using a state-space model. The latent variables are estimated in an online manner with a FastSLAM2.0 algorithm.

In recent years, the demand of robots that can work in dangerous environments or remote places is increasing. In such environments, teleoperated robots that can work instead of human are desired. However, it is not easy for them to achieve tasks smoothly and quickly because of the kinematic and kinetic difference or the time delay of communication. To solve the problem, operation assist control is introduced to a teleoperated robot, and reaching motion is in particular focused on in this paper. Operator's reaching motion is always predicted and when it is detected, slave's robot hand is guided to the targeted object with assist control. First, a predictor to detect reaching motion from any distance is proposed. Next, a predictor of the end time of an operator's reaching motion and a method to modify the reaching trajectory are proposed.

This paper proposes a novel locomotion style to move on rubble with the possibility of collapse for a life-sized four-limbed robot. A locomotion style which can increase normalized energy (NE) stability margin seems to be useful to move in extreme environments. Therefore, we propose crawling motion, which makes robot’s body contact on the ground. The robot moves four end-effectors forward simultaneously, lifts the body and moves forward. Verification of the proposed crawling motion is conducted through simulations with a dynamics simulator on rubble.

This paper describes ROS integration of the disaster response robot simulator“Choreonoid”. Choreonoid is an open-source robot simulator and ROS is a widely used middleware for robot software development. Integrating Choreonoid and ROS enables to develop disaster response robots quickly using rich software asset of ROS and without spending a large amount of time and money to build and repair robot hardware. The integration is done by adding a plugin to Choreonoid. The plugin publishes sensor information and other information such as absolute position/orientation of links as ROS topics, subscribe ROS topics to control joints and provides services such as pausing and unpausing simulations.

In this paper, we propose a region of interest based topological structure extraction method for a 3D point cloud. Our method is based on Growing Neural Gas with Utility (GNG-U) for extracting the topological structure. GNG-U can apply to the non-stationary data distribution by using a local measurement value. However, GNG-U cannot control the number of nodes because GNG-U cannot reduce the number of nodes basically. Therefore, we propose the improved GNG-U for controlling the number of nodes. Next, we explain a feature extraction method using the topological structure. Furthermore, we propose the environmental recognition method based on central and peripheral vision. Finally, we show an experimental result, and discuss the effectiveness of the proposed method.

A hose-shaped rescue robot is one of the robots that are developed for disaster response in case of a large-scale disasters such as a great earthquake. The robot is suitable for entering narrow and dark places covered with rubble in the disaster site, and for finding inside it. This robot can transmit the ambient sound to its operator by using the built-in microphones. However, there is a serious problem that the inherent noise of this robot, such as the vibration sound or the fricative sound, is mixed into the transmitting voice, therefore disturbing the operator's hearing for a call of help from the victim of the disaster. In this paper, we apply the multichannel NMF (nonnegative matrix factorization) with the rank-1 spatial constraint (Rank-1 MNMF), which was proposed by Kitamura et al., to the reduction of the inherent noise.

Recently, infrastructure inspection is one of important topic to avoid critical accidents. Many researchers have tried to realize automated inspection robots. A wall climbing ability is one of the important ability to realize infrastructure maintenance and inspection. However, in the previous research, there are no tough suction components for long-term inspection. To solve this problem, our group have developed the new tough air suction component for wall climbing robots. In this paper, we will introduce the robot system to evaluate the effectiveness of developed air suction components. through the experiments, we confirmed effectiveness of developed robot system.

A multi-rotor unmanned aerial vehicle, Mini-Surveyor, developed by Autonomous control systems laboratory Ltd. has a high flight performance because of its advanced flight controller. However, the flight envelope especially the limitation of wind speed is still narrow compared with conventional aircraft. It is intended to improve the agility of Mini-Surveyor through the following two technologies: (1) In the original Mini-Surveyor, the flight is controlled by varying the rotor speed. In the new multi-copter, it will be controlled by varying the pitch angle. (2) In the former one, six rotors are driven by the six motors. In the latter, they will be driven by just two motors. Flight test will be carried out to verify these improvements.

This paper presents an online method that estimates a 3D posture of a hose-shaped rescue robot using a microphone and accelerometer array. Posture (shape) estimation of a self-driving hose-shaped rescue robot is crucial for handling the robot body because the unseen robot posture deforms in narrow spaces under collapsed buildings. Conventional sound-based method that uses time-differences of arrivals (TDOAs) works only on a two-dimensional surface and is often hampered by the rubble around the robot. Our method eliminates the outliers of sound-based TDOA measurements, and compensates the lack of the posture information with the tilt information measured by accelerometers. Experimental results using a 3-m hose-shaped robot that was deployed in a simple 3D structure demonstrate that our method reduces the errors of initial states to about 20cm in the 3D space.

This paper analyzes a human tactile sensation based on the physiogical results. Then, the tactile sensor was designed by using the obtained results. Proposed sensor mechanism consists of the linkage, the wire, and the nonlinear spring. The mechanism has nonlinear properties and it can realize both the high sensitivity in the low lord region and wide sensing range. The effects of the dermal-epidermal structure were explained by using two different depth of the structure.

This paper analyzed the effects of the model predictive controller for the iterative learning control tasks of the human motion. In our assumptions, the predictive controller most affects the beginning of the iterative learning trials. Some experimental results indicate an example of this case, and they show a possibility of the effects of our assumption.

Head tilting motion is one of the available body parts to operate the assistive products such as auto wheel chair due to the fact that the head motion is unburdened and non-contact operation can be done. Here, we measured the head tilting motions and investigated its characteristics to construct a head tilting interface. In the experiment, the tilting angle of the head and the back of the neck during conversation, blowing, and daily motions were measured by the three dimensional accelerometers and the time series of the data was investigated by the FFT analysis. The experimental results said that the neck and head interlocked with each other and the bowing motion, which is conscious movement, was 0.3[Hz]. During conversation, the nodding motion, which is unconscious movement, was less than 3[Hz]. When using smartphones, watching videos, and writing something, the movement was less than 1[Hz]. We could find the differences between the conscious and unconscious motions.

Man-machine interface based on the biological signals from head is essential to the development of life support and personal assistance tools for handicapped people. We believe that user-adaptive recognition techniques are necessary for the spread of such life support tools and that the generalization ability of the recognition techniques is indispensable for achieving user adaptation. In this work, we propose combination of a parametric t-SNE and a 3-layer neural network for enhancing the generalization ability of our expression recognition system based on biological signals from head. Through experiment in real environment, we confirm the validity of our method.

In this paper, we propose a method to recognize the type and load of a measured human motion motion. The operation type can be classified using K-means and the Dynamic Time Warping method. For each type of motion, the apparent load of the motion can also be recognized by calculating the difference of the absolute sum of the angular acceleration vectors. In an experiment, we measured several types of motion to manipulate a box with three different weights. The motion type and the load can be recognized correctly with recognition rate of 83.3% and 89.6%.

In mold manufacturing, hand-polishing with grindstones is finally conducted to keep the mold surface uniform and smooth. After hand-polishing, it is necessary and important to evaluate if the surface is polished enough. This evaluation has been usually dependent on the experience and intuition of skilled workers. This study has developed the automatic evaluation of the polished surfaces using image processing. First, we investigated the features of the polished surfaces, based on the interviews to skilled workers, and attempted to extract these feature quantities by image processing. The surface evaluation by image processing had good agreements with that by a skilled worker.

This paper describes an intuitive interface to operate the computer based on upper limb motions. The skeleton information in humans of the Kinect sensor was used for detection of joint positions of upper half of body. From the obtained information, the motion of the operator was recognized to reflect a mouse operation. Here, we constructed the system which could choose two displays, scroll and move the position of the cursor by changing the functions according to the length of the extended right hand, and click with the left hand. In addition, we made the wearable device on the wrist for sensory feedback to user. The operability of the developed system was examined by the trial experiments.

This paper proposes a pedestrian self-localization system using propagated vibrations on lower limbs. The goal is to establish a navigation system with high accuracy that is convenient to use wherever we are. The system utilizes a conventional inertial navigation system (INS). In order to cancel the integration errors from inertial measurement unit (IMU), we focus on the human gait cycle. Based on the tendency that the ankle’s velocity is close to zero in the mid stance, we use the zero velocity update (ZUPT). To estimate the mid stance times, we construct a new algorithm that combines the piezo sensor and the IMU. As a result of the self-localization in a short path indoor, there is an average position error of 0.88 m in the x-y plane and 1.26 m in the z direction. These errors can be reduced by constructing more accurate correction models.

In this paper, we proposed a new interface by using lower limbs motions. Here, we utilised the center of gravity on the sole of the foot to reflect the operation of the equipment by using the slippers, whose sole shape was curved just like a cradle. The operation was composed of four modes and the LED on the slipper indicated the operation mode. Four modes could be changed by the shift in center of gravity movement to the heel by deep pressing motion. In each mode, the operation was carried out by the shift in center of gravity movement to the heel and the toe.

Human can perform various tasks by their two hands. If we can use additional hands, breadth of the tasks that we can perform will be larger. Currently, artificial limbs technology for people with disabilities has been developed in a practical level. However, development of additional robotic body is not progressing. In this paper, we propose to use the surplus muscle strength as an operating method of an additional robotic body. We expect that the operating method can get many input channels by using muscles of whole body , and does not depend on the motion of the body. In addition, this paper describes the disk rotation task using the third arm robot, which is operated with the surplus muscle strength.

I investigated the relationship between the layout of objects placed on a table and perception of order and disorder. In developing a system for tidying up tables through the cooperation between a robot and a human, it was necessary to identify table-tops that people consider to be untidy. Therefore, experiments were conducted in which participants were asked to evaluate the degree of order seen in images of tables that were presented to them. In this study, I used patterns consisting of several images of single stationery placed on a table. Participants were also asked to indicate the number of groups into which the stationery on the table could be separated. The results indicated that objects organized into more groups were rated as untidier than objects organized into a fewer groups.

Recently, there is an increasing demand for the method of 3D rendering which achieves both reducing the processing load and keeping the quality of generated image. One of the solutions to this trade-off is ”foveated rendering”, in which rendering is simplified as far as human cannot perceive. In the most of the previous researches, they adopted foveated rendering using a heuristic and discrete evaluation which causes generated image to be unnatural for the human perception. Then we suggest the method of foveated rendering which continuously simplifies rendering based on spatial frequency response of human vision system. We experimentally show that proposed method enables to render images approximately three times as fast as maximum-resolution rendering.

In this research, we propose a method for estimating the grasped object by inputting vibration actively. A vibration actuator and a contact microphone are mounted on thumb and index finger respectively, and the grasped object is estimated by obtaining the vibration through the object. A power spectrum of the vibration is calculated when a user grasps an object, and dimensional compression is performed for calculating the feature value. Support vector machine is employed for estimating the grasped object, and high recognition rates were confirmed through two evaluation experiments.

Human motion to return to upright position is measured and analyzed to reveal the mechanism to generate human motion trajectories for rising up from a bowing posture. The measured results suggest that the performance index to minimize the square sum of torque-changes may be used to form the motion trajectory. Then, a way to derive the optimal motion trajectory using the assumed performance index is described.

This paper aims to search how the cared person stands up. Recently, we are facing rapidly aging, and we need to use robot technology in care field. On developing care robot, we should understand the cared person movement. However, human movement is not revealed enough. If we construct human model, we can comprehend their movements. Then, we try to make simple human model. This time, we focus on standing-up from a chair that is one of the most basic movement. We use 3links-3segments model as a simple human model and develop the standing up robot. Dividing standing-up 5 phases and controling these phase properly, we investigate the similarity between human and robot.

It can be considered that mouth is available for the operation of the equipment due to the fact that its motion is very fast compared with other body parts. This paper proposed an assist system for bed user with omnidirectional wheelchair. The user's face was observed by the USB camera and the mouth open/close motions were applied for the operation of the equipment such as a reclining bed, omnidirectional auto-wheelchair, and so on. Here, we utilized the area of the user's mouth, which was chased in real time. We prepared three layers which had the select menu and it was selected by the mouth operation. The selected equipment could be controlled via a bluetooth.

In the field of cycling competition, the pedaling skill has been vaguely discussed for a long time. This paper aims to develop the skill training system for pedaling exercise, where the muscle activity pattern of a cyclist is visualized while pedaling exercise. The validation results of our proposed system will be discussed.

This paper proposes a simplified standing-function evaluation system based on virtual light touch contact using a Wii balance board and a wearable VLTC device. In this system, a virtual partition by VLTC is first created from measurements of the subject’s trunk and finger positions. Random on/off control of virtual forces from the partition enables evaluation of standing function based on the presence or absence of somatic sensory stimulation to the fingers. Evaluation and comparison experiments were conducted with 40 healthy male subjects. The results suggest that the proposed system allows to evaluate the standing function evaluation quantitavively.

In this paper, we propose an eye tracking method which can track pupil and iris simultaneously using near-infrared and color images captured by a 2CCD camera. Most of the conventional methods employ either the iris or pupil for estimating a gaze vector and therefore, both of the iris and pupil are tracked using 3D eye model. The method can compensate for weak points in the previous method. The high accuracy and low computational cost are achieved, and the performance of the proposed method was confirmed by the evaluation experiment.

In this study, we propose the air cell lumber support device with a hip auxiliary mechanism for the reduction of the driver fatigue increase during long time driving. We measured the driving fatigue by using the driving simulator without support system or with lumber support system or the proposed system. From the experimental results, the reduction effect of the proposed system is more than that of the lumber support device and that of no support system. But there is a subject that the reduction effect are similar between the lumber and the proposed system because he does not slide the hip position during driving. Therefore, the effectiveness of the proposed system could be verified for the hip sliding driver during driving.

This paper proposes a spatiotempolal analysis method of the tongue behavior in eating the nursing-care gel. First, we measure the tongue pressure in the process of eating the gel, by attaching the multipoint pressure sensor at the palate. Then, we introduce Lissajous figure drawn with two pressure waveforms measured at two different points. From the figure, we define three feature values: the pressure level, the pressure distribution angle, and the pressure phase difference. Finally, we confirm the correlations between the feature values and the values of sensory evaluation about the texture. The proposed method has a potential to be a useful tool for investigating the relationship between the tongue behavior and the texture of gel.

It can be considered that mouth open/close motion is very available because it is unburdened and non-contact operation can be done. In this paper, we proposed a new intuitive interface based on the mouth opening and closing motions. The user's face was observed by the USB camera, and it was taken to the computer to recognize the user's intention. Here, we utilized the area of the mouth and the aspect ratio between the height and width of the mouth. In the test trial, it was found that the mouth shape in the pronunciation of “A” and “I” could be easily recognized.

Relationship between the limitation of the fall avoidance motion and severity of the fall should be explained because the fall is one of the most severe risks in the industrial site. However, the study which recorded whole of the fall motion is rarely performed because of the difficulty of reproducing fall in the laboratory due to the safety concerns. In this study, the fall motion, which was caused artificially, was recorded as long as possible within the bounds of safety. Then, the successive motion until the ground contact was simulated. As a result, the position and velocity of the ground contact could be connected to the fall avoidance motion, which means the step length after tripping, and the motion during the fall, forward tilt of the body and vertical velocity of CoM.

This paper proposed an intuitive interface based on blinking input with non-contact state. Fast of all, user's eyes taken to the computer was detected by the Haar-like feature amount. The position of user's iris was estimated by varying the threshold in the binalization procedure. Here, we utilized the blink actions as an input interface and the open/close state were judged by the area of the iris. When the area was decreased, the system recognized that the user closed the eye. This constructed system was applied to the operation for the communication-aid.

This paper presents a driver turning operation model based on the results of the outer environment information selection. At first, we collected the driving profile of the turning operation by using a driving simulator at a city intersection and a visual point measuring device. Next, we identified the driver tuning operation model from the obtained information and conducted experiments to verify the validity of the identified model. As a result, we verified the effectiveness of the proposed method by comparing the driving behavior of the driver and the identified model.

As a motion classifier of the myoelectric prosthesis, neural network is usually used. In order to realize intuitive operation, improvement of identification rate of the motion classifier is necessary. In the field of the machine learning, deep neural network has attracted the attention, A study of the past of the machine learning , feature values which encourage the learning when learning or classifying was given by human. On the other hand in the deep neural network, it was extracted automatically. In this research, four layered neural network is applied as a motion classifier of human's forearm. Relationship between each layers of the four layered NN was analyzed visually.

This paper proposed a new intuitive interface based on lower limbs exercises. Here, we considered that the center of gravity on the sole of the foot was observed to reflect on the operation of the equipment by using the slippers, whose sole shape was curved just like a cradle. The developed foot interface was applied to the control of the auto wheelchair. The operation mode was composed of three states, forward and backward mode, diagonal mode, and turn mode, and these were changed by the shift in center of gravity movement to the heel. Each operation was carried out by the shift in center of gravity movement to the heel and the toe.

This paper demonstrates safeguarding for caring workers who operate bathing assist devices. Exact mechanical hazards of a bathing assist device are identified. Protective measures against the hazardous compress of a worker's upper extremity are proposed. Protective devices to perform the protective measures are developed and evaluated. An interlock structure and a sensitive pressure sensor are implemented into the assist device. Their performance and feasibility are discussed. A wireless proximity sensor based on human body communication technology is illustrated.

In this paper, impedance control methods based on pressure distribution have been proposed to improve the safety of Human Cooperative Robot. Considering that Human Cooperative Robots can hardly realize physical separation with cooperators or workplace, it is largely expected on how to avoid collision between Robots and Human and ensure safety. Therefore, disturbance pressure distribution can be obtained as forces created by collision on the basis of the pressure distribution sensors located on robots’ surface, so that impedance control methods are proposed to actively eliminate the disturbance pressure distribution. And the theoretical feasibility of the methods is finally ensured in simple ways.

This paper describes the base mechanism of a human-friendly robot. The base has four omnidirectional wheels, and is covered by an air cushion bag to absorb collision forces. The collision force suppression method of its base is also discussed. When a collision between an object and the base occurs, the collision contact force is suppressed by the air cushion bag and the base moves to the collision direction by the suppression method. Through collision experiments, the effectiveness of the base mechanism and the suppression method is confirmed.

In industry, establishing methods to estimate lumbar burden is required for the safety validation of assist devices aiming at reducing the risk of low back pain in manual lifting tasks. However, estimating lumbar burden is difficult even when without using the devices. This paper proposes a method to estimate lumbar moment, one of the components of the burden, with modeling lumbar spine as an elastic beam. The modeling enables to estimate the moment without using body parameters, e.g. center of mass in link segmented models, which can affect the accuracy. Measurement experiment was conducted for estimating the moment with the proposed method under the movement of subjects’ leaning forward, backward with and without handling loads. The results indicated that lumbar moments were able to be estimated with the proposed method in their maximum bending posture, though accuracy enhancement were needed as a part of future work.

In this paper, we proposed a stiffness changeable joint mechanism with a fail-safe function. The mechanism can change its stiffness to three states, elastic-solid-elastic, according to the safety and task situation. Each stiffness state can be switched by positioning a stiffness changing part. The design parameter was decided based on the kinematic and geometric condition of the mechanism and confirmed its validity through the simulation by using a dynamic simulator.

Artificial bones made from biomaterial simulated the physical properties of human tissue are developed by several suppliers. It is necessary to obtain the strength of artificial bone by a strength test in order to investigate the reproducibility and validity of the developed artificial bone. However, an evaluation test for strength of artificial bone has not been established. In this study, an availability of Finite Element Analysis (FEA) on evaluation test for strength of artificial bone is shown by comparing the strength of the artificial bone obtained by experimental. The analytical results agree well with the experimental results, because the difference between the FE result and the average of test results is within 6%. Thus, it is considered that the breaking load of a human ulna and an artificial ulna are able to estimate by using Finite Element Analysis.

Safety of human-robot collaborative operation is strongly demanded in the industrial field of manufacturing robots. In the study, we discuss the collaborative operation condition of a fixed type manipulation taking human characteristics of avoiding the end effector which is approaching his/her eye. Risk assessment shall be done in consideration of more close human-robot safety space management. In the study, more concretely we focus on the maximum speed of the manipulator and the initial distance between it and a human eyeball. Relationships between the maximum speed and separation distance by using data obtained from psychological experiments where 3 patterns are set as the initial distance. We further discussed the extension of the relationships to reach the maximum motion area of the robot where the human avoidance reaction time becomes infinity.

Recently care robots and care equipment with a charging battery are used in bath. Risks of electric shock while bathing by the charging battery are high, because water is conductive material. We considered these risks quantitatively with a simple electrical equivalent circuit method and a 3D electric simulation method. These consideration results will be verified to use risk assessment for an actual care robot or equipment used in bath.

The operation to urgently stop machines by humans is required to be “sure” and “quick or reliable.” To fulfill these two requirements are characters considered to be conflicting in terms of human operation, the emergency stop button should be suitably positioned. The basic data to examine the optimum balance between both characteristics was discussed.

It becomes increasingly essential to have unmanned robots and drones that can replace manned apparatus as to make works more efficient and safer when patrolling hazardous areas. This study introduces an enhanced display based on augmented-reality technology with markers. The display system is being developed for tele-operated robots to carry out missions inside buildings. The camera view from the camera mounted on robots is augmented by three-dimensional CAD data of buildings. This feature allows the operator to figure out paths that cameras cannot see, to compare them with CAD data, and to recognize unexpected obstacles and broken structure. The effectiveness of the developed display system was tested with a tele-operated RC car using three displays (AR video, camera video, and 2D map) in corridor type structure.

The industry's recent growing interest in virtual reality, augmented reality and smart wearable devices has created a new momentum for eye tracking. Eye movements in particular are viewed as a way to obtain natural user responses from wearable devices alongside gaze information used to analyze interests and behaviors. This paper extends our previous work by introducing a wearable eye-tracking device that enables the reconstruction of 3D eye models of each eye from two RGB cameras. The proposed device is built using high-resolution cameras and a 3D-printed frame attached to a pair of JINS MEME glasses. The 3D eye models reconstructed from the proposed device can be used with any model-based eye-tracking approach. The proposed device is also capable of extracting scene information from the cornea reflections captured by the cameras, detecting blinks from an electrooculography sensor as well as tracking head movements from an accelerometer combined with a gyroscope.

An IT enterprise is making robot industry develop in recent years. It's necessary that man coexists with a robot. There is an use of the robot variously, and there are also a shape and control method variously, but an actual control method is often unified. When a control method is unified, to make small movement coexist with big movement becomes difficult. Therefore, the control space that many control method is considered. The system to support control of various robots is suggested. Various control method is integrated by using virtual reality. Addition of a robot and control method becomes easy by using virtual reality.