In this paper, we propose a new method for audio-visual event localization ¹⁾ to find the corresponding segment between audio and visual event. While previous methods use Long Short-Term Memory (LSTM) networks to extract temporal features, recurrent neural networks like LSTM are not able to precisely learn long-term features. Thus, we propose a Temporal Cross-Modal Attention (TCMA) module, which extract temporal features more precisely from the two modalities. Inspired by the success of attention modules in capturing long-term features, we introduce TCMA, which incorporates self-attention. Finally, we were able to localize audio-visual event precisely and achieved a higher accuracy than the previous works.

Manufacturers need to keep high efficiency and sufficient production capacity even in any emergency situations such as natural disasters or epidemic diseases as well as steady situations. We proposed a divisional and high-efficient line “DHEL” which is reconfigurable by modifying the production resources that include line workers and multipurpose equipment called robotic cell. It is difficult to determine an appropriate combination of the resources because individual time required for training the workers or adjusting the robotic cells and capabilities to perform assigned tasks should be considered in total. This study proposes a combination design method that can maximize the total production efficiency by considering a time-series behavior of capabilities of individual resources from the viewpoints of quality, delivery, and cost level.

In this paper, we propose a new human tracking method equipped with a function to return tracking a person. Under the stereo framework of fusing the processing results of two cameras, there is a three-dimensional tracking system for a specific person by a mobile robot using OpenPose that can acquire skeletal information using deep learning. We propose a new human tracking system equipped with the function to return tracking a person by template matching that limits the search range by epipolar constraint. The function is to return tracking a person who failed to be tracked by using the image information that was successfully tracked. With this function, even if one camera fails to track a person, the robot can continue to track the person. The effectiveness of the proposed human tracking system is verified by experiments.

The demand that a teleoperated robot performs tasks in hazardous area instead of humans has been grown recently. Conventional teleoperated robots are designed for operators to recognize the surrounding environment by presenting images from multiple cameras. However, a high level of skill is required to accurately grasp the situation by referring to multiple images simultaneously. To show understandable images to the operator, this paper proposes occulusion-free video super-imposing system using a high-speed blinking LED markers and multiple RGB-D sensors. For superimposition of multiple RGB-D data with a small misalignment even from high-speed teleoperated robots, the proposed video superimposing system updates the relative pose at high speed by using blinking LED markers. Through the experiment, the blinking LED markers obtained the position with an accuracy of about 1 cm and the attitude with an accuracy of about 1^◦ and the update latency of the pose from the LED markers was 2.06 ms. Moreover, we also superimposed images with an update frequency of 60 Hz and an accuracy of about 1 cm. The high-speed feedback from LED markers realizes video superimposition with a small misalignment against high-speed pose fluctuation between RGB-D sensors.

This paper discusses chatter vibration in turning of a low-rigidity workpiece with parallel leaf spring structure shanks. Because the chatter vibration is suppressed by control of the tool vibration direction, the parallel leaf spring structure shank that reduces the stiffness artificially in the tool height direction is studied to design the tool shank in the 1^st report. In this report, the vibration directions of the workpiece and the dynamic component in the cutting force were characterized in cutting with the chatter vibration. The compliances of the workpiece were measured to identify the principal direction of the vibration. Because the measured cutting force includes vibration, the steady (average) and the dynamic components are decomposed to specify their applied direction. In turning with the parallel leaf spring structure shank, a large difference in the direction of dynamic component is confirmed from that of the workpiece vibration. A model considering the direction of dynamic components is presented to verify the effect of the difference of the vibration directions on control of the chatter vibration.