In order for a robot to make a correct action decision, it is necessary to accurately provide environmental information to the robot. Therefore, this research proposes a method that can process environmental information more accurately and flexibly by modeling the environment over time based on object information from sensors. Unlike the conventional method of creating an environmental map, structuring is performed simultaneously with acquisition of environmental information. With this structuring, processing is flexibly performed even on an object that does not exist in the deep learning model. An evaluation experiment was performed in a laboratory to show the usefulness of the method.

This paper presents a misalignment recognition method using a Markov random field with fully connected latent variables for LiDAR-based localization. The major difficulty for the misalignment recognition is that considering entire relation of sensor measurement in localization is impossible because it must be assumed that the sensor measurement is independent to one another. The presented method enables to consider the entire relation via the fully connected latent variables. This paper also presents a calculation method of localization failure probability based on the misalignment recognition. Experiments show that the presented method can exactly detect misalignment even though partial sensor measurement overlaps with a map and can exactly recognize success and failure localization results.

SLAM-SEAN (Simultaneous Localization and Mapping-Simultaneous Exploration and Navigation) framework for a mobile robot to work autonomously in unknown environments is proposed. An implementation of the framework was completed by a combination of a local-minimum-free artificial potential method and a frontier-based approach suggested in the previous work where a particle filter SLAM is involved. The validity of the implementation was verified through simulations. Some other conditions to see applicability of the method to various environments were also discussed.

Many autonomous vehicles use 3D-LiDAR to perform localization. Localization accuracy of these autonomous vehicles is greatly affected by LiDAR performance, LiDAR mounting position, and surrounding environment. Because localization requires high reliability, we need to know the accuracy that autonomous vehicles can achieve in that location. Therefore, we must repeatedly perform driving tests and simulations using raw data in that environment. However, we need a lot of cost and effort to do these. So, I evaluate LiDAR performance using raw sensor data. In addition, I evaluate the sensor mounting position using sensor data generated in the virtual city area of the sensor simulator. Based on these evaluation results, I will verify the effect of LiDAR performance and LiDAR mounting position on Localization accuracy.

Recently, the use of autonomous mobile robots is remarkably increasing. Among them, mobile robots using omni-directional wheels are useful in narrow passageways such as factories and warehouses. In operation with narrow, moving accuracy of a robot is important factors. For this purpose, accurate self-position estimation is required but in the case of a robot that operates only by self-position estimation by the wheel, the self-position estimated value may deviate from the true position.

Therefore, in this paper, we aimed to improve moving accuracy of the robot by correcting the robot's angle error and the position error using a gyro-sensor and observer estimation.

To study methods of developing 3D printed robots, I converted metal frames of the robot arm to 3D printed frames, and compared them about joint stiffness and position repeatability.Technology of 3D printing is progressively developing, and above all 3D printer which can make fiber reinforced plastics(FRP) already have been developed.So application of it to robots is expected. However, making by 3D printer depends on individual know-how, and uniform methods of developing robots doesn’t exist. Therefore I converted metal frames of the robot arm to 3D printed frames, and verified problems. The joint stiffness was significantly reduced under a load in a specific direction. But the position repeatability don’t deteriorated.

In recent years, the development of Bernoulli gripper is advancing because it is used as suction cup that can adsorb in a non-contact way. Currently, we don ’t know the element of loss when the Bernoulli gripper is adsorbing an object. The energy analysis is important for the study of Bernoulli gripper since it is the base for the design and modeling of the Bernoulli gripper. Firstly, we assume some elements influencing the loss of energy. We studied the theoretical value of flow rate loss when the Bernoulli gripper is adsorbing an object and verified the model with experiment result.

Thin products such as silicon wafers that used for semiconductor parts in many electronic devices are fragile. The scratches and dirt will cause fatal defects for these products. For this reason, non-contact gripper that can transport thin products is desirable. This device grasps an object by air jet. However, conventional non-contact gripper is difficult to grasp an object without contact assistance when the air jet is uneven. In this research, we developed a new complete non-contact gripper by designing the passage of the air jet. The structure and the principle of grasping are introduced. The function of stable grasping was verified by the experiments using gripper prototype.

Three-dimensional (3D) measurement is the initial part to reconstruct as-built pipeline systems. However, many pipeline systems in factories are intricate and install in a high place near the roofs, and most commercial 3D measuring devices using laser technology are big, heavy and cannot reach such high altitudes especially passing through the narrow gaps between pipelines. Therefore, this study aims to develop a small and lightweight 3D pipeline measurement device which can work at high altitudes. In the paper, the design of the device including mechanical structure, electrical system and measurement techniques is introduced. The experimental result showed the validity of the developed measurement device.

In recent years, with the increase of infrastructures with magnetic surfaces such as bridges and plants due to urbanization, the necessity of maintenance requirement of them is increasing. And, in order to realize the unmanned operation of this work, magnetic surface traveling robots are being developed. In a conventional magnetic surface traveling robot, an extra actuator is required to change the adsorption direction when climbing steps. In this study, we proposed an adsorption direction changeable mechanism using the underactuated principle and developed a new magnetic surface traveling robot that can automatically climbs steps. We evaluated the step-climbing performance of this robot by prototype experiments.

This paper presents the way of giving the target angle to save energy consumption of snake robots and the influence of the position controller gain in the cost of transport (CoT). Conventional snake robots propel along the serpenoid curve reproducing a real animal snake locomotion. In this study, we propose efficient locomotion for snake robots by giving the target angles of square waves with a constant period. We call this binary control. Experiments verified that the proposed method also generates serpentine-like motion and save energy consumption.

In recent years, polymer materials as an elastic component of robots are receiving attention. Among the polymer elastic materials, it has been well known that polyurethane has superior mechanical properties. This is because the polyurethane rubber spring has a larger working range of the deformation and viscosity compared to conventional steel springs. This study evaluates the elastic and viscous properties of polyurethane rubber spring in the shear direction, which could be useful for robotic revolute joints.

These days, the demand for robotic inspection of aging pipelines is increasing. Among the existing studies on in-pipe robots, traveling through T-branches has been the hardest challenge. This is because it is necessary to install more actuators to select the pathway, that leads to a complicated structure and an increase in size. In this paper, we propose a way of adaptation to T-branches using only angle control and torque control of the single active joint of the multi-link articulated wheeled in-pipe robot based on the cosine curve trajectory. Our proposed robot is comprised of three joints, however, only the middle joint is active and the other two joints are passive. From the experimental results, although it was confirmed that the robot successfully passed through horizontal T-branches, it could not succeed in the vertical one.

Tactile presentation devices using mechanical stimuli require that the stimulus elements are properly contacted with the user’s skin. Many researchers have developed various kinds of the haptic display to realize the reproduction of skin sensation. Human body has complex shapes so that it is sometimes impossible to contact between stimulus elements and skin. In this study, we introduce the two-axis gimbal mechanism to haptic display. We referenced the ankle mechanism of walking robots for extreme terrain, which can handle complex shapes. In this paper, we conducted the experiments and evaluated the fitting mechanism with the acquired contact area and the pressure value. The results show that our proposed mechanism makes contact area increase and the pressure value in contactors uniform.

This research studies effects of a Virtual Coupling (VC) designed with fractional derivatives for virtual reality environment. In the previous researches, we proposed a design method of VC with various fractional derivatives instead of a virtual damper to express various hardness, concerning the passivity distribution in frequency ranges. In a characteristic passivity distribution, we have found to get a feel like stickiness or adhesive peeling. This study, therefore, focuses on and considers the relationship between stickiness and passivity. We also compare the passivity of VC making sticky feeling, and the experimental results in the previous research of peeling adhesive tape.

In this paper, we propose a non-contact method to display a cooling sensation using ultrasound-driven focal point cooled by mist vaporization. Our system uses ultrasound phased arrays that can create focused airborne ultrasound and contains mist generator. The focused ultrasound is spatiotemporally controllable in terms of its direction and position. In addition, it can cause acoustic streaming by sound pressure. Thus, the mist is traveled by air flow from acoustic streaming and converges at the focused ultrasound. In this way, when mist hits user’s skin surface, users can feel spatiotemporally pinpoint cooling sensation. The application of this research will be expected to improve the performance of a haptic display in the future.

In this study, we revealed the enhancing mechanism of the tactile sensation through a flexible coil spring. Flexible coil is a new device that amplifies tactile sensation on small undulating surface, and is a small structure of 1.2mm thickness and 2.6mm width. A two-dimensional kinetic model of the spiral coil was developped to analyze the behavior of the "pin-part" of the flexible coil when the coil is tracing undulation surfaces. The results of the simulation analysis indicated that the coil causes lateral strain on the skin surface of ball of a finger.

Remote Machine System is expected to compensate a shortage of workforce owing to emerging technologies of VR, AR, and 5G. There is still big difficulty to achieve remote machine manipulation with sense of oneness. One of the key technologies is transmission of haptics sense between a remote machine and an operator. Although, there are a lot of devices to transmit the haptics sense, they are not enough to get the feeling of “sense of oneness”. In this paper, visual haptics technology is proposed as a pseudo haptics to improve a remote manipulation task, and the efficiency is verified through the evaluation experiments by the subjects.

In this study, we compare the passivity of Virtual Coupling (VC) using fractional derivatives and the passivity obtained from real objects, and consider how it affects the tactile sensation. In the past, we have performed experiments to obtain associative objects from VCs using fractional differential terms, but have not compared them with real objects. Therefore, in this study, we calculate the parameters from the data obtained from the real object and the model including the fractional derivative using the shaker method, and examine the passivity of the real object. Then, we compare it with the passivity of VC using non-integer derivative, and consider how tactile sensation is given by frequency passivity.

A flat display with tactile feedback of arbitrary button click sensations, instead of physical buttons, will allow us to design operation panels more flexible and adjustable in accordance with functions. This study proposes a reproduction method of the click sensation of physical buttons in a flat vibrotactile display based on simultaneous measurements of force-stroke (FS) characteristics and vibrations propagated to the finger. We introduced a concept of FS stage and vibration templates to duplicate the sensations of four different types of buttons. Multi-dimensional scaling evaluations showed that duplicated sensations were similar to the real buttons.

In recent years, research has been conducted on robots of various shapes such as humanoid robots and quadruped robots according to tasks and environments. As the legs are oriented to move on steps and the wheels are oriented to move on flat ground, there is room for considering the optimal hardware configuration. However, in studies that do not assume the shape of the robot body, such as modular robots, it is difficult to achieve both hardware expandability and sensor configurations required for the robot to autonomously accomplish tasks. Therefore, in this research, we construct multiple sensor modules with built-in joints, which has a shape and communication system that can be used with existing small multi-type robot configuration kits. By changing the combination and assembling of existing parts, we aim to construct a platform that can verify various body shapes and sensor configurations.

We developed a data-driven method for modeling flexible objects that interpolates measured data and presents it as a target value for haptics presentation. Furthermore, in order to realize visual and haptics presentation, this method was combined with visual presentation by deformation simulation. As a result, we succeeded in reproducing the visio-haptic display of actual flexible objects.

In this paper, we introduce a novel wearable sensor that can measure contact force applied to fingertip. The sensor measures the amount of finger width deformation by attaching a tactile sensor to the fingernail side. That is, because the proposed sensor does not cover the palm side of fingertip, sensing can be performed without impairing the feel of the fingertip of the person wearing. Through several experiments, it was confirmed that the fingertip force could be obtained as relative values. Moreover, the applicability to motion classification was confirmed using the drawing problem as an example.

In general, a driving simulator requires a motion platform to represent the centrifugal force during curving by swaying and translation the whole body. This paper reports a new method of presenting centrifugal force by presenting virtual shear forces, which are induced illusionary by buttock skin stretch stimuli. This paper focuses on the stimulus differences on the left and right buttocks, considering the body movement caused by the centrifugal force. Psychophysical experiments showed that the skin stimuli in the asymmetric condition tend to deliver a stronger feeling than that in the symmetric condition.

Along with spread of low-cost HMD and high-quality VR experience, many proposals have been made to improve the VR experience by providing tactile information to the fingertips. However, attaching and detaching a device is difficult and the movement of the finger are limited. In order to solve these problems, we have proposed Haptopus that incorporates a suction tactile presentation mechanism in the HMD. Haptopus can transfer the fingertip tactile information to around the eyes without impairing the quality of the experience and improving the wearability. In this paper, we investigated that the hardness of VR object could be presented by the adjustment of dynamic air suction pressure in order to increase the types of haptic information.

A temperature-displacement sensing method whereby a negative temperature coefficient (NTC) thermistor can be applied to design a polymodal tactile device is proposed. A trial sensing device with an NTC thermistor to detect simultaneously temperature and displacement was investigated. The trial sensing device consists of an NTC thermistor that is embedded in a silicone tube filled with water. The capacitance of the thermistor depends on not only temperature but also deformation of surrounding water based on displacement of the silicone tube. We formulated the function of an inverse response function to identify two tactile parameters (temperature and displacement) using two output signals (capacitance and resistance of the thermistor). The experimental results show that it is possible to detect displacement and temperature by obtaining two kinds of signals that respond to both the displacement and temperature.

This paper describes the development of a surgical tool with a single strain area for measuring biaxial cutting forces. For measuring bending and compression forces in a single strain area, strain area needs to be sensitive to both bending and compression. In general, the bending strain is easier to measure compared with the compression strain. In this paper, an oval shape is designed in the strain area to obtain both forces. With the proposed shape, two axes forces could be measured. However, the residual strain in the direction of bending remained. Therefore, a spring is inserted in the tool to compensate for the residual strain. Consequently, the residual strain can be canceled out, and the surgical instrument with a single strain area for measuring biaxial cutting forces can be developed.

We are exploring new applications using haptic sharing. If we were able to capture and perceive the sensations and movements of others into ourselves, we could conduct cooperative works with others on the basis of shared haptic perception. We demonstrated human-robot cooperation by sending human haptic information during the task to the robot. Regarding human-human cooperation, humans need to perceive haptic information and identify the other’s state on the task. Therefore, in this paper, the identification of different tool operations by presenting vibrotactile stimulation that reproduces the skin vibration collected during each operation was tested. Performance of the identification was investigated on different positions to present vibrotactile stimulation and on the multi inputs of vibrotactile stimulation.

The HR-Recycler project performed in Europe targets the development of a hybrid human-robot recycling plant for electrical and electronic equipment operating in an indoor environment. For example, when a robot disassembles a TV back panel for recycling, the vision systems detect the screw positions automatically. However, sometimes there are several hidden fixed points that are not detected by the vision system. Humans have a sensation for finding the hidden screw positions by lifting or moving the back panel. Force reactions and deformation displacements are important information for this estimation. In this paper, an end effector of an arm robot was designed and evaluated for measuring a deflection angle required for estimating the Young's modulus. A 3-axis MEMS tactile sensor was employed as an end effector sensor and the deflection angle was measured by pushing up a cantilever made of an aluminum plate.

In this research, we employ spatially controlled thermal radiation to display a virtual shape. Using controlled-thermal radiation, our proposed system displays the virtual 3 dimensional area (e.g. a dangerous area). In this paper, we propose two tracking method of a user’s palm. Through an experiment we revealed that RGB image is better than depth image for the tracking.

We present a 3D hand pose estimation method works in a high frame rate with a high speed vision system and CNN. We employ a high-speed camera for a fast gray-scale image acquisition which enable us to recognize fast and small finger movements and a normal RGB camera for an accurate estimation. In the proposed method, the 3D hand pose is estimated by CNN with RGB images and the estimated pose is corrected at every frame using difference of pose estimated by CNN with gray-scale between adjacent frames. Firstly, we develop CNN models for RGB and gray-scale images, and the former was more accurate than the latter in a depth direction. Secondly, we verify our method through an experiment evaluating accuracy of differences of estimated poses between adjacent frames.

One of an important mission for patrolling robots is to find suspicious and lost objects in an environment. For this mission, we use a mobile robot equipped with an RGB camera. In order for the patrolling robot to find such objects, we propose a change detection method in 3D space based on RGB images. This method adopts a background subtraction technique. For background subtraction, an original image without objects is captured beforehand. After that, target images with objects are captured, and the differential images from the original are used for Sauvola’s binarization. For difference possibility areas in the two images, moreover, the similarity is calculated on the basis of ZNCC (Zero-mean Normalized Cross Correlation). Through the experiments, we show that it is possible to find small objects, such as ID cards and cell phones in an indoor environment regardless of the illumination.

Nowadays, a decreasing birthrate and an aging population have been progressing in Japan, and it caused a problem of a decrease in the labor force. Therefore, the needs of robots to replace the work that humans did far have been increasing. To replace the work , robots need to operate autonomously in constant changing environments. In order to solve this problem, a method called visual servoing was proposed. Visual servoing is a method of controlling the operation of the robot by visual information obtained from the visual sensor. But the conventional method requires prior knowledge of the target concerning the shape, size and color of the object. This impairs the flexibility of the system in constantly changing environment. So our research group proposed and has been studying Projection-based Matching Method that does not require prior knowledge. In our laboratory , we have been studying robots to explore the ocean. We would like to track target in the ocean. In this study, the accuracy of position of objects that are estimated in low illuminance environment by this method is verified.

To determine the positional relationship between a camera and a robot, calibration is required. Since it takes time, it needs a simple calibration. In this research, we propose a robot positioning method by reducing calibration time. First, multiple robot coordinates are measured from visual tracking, and calibration is performed using the robot coordinates. After that, a target coordinate is calculated using the camera parameters estimated by the calibration, then a robot moves to the target location. Because it is a simple calibration method, the positioning accuracy of the robot is low. Therefore, the camera parameters are modified online using the Levenberg-Marquardt method during the robot operation, thereby the robot positioning accuracy is improved.

Ota City in Gunma Prefecture is suffering from wild animal damage, so the city has installed an infrared camera with motion sensor as surveillance devices to investigate the ecology of wild animals. But a huge amounts of movies captured wild animals. There were some things that were not animals because the motion sensor also responded to the movement of plants and trees, and it was necessary to select only the video that showed the wild animals. It took a long time and was very burdensome. The system developed in this study aims to automate the task of selecting only the video that shows the wild animals. Here, we developed a system that automatically determines the presence or absence of wild animals in the recorded video．

3D measurement is required in various fields including an automatic driving, an infrastructure inspection, and a robot industry. The technique to be required in such fields has rapidity. Therefore, we suggest the stereo method using a high-speed vision. A high-speed vision can do high-speed operation including shooting and the image processing at 1000 fps. However, there often exists a problem that the occlusion occurs by changing environments. Therefore, suggested system performs 3D measurement by capturing a stereo pair from the high-speed multi-vision, after having removed the camera that occlusion occurred. The system is available for the 3D measurement of a moving object maintaining rapidity by the suggested algorithm. This paper refers to the algorithm and the result of the occlusion judgment experiment.

This paper presents the development of a right turn method at crossroads based on road area detection by using deep learning for automatic running wheelchair robot. Road images captured by a monocular camera and used as an input image. The road and non-road areas of the images are detected by using a deep learning. The wheelchair robot turns to the right at a crossroad by changing its control methods based on presences of non-road areas in front and on left. Also, the performance of this method is confirmed by constructing a wheelchair robot system to perform control applying this method.

Deep learning promotes the development of computer vision, but like R-CNN series models with accurate advantage and YOLO series models with speed advantage, which are heavily dependent on the powerful GPU. So it is difficult to deploy on mobile applications to achieve higher performance in mobile vision. In this study, we used the light weight MobileNet as the backbone to extract objects feature maps and referred to the training strategy of SSD and YOLO to achieve real-time multi-object detection. In addition, we carried out the multi-object tracking after detection automatically through processing some cached information like feature maps and motion position in a sequence of frames. Our proposed detection and tracking model achieved 80 categories detection at the speed of 12 FPS and up to 16 objects tracking simultaneously at the speed of over 30 FPS on iPhone 8.

In this research, we develop a near-future perception system named “Smart Previewed Reality” for humans and robots to coexist safely. This system consists of a smartphone and the management system of informationally structured environment “ROS-TMS”. Informationally structured environment is so-called a smart environment, in which various sensors are placed in the space where the robot works, and the information of humans and objects and current and planned robot motion are stored structurally in a database. Therefore, by showing planned robot motion in the database by a smartphone beforehand (Previewed Reality), the user can perceive the near-future robot motion and avoid dangerous situation such as collisions. We develop smartphone applications for Previewed Reality including visualization of robot motion using AR, UI indicators of robot position, and push notification of dangerous situation.

Mice are extensively employed in biomedical science and their responses to disease or therapy are frequently detected by measurement of their behavior patterns. Improving the accuracy of the detection, the automation for counting the mouse’s behavior is necessary and the evaluation must be quantitative. Conventional automatic detection systems could not detect some behaviors like a digging behavior which subject to materials in the cage. The detection of the mouse's digging behavior is based on the temporal differencing features. The detection method calculates the changes of the mouse’s neighboring region using frame subtraction method. Our proposed method could detect the digging behavior more than 80 % accuracy on par with human recognition rates.

In recent years, 3D printers have become a part of our lives. To date, various additive manufacturing methods have been developed for 3D printers. Among them, Fused Deposition Modeling is simpler in structure than other additive manufacturing method and can be applied in various situations for household use to business use. However, as compared with other additive manufacturing methods, the Fused Deposition Modeling has a long molding time and high speed additive manufacturing make low accuracy manufacturing. Therefore, high speed and high precision forming are desired. In this research, we propose a new molding method that measures the molding line using a high speed camera and performs high precision molding even at high speed molding based on the relational expression.

Japanese plantations occupy about 10 million hectares. Many trees have reached the cutting season. Focused on tree survey. Tree species classification is important for tree survey. This paper presents a method for classifying individual tree species on RGB images. CNN (Convolutional Neural Network) is a neural network model using convolution. CNN consists of convolutional layer and pooling layer. CNN can extract features in images．We used RGB image (cedar and cypress) taken in Japanese plantations. Also, We calculated the accuracy using 200 images (cedar and cypress)．The accuracy reached 80% overall. It is possible to classify trees in the image.

In this paper, for robot vision as home assistance, a pose estimation method of the generic objects is proposed. The proposed method approximates genetic objects as primitive shapes using a deep learning technique. Furthermore, the surfaces are extracted from normals and curvatures which are calculated by using depth data. Finally, the positions and postures are estimated using pose of surfaces.

Recently, drone technology has developed rapidly. Because, drone is expected to play an active part in various fields such as surveylance, life saving, monitoring. Especially, the necessity of automatic monitoring using drones in the ocean and the beach is high. This research proposes a new method that detects the location of ripcurrents and performs danger advisories, flow survey. First, UAV detects areas where rip currents are occurring. Next, Inspecting the rip current region by USV and conducting a flow survey. This paper describes a method for detecting areas of ripcurrents using temporal luminance changes on the sea surface. Experimental results at Ocean Wave Research Laboratory in Kagoshima University have shown the effective of the proposed method.

Recently, drone technology has developed rapidly, expected to play an active part in various fields. Focusing on USV, its usefulness is increasing mainly in the coastal area, such as feeding and red tide damage countermeasures. Survey on turbidity and drifting sand phenomenon is important. Used for water quality control and beach erosion detection. Traditional methods of turbidity and drifting sand phenomenon are expensive and time-consuming. Furthermore, the beach topography changes frequently due to typhoons, etc. It is necessary to improve the efficiency of wide-area surveys. This paper proposes new water turbidity and drifting sand estimation method of coastal area using USV. Estimate turbidity by comparing underwater images of a reference point and an observation point. In addition, by separating the movement of the wave and the movement of the density, the direction and speed are calculated, and the littoral-drift estimation is performed. Experimental results have shown the effective of the underwater images area division, and temporal change of brightness.

In this paper, the author aims to embed arbitrary information in arbitrary images and restore them using convolutional neural networks. To achieve this goal, we propose two sets of CNN models for embedding and restoring information in images.Thanks to the error correction nature of QR code, embedded information is expected to be restored without errors. However, in the previous model, the proposed method could only be applied to blurred images. This time, the model has been improved so that the QR code can be embedded in a high-resolution image and restored.

We developed a gesture classification algorithm that is installable to a small edge device. The algorithm is composed of neuro-inspired spatial and spatio-temporal feature extractors and the echo state network (ESN). The spatial feature extractor is based on the HMAX algorithm, which extracts scale-invariant features of line segments, and the spatio-temporal feature extractor was developed by combining the HMAX algorithm and the motion detection model of the visual cortex. The ESN, which can classify spatio-temporal data in spite of its simple three-layer architecture, was employed to avoid employing deep neural network, which is not suitable for small devices. We evaluated the proposed algorithm by classifying ten kinds of sign language. The results showed that the algorithm was successful in classifying the sign language.

In recent years, in object tracking of expecting the children’s monitoring and the automotive applications, modeling the motion of a tracked object is an important task for determining the performance of the tracking algorithm. If there is no prior knowledge about the motion of a target, it often tracks it with a particle filter under the assumption of “ smooth motion. ”Note however that, this doesn’t work well for sudden movement changes and unexpected movements, and fails to track them. Thus, a system has been already developed to model the motion of a tracked object by optical flow and track it with a particle filter. The Lucas and Kanade (LK) method is known as a typical algorithm that acquires the optical flow. In this method, the brightness is assumed to be preserved such that the brightness value in the same pixel doesn’t change between frames. However, the brightness preservation is not held in reality due to noises caused by the change of the photography environment and the change of amount of solar radiation. In this paper, an algorithm for estimating the optical flow, which is robust against luminance changes, is developed by calculating the amount of luminance changes with the difference average and by correcting the temporal luminance gradient. In addition, some experiments on object tracking are conducted using an optical flow–based particle filter that is robust to luminance changes, so that its effectiveness is verified through such experiments.

We developed a binocular stereo vision system inspired by disparity detection cells in the early visual cortex. This system consists of two image sensors, field-programmable gate arrays (FPGAs) and a personal computer (PC). We developed an algorithm that searches for corresponding points from the left and right visual inputs using spatial changes in spatial frequency in order to avoid false matching of visual features with spatial repetition. We also implemented a part of the algorithm in the FPGA in order to perform spatial processing efficiently. We evaluated the performance of the system by presenting visual stimuli consisting of sinusoidal gratings with different spatial frequencies for the background and the object. The system was successful in detecting depth from local changes in spatial frequency without causing false matching.

We developed an algorithm for estimating ego-motion of a drone from the visual information acquired by the sensor on a drone. The ego-motion is estimated by the echo state network (ESN) whose input is provided by low-computational-cost optical flow detectors. The ESN was employed because its three-layered simple structure is suitable for installing it into a system embedded in a drone whose size and computational resource are limited. We evaluated the algorithm by classifying actual 6 degree-of-freedom motion that were applied by motors. The results showed that the algorithm was successful in classifying the motion.