It is important to measure shape of objects in a transparent vessel for 3D digital preservation of underwater creatures. We propose a measurement method using a bi-path stereoscopic image of the objects acquired with a monocular camera via rectangular vessel planes. The method estimates the posture of rectangular planes based on bundle adjustment considering light refraction, and measures the 3D shape of underwater objects using ray tracing. Experimental results demonstrated the effectiveness of the proposed method.
Critical Dimension Scanning Electron Microscope (CD-SEM) is widely used as a measurement tool of semiconductor patterns. It is necessary to set an image processing range for the measurement and to register the setting with a recipe every evaluation point. With increase of the number of the measurement points by the pattern miniaturization, burdens of the recipe creation increased. We propose a technique to generate the measurement recipe automatically based on design data of the pattern layout. The proposed method extracts all measureable pattern candidates by evaluating the positional relationship between each segment pair of design data. Next, the most suitable pattern for the measurement is decided relatively from the candidates. The proposed method was verified with 662 evaluation points and correct ratio of recipe generation and measurement using CD-SEM was 100% (662=662) compared with a manual operation. The recipe generation time for all evaluation points by the proposed method was 8 minutes. A large time shortening can be realized whereas the processing time of the manual operation was 3.5 hours.
We propose GIFTS (Goods Image Feature for Tree Search) local image feature for large scale object recognition system. GIFTS is a sort of keypoint feature and its feature vector consists of intensity delta of selected 128 pixel pairs around the keypoint. By generating KD-Tree from GIFTS feature vectors of training images and using the KD-Tree for searching nearest neighbor feature vectors of query image, order of log N query time for the specific object recognition is attained. We apply the proposed method to the query of book covers among 100 thousand training images of books, and performed over 99% recognition accuracy in the query time within one second.
Advanced driving assistance systems (ADAS) for various driving scenarios are developed. To develop and to evaluate the ADAS, a large quantity and high quality database of actual driving data is needed. In this paper, we propose a driving trajectory estimation method based on a 3D point cloud. The registration methods of the point cloud (e.g. ICP) has advantages in the accuracy of the estimation of the local (short range) motion. However, since the registration methods estimate the relative motions of frame by frame, estimation errors are accumulated and it is difficult to apply the method for long distance analysis. To solve this problem, a global trajectory correction method based on GNSS information. The evaluations of the proposed system are performed by using the actual driving data in an urban road.
This paper describes a novel rough terrain traversability analysis and behavior generation method for mobile robot navigation. We focused on the scenario of mobile robot operation in complex environment such as disaster sites. The proposed method enables mobile robots to analyze terrain traversability of surrounding area and select safe course direction. According to the selected course direction, appropriate control input is calculated and mobile robot moves following the calculated control input. Terrain traversability analysis and course direction judgment are realized based on feature value calculation of terrain, fuzzy inference and vector field histogram. Mobile robot's behavior generation is realized based on control method which takes advantage of reference posture tracking. The experimental results show that the proposed method can navigate mobile robot successfully to the target location in complex environment including terrain area.
In order to operate a camera-based patient monitoring system that detects when a patient gets up from or leaves the bed at hospitals and care facilities, we developed a technology to recognize the position of a bed at a high speed and with high accuracy. The conventional technology is not well-suited for use in a hospital setting because processing requires an excessive amount of time, the camera needs to be set at a particular position, and there must not be any objects on or around the bed. In this paper, we propose a method that distinguishes between a bed and other objects by hierarchically deciding if each line segment in the image comprises the outline of a bed or not. For each level in the hierarchy, we filtered the candidates passed to the next level based on shape features and other shape-independent features. By using the proposed method, we were able to accurately recognize a bed area at a high speed and with high accuracy even when the camera was placed at various positions and other objects were present on or around the bed.
This paper makes a proposal regarding a new construction of automatic visual inspection system for web-shape products. The inspection process is divided into “detection” and “investigation” stages aim to imitate a retina function of human vision. LCI-method, developed by author as a new defect inspection method, is introduced to the detection process. LCI simulates “Peripheral Visual Inspection”, recently have been quite effective for various production fields. Then 3D measurement of defect shape by using Quasi-phase-shift method is adapted for the investigation process. This separated inspection system makes it possible to improve the sensitivity of defect detection and reduce the number of imaging equipment.
In this paper, we will propose an automatic inspection system for fasteners using 3D measurement. 80,000 fasteners are mounted on an airplane. Inspection for fasteners is currently operated by manual labor and takes about 4,000 hours for main wings of an airplane (in the case of Boring787). In the proposed system, fasteners on main wings are automatically scanned by a robot arm mounted a 3D measurement device, and inspected their depth and angles. We developed an algorithm to inspect the depth and angles of fasteners by using an image and a 3D shape. In the experimentation, the proposed system could inspect fasteners within 5.6 [sec]. The error range of measured depth is 8.47 [μm] and the error range of angles is 0.054 [deg].
We propose a method of ego-motion estimation for a self-driving car on which we installed cameras with non-overlapping views. By finding corresponding points between the multi-camera images, we aim to enhance the accuracy of the ego-motion estimation. However since the viewing directions are very different from one camera to the other, a conventional algorithm such as SURF cannot detect a sufficient number of correspondences. Additionally in the case where cameras have low frame rate and the vehicle has high speed, the scene changes might be too big to find correspondences between the same camera images. We propose a novel matching algorithm by warping feature patches detected in different cameras based on urban 3D structure. We assume that detected features exist on the surface of buildings or roads and the patch around the feature is planar. Based on this assumption, we can warp the patches so that the feature descriptors are similar for the corresponding feature points. We apply Bundle Adjustment to the found correspondences to optimize the odometry. The result shows higher estimation accuracy when compared to other matching methods.
In this paper, a method of a novel combination photometric stereo which can estimate surface normals precisely even for images including shadows and specular reflection is proposed. Assuming that the number of input images for photometric stereo is more than three, the proposed method can exclude pixels affected by shadows and specular reflection by analyzing distributions of albedos and normal vectors computed from nC3 combinations for n input images. In these distributions, the proposed method define a novel value “compactness”. The compactness indicates the degree of concentration of albedos and surface normals, which should be the same values if all pixel intensities of input images perfectly obey Lambertian model without any error. Finally pixels which are included in neither shadows nor specular reflection are chosen by voting using the compactness. The proposed method is experimentally verified that it can provide accurate surface normals in the presence of shadows and specular reflection and it is superior to with better accuracy than previous works. Moreover a small device have been developed which supplies eight images varying in light positions and can be attached to smartphones. A possibility of practical use of the proposed method with the device is also verified.
Human tracking in surveillance camera has been challenging task in the field of computer vision. Tracking objects have large variations such as pose, body shape, clothes and so on. Especially in parts-based methods, postural change is big problem since appearnce of human changes drastically. We deal with this problem to use statistical shape model for tracking and detection. It represents the variations of postural change and body shape with low dimensions. Our trakcing result includes more detailed the position and shape of body parts. So we recognize rough pose and body direction to analyze it. These data is useful for seculity system or marketing decision in surveillance.
Machine vision systems on vehicle act as key technology onto Advanced Driving Assistance System (ADAS). These vision systems at visible light wave length detect lane markers, vehicles, pedestrians and other objects around vehicles. In the case of less visible weather-condition, these image output do not have clear contrast due to wave length for machine vision use. In this paper we propose a road recognition technology at Terahertz (THz) wave band by detecting two types of polarizations and radiation power from road surface and obstacles. In order to detect road surface and obstacles, we apply to compare horizontal and vertical polarized detecting power. A derivation and fabrication are represented in this paper. Our proposal system is addressed to improve the road boundary identification even if less visibility condition by detecting two polarizations at THz-wave band through our preliminary experimental results.
In this paper, we propose 3D reconstruction by structure from motion (SfM) using spherical panoramic imagery and its application. The first part starts from describing a brief overview of the proposed SfM and discussing advantages and disadvantages of using panoramic images for SfM. Since the major problem is induced by panoramic image distortion, the proposed SfM overcomes it by feature detection and description with the panoramic image rectification and by evaluating the angular errors in the geometric estimation. In the second part, as an application of 3D reconstruction by the proposed SfM, we propose a system for estimating positions and directions of input query images. To achieve this, we build the geo-located image database by using the camera positions and directions estimated by the proposed SfM. Using this image database, we perform fast query image localization based on tiled BoVW matching. We evaluate the proposed SfM on real image sequences and also evaluate the system of query position and direction estimation.
An inner wall surface inspection method for metal cylindrical parts of automobiles, such as engine cylinder bores, has been developed. This method is realized by combining a specialized optical system and image processing. The specialized optical system named “Endoscopic Camera” images the texture of the cylindrical inner wall surface. The specialized image processing named “KIZUKI” algorithm detects defects such as scratches from the successively captured images. In the optical system, a lens with a concave dome-shaped reflective surface was developed. This lens will allow not only obtaining an image of the entire circumference of the cylindrical inner surface with a single shot, but also coaxial incident illumination which highlights the scratches. “KIZUKI” algorithm is a technique inspired by the peripheral vision and the involuntary eye micro-vibration of the human vision architectures. It consists of three steps : first step for forming low resolution images by a variable re-sampling, and the second step for detecting relatively salient regions at each low resolution image, and the last step for integrating these regions. To evaluate the effectiveness of the proposed method, a cylinder sample with a known flaw size was measured. As a result, a scratch small as 0.16mm × 2.0mm in size was detected, thus the validity of the proposed method was confirmed.
This paper presents a methodology to compose half-diminished reality images for operating remote control robots. At sites for disaster response, robots are desired to achieve various tasks. However, operators have problems concerned with camera images shown to them for controlling robots. For example, operators have to understand the environment by comparing many display images from multiple cameras, because the robot arm itself occludes target work objects in main camera image. Half-Diminished Reality technique is used for seeing through foreground objects and viewing occluded backgrounds. We have applied the technique to remote operating. In this paper, a fast algorithm to estimate the background image is proposed. Furthermore, an online half-diminished viewing system for remote control is constructed. In the experiment, we confirmed the validity of proposed method with three RGB-D sensors and a robot arm : The proposed method could display online half-diminished reality images to the operator to see through the target work objects occluded by the robot arm.
In this paper, we discuss a method for automatic programming of inspection image processing. In the industrial field, an automatic program generator or expert system are expected to shorten the time form order ing of individually designed and manufactured appearance inspection systems to delivery completion. So-called “image processing expert system” have been studied for over the nearly 30 years. We are convinced of the need to adopt a new idea. Recently, a novel type of evolutionary algorithms, called genetic network programming (GNP), has been proposed. Consequently, we use GNP as a method to create an inspection image processing logic. GNP develops a distinguished directed graph structure for its individual representations, consequently showing an excellent expressive ability for modeling a range of complex problems. We have converted this network program model to Image Processing Network Programming (IPNP). IPNP selects an appropriate image processing command based on some characteristics of input image data and history process information. It is verified from experiments that the proposed method is able to create some inspection image processing programs. In a basic experiment with 200 test images, the success rate of detection of target region was 93.5%.
Recently, demand has grown for defect detection processes utilizing machine vision applications. This is especially needful for the abovementioned IC lead frames used in semiconductor manufacture, which require both high quality and miniaturization. In previous work, we proposed a detection method that assumes the variance in the intensity of oriented gradients in images that include defective areas will be larger than that found in acceptable areas. Therefore, there is a tendency to detect defects that has large variance in the local image. However, when conventional image processing methods are used for verifying a defect of deformation in flat parts, it was confirmed that the detection was difficult. Image processing methods using the surface normal direction is proposed in order to detect defect of deformation in flat parts. Since most of these methods use a fixed parameter, when these methods detect for various defects in industrial parts, there is a risk of missing a defect. In this paper, another defect detection method is proposed for detecting various defect sizes and defect types. This method determines appropriate block size based on median value of luminance dispersions calculated on several block sizes. In our experimental results, our proposal method can achieve good performance to detect defects in several sizes.
This paper describes a new method for improving fault tolerance of bird's-eye view systems that are used to teleoperate robots. The bird's-eye view image helps an operator understand the surroundings easily and teleoperate the robot more accurately. The image is created by synthesizing multiple images captured by the cameras that are attached to the teleoperated robot. With a conventional method, if a camera does not work well, blind spots occur in the bird's-eye view image, and it causes difficulties in the teleoperation of the robot. This paper presents a new camera arrangement method so that camera can capture the area required for the bird's-eye view generation in case of camera troubles, and the bird's-eye view image is created by using images captured by only cameras that are not failed. Experimental results show the effectiveness of the proposed method.