Structure from Motion is a popular 3D measurement method. This technique uses images captured by a single camera with motion. Structure from Motion can calculate the 3D position of objects and the camera motion simultaneously. Because of its simplicity, Structure from Motion has been implemented in various ways. However, there is an essential problem that the scale of the objects cannot be calculated by Structure from Motion. In this paper, we propose a method which solves this problem using refraction. Refraction induced by introducing a different medium results in a change in the path of a light ray. This method is implemented using only a refractive plate and a single camera. The refractive plate is placed in front of the camera. We can obtain the refractive images using this system. The results of simulations show the effectiveness of the proposed method. It is also shown that the precision of reconstruction in the proposed method is improved when the thicker refractive plate is used.

View full abstract

We propose a fast and robust algorithm for rotation invariant template matching (RITM). Correlation-based template matching is one of the basic techniques used in computer vision. Among them, RITM, locates a known template in a query irrespective of the template's translation and orientation has been widely put to use in many industrial applications. A naive implementation of RITM requires intensive computation since one needs to correlate query with a set of rotated templates (Template matrix). Eigen template method utilizes the Eigen decomposition of Template matrix to approximate the matching score, thus speeded up the RITM by reducing the number of correlations. We utilize the circularity of inplane rotation and decomposed Template matrix using low-frequency part of the Fourier basis. This decomposition further speeds up the matching process by FFT. We also extract phase information of complex gradient images as a feature, which improve the robustness for illumination changes and background clutter. Experiments revealed that the matching processes becomes 15 times faster compared to the naive implementation of RITM while obtaining robustness for illumination and background fluctuations.

View full abstract

In the semiconductor manufacturing, a critical dimension scanning electron microscope (CD-SEM) is widely used as a process monitoring tool. As design rules become finer and denser in recent years, needs for controlling three-dimensional (3D) shape in addition to one- and two-dimensional size of the pattern, such as the CD value, have risen for the process monitoring. In this paper, we propose a novel approach for estimating 3D bottom footing shape of gate patterns from two SEM images, with 5 and 10 degree tilt. Bottom footing shape can provide valuable clues for inspecting and analyzing electrical properties of semiconductor devices. The proposed method realizes a highly precise estimation by combination of existing inverse stereo matching (ISM) method with a bottom footing index, which is the width of the range where the bottom footing exists. The measured shape of ISM method is represented with a cubic spline curve and is then corrected by transformation of the curve on the basis of the pre-learning relationship between bottom footing index and curve parameters. The proposed algorithm was verified with 12 samples of gate pattern. As the results of the evaluation, the average and deviation (three sigma) of the measurement error in the part of bottom footing were 1.3nm±1.0nm. The proposed method can effectively provide an accurate 3D shape and is expected to be applied to the process monitoring in actual production lines.

View full abstract

When detecting persons from time-series camera images, shielding by obstruction (occlusion) seriously decreases the detection accuracy. For example, a person in an agricultural field represents a semi-overlapped situation by the crop and an accident with farm machinery could occur. In this paper, we propose a person detection framework to prevent accidents in an agricultural field. We use multi-camera array to acquire 3D light field of scene, and refocusing process reduces the effects of occlusion. We also use deep learning with the features of convolutional neural networks (CNNs) and classification by a support vector machine (SVM). The experimental results using datasets of a real agricultural field show the effectiveness of our approach.

View full abstract

A skin color is one of the features that is often used to detect a face and a hand of human. A skin detection method based on skin color is often introduced in an early stage of detection process. The method can be realized by a relatively simple algorithm. In addition, it is a powerful method that gives a major impact on the detection accuracy. However, there is a problem that the method cannot adapt to change in a color of ambient light. In order to perform the method even if a color of ambient light is changed, we proposed an estimation method of spectral ratio of ambient light. In proposed method, first, we estimate a reflectivity of an object by using a reference light whose spectral ratio is known. Then, we estimate a spectral ratio of ambient light by using the estimated reflectance of the object.

View full abstract

In this paper, we will propose an evaluation method for reliability on phase-shifting methods. Environmental noises (e.g. vibration, air turbulence) affect quality of 3D measurement results in phase-shifting interferometry. Proposed method computes the score relating to quality of phase images. The correlation between the computed scores and ripple errors (surface roughness) was revealed by the experiments with dent sample made of silicon. As a result, the correlation with proposed method are higher than conventional ones. Hence, the qualities of measured phases can be ensured by the proposed method.

View full abstract

In recent years, demands for Advanced Driving Assistance Systems (ADAS) is increasing, and pedestrian detection has become one of the most important and popular technologies in this system. In the case of pedestrian detection using an in-vehicle camera, since the road environment varies widely according to difference in lightning, weather, etc., it is very difficult to handle them with a single classifier, and numerous false positives are detected. To overcome this problem, this paper proposes a novel pedestrian detection method by scene adaptation based on false positive mining. When we observe the appearance of false positives in in-vehicle camera images, those with similar features are found even in different road environments. The proposed method focuses on the appearance of the detected false positives, and considers it as a scene that the classifier is not good at. By analyzing such a false positive tendency in each scene, the proposed method associates the false positive tendency to each scene and then associates them to each training image. Then, classifiers are constructed so that they can cope with false positives observed in each scene. To evaluate the effectiveness of the proposed method, experiments were conducted on the Caltech Pedestrian Detection Benchmark datasets. Its results showed that the proposed method outperforms the method without adaptation.

View full abstract

We propose a method for distinguishing multiple objects. A general template matching algorithm essentially has an ability of classifying two-class patterns, one is a target pattern and the other is a non-target pattern. This basic ability is created from the fact that even one pixel can identifying patterns. By focusing this point, in this paper, we propose a new template matching method which has high ability of classification of multiple patterns simultaneously. Experimental results have shown the recognition rate of the proposed method was 97.0% and processing time of that was 7.1sec.

View full abstract

In this paper, we describe about classification of a defect image for automatic visual inspection. Conventionally, a machine learning approach has been effectively utilized as a method for determining the quality of a defect candidate image in recent years. However, because the logic constructed in the classifier is black box generally, it is difficult to perform the maintenance and operation by inspectors. Therefore, we propose a method of machine learning based on inspector's impression expressions about a defect image, such as shape, density, texture. The logic constructed by our method is not only easy to understand by inspectors, but also expected to abstract a tacit knowledge of inspection. Experimental results indicate that our approach is applicable to the defects provided from the production line.

View full abstract

In this paper, we discuss a method for automatic construct system of an image inspection software for production engineer who has not enough knowledge of image processing programming. In the industrial field, an automatic program generator or expert system are expected to shorten the time from ordering of individually designed and manufactured appearance inspection system to delivery completion. So-called “image processing expert system” have been studied for over the nearly 30 years. We are convinced of the need to adopt a new idea. Taguchi method(TM) is one of the most popular technological development technique in the factory field. We have adopted this method to the image inspection processing expert system. So production engineers can make image inspection algorithms only by their production knowledge.
It was verified from experiments that the proposed method was able to create inspection image processing programs. In the experiment result, our method could improve the success rate of almost all automatic visual inspection algorithms.

View full abstract

This paper proposes a system to assist the visually handicapped by extracting text characters on the surface of an object. The visually handicapped user cannot know exactly what the object is and cannot read the text written on the surface. The user touches a point on the surface with his or her index finger; the system extracts characters around the fingertip. To implement these functions, we propose fingertip detection and character extraction methodologies. For fingertip detection, the user wears a ring, in which three acrylic infrared-reflective beads are embedded, on the index finger. The system detects the beads and locates their positions from an infrared image, then, calculates the fingertip position based on bead arrangement. For character extraction, a region is generated according to the position of the fingertip. The characters are extracted from the region based on color histogram and shape. Then, the extracted characters are combined as text. Finally, the text is recognized by commercially available OCR software and is read aloud to the user by reading software. We have experimentally verified that these methodologies achieved character extraction accuracy of at least 70.0% expect the case of narrow character extraction. The experiments demonstrated the basic validity of our proposed method.

View full abstract

Person re-identification is the task of finding and matching the same individual in different camera views. For robust person re-identification, we propose a weighted feature integration method that adapts to illumination changes and appearance differences caused by different camera views. First, we extract four kinds of local features (color histograms, frequency features, gray-level co-occurrence matrices, and histogram of oriented gradient features) from the image of a person as cloth appearance information. Second, in the pre-training phase, we calculate the difference in value of each local feature for a pair of images taken by different cameras. Local features are then weighted and integrated based on these differences. We tested three weighting functions: reciprocal, probability density function, and average Bhattacharyya distance. In the experiments, we utilized four public datasets, iLIDS-VID, GRID, PRID, and VIPeR, and verified the effectiveness of the proposed method. The results demonstrate a general improvement in person re-identification performance when the feature integration is weighted by the average Bhattacharyya distance.

View full abstract