In this paper, we report our efforts and challenges on the TRECVID ad-hoc video search (AVS) task. The goal of the AVS task it to build a zero-shot video retrieval system using a complicated query phrase. Our system has the following two characteristics. First, we prepared a large number of pre-trained concept classifiers in advance that can detect various kinds of objects, persons, scenes, and actions. This strategy contributes to improve the word coverage rate of keywords in query phrases. Second, we selected additional concept classifiers by natural language processing techniques such as using word similarities or synonyms. We submitted our systems with these two characteristics to the TRECVID AVS task in 2016 and 2017, and one of our systems ranked the highest among all the submitted systems for the second consecutive year.

This paper improves the accuracy of fisheye stereo camera by correcting images using a disparity offset map. A disparity offset map is a map of disparity errors that are calculated from the disparity of the feature points obtained from an object with known distance. By correcting the disparity errors and aligning the images, errors due to the insufficient calibration of the camera parameters, etc. are reduced. A disparity offset map is constructed and the accuracy of distance measurement of a fisheye stereo camera using the disparity offset map is evaluated by experiments.

This paper presents a method to measure foot shape from a 3D point cloud as input captured from multiple directions using a smartphone depth camera. Such a 3D point cloud could potentially include noise or omit parts of the foot due to occlusion. To deal with this occlusion problem, we propose to use a dataset of 3D foot shapes collected by a precise 3D shape scanner of foot shapes. According to the dataset of 3D foot shapes, we can generate a deformable model by performing a principal component analysis (PCA) on the dataset. Then we minimize the error of the shape represented by the deformable model and the 3D point cloud acquired by the smartphone camera, to recover a complete 3D shape of the entire foot with high accuracy. We test this method by comparing the 3D shape produced by our proposed method to the 3D shape precisely measured by the 3D scanner. Our proposed method can scan the foot shape with an error of about 1.13mm.

This paper improves accuracy of fisheye camera's intrinsic parameter estimation method using camera rotation by adding constraint to the camera rotation. The method takes advantage of trajectories of feature points in the scene. The trajectories of feature points are obtained by a rotation movement of the camera in a specific plane. The method therefore can utilize rich feature points for calibration, and furthermore, specific calibration targets are not required. The effectiveness of the proposed method is evaluated by simulations and experiments using a real fisheye camera. The improvement of the proposed method is verified by converting fisheye images to perspective images using the estimated parameters. Comparison with an existing method is also performed.

This paper proposes a method which measures individual behavior in a group from images of swimming salmon fry. This measurement has various difficult problems such as water surface reflection, object rotation, and replacement with other objects. To solve these problems, the proposed method uses Color OCM (Orientation Code Matching), rotation angle calculation by OC values, and motion prediction. Color OCM reduces reflection noise on the water surface. Rotation angle calculation by OC values solves rotation of object. Motion prediction prevents replacement with other objects. This method succeeded in measuring individual speed, acceleration, and group trajectory. The behavior measured by proposed method is an important guideline for inspecting the growing condition. The experimental results shows the new correlation between individual behavior and group. It is thought to be a new guideline for analyzing the ecosystem.

In recent years, the demand for pedestrian detection using LIDAR is increasing, as it can be used to prevent traffic accidents involving pedestrians. To avoid traffic accidents, detection of distant pedestrians is very important. However, they are scanned sparsely even if a dense-scan LIDAR is used, and this causes the degradation of the detection accuracy. There-fore, pedestrian detection from sparsely-scanned LIDAR point-clouds is expected to be developed. This paper proposes a LIDAR-based pedestrian detection method using 3DCNN. Since it is difficult to train a 3DCNN directly from sparse point-clouds, the proposed method converts them to a voxel representation using the kernel density estimation based on LIDAR characteristics. To evaluate the performance of the proposed method, an experiment using real-world LIDAR data was conducted. The results showed that the proposed method could detect pedestrians more accurately than detectors trained with other conventional features.

This paper proposes a Deep Learning-based action recognition method from an extremely low-resolution FIR image sequence. The method recognizes daily actions by humans (e.g. walking, sitting down, standing up, and so on) and abnormal actions (e.g. falling down) without privacy concerns. While privacy concerns can be ignored, it is difficult to compute feature points and to obtain a clear edge of the human body from an extremely low-resolution FIR image. To address these problems, this paper proposes a Deep Learning-based action recognition method whose inputs are the FIR images and their frame differences cropped by the gravity center of human regions.

In this paper, we address the problem of performing natural paste synthesis by color adjustment and image completion, in order to solve the completion problem that can specify an object appearing in a completion area. We propose a synthesis network that can extract the context features of the input image and reconstruct an image with the feature, making the inserted object appear in the completion region. In addition, we propose a ingenious method to make input images and learning method using Generative Adversarial Network (GAN) that do not require collection of high cost learning data. We show that color adjustment and image completion based on context features are executed at the same time, and natural pasting synthesis can be performed by using these proposal methods.

We propose a novel method for synthesizing training samples to obtain high accuracy of object detection under the condition that the number of acquisition images is small. The convolutional neuronal networks for object detection require the large number of acquisition images that the angles of postures of each object are varied. Thus, it is very time-consuming to collect training samples. On the other hand, our method synthesizes training samples from the small number of aspect images that determine the variation of appearances of objects. We design how to collect aspect images based on the knowledge that there is a bias in the postures of objects. Experimental results show that our method significantly reduces the number of acquisition images while keeping high detection accuracy of a comparison method that requires the large number of acquisition images.

In this paper, we propose a method to discriminate multiple materials with high-speed based on the spectral reflectance properties of each material in near-infrared ray (NIR) band. A method to discriminate multiple materials with high accuracy using the same properties had already proposed. The previous method discriminates materials by using multiple binary classifiers generated by using PLS1 which is one of the Partial Least Squares (PLS) regression analysis algorithm. However, the method has a problem that it has complicated processes and takes long processing time. In order to solve this problem, we used PLS2 which is one of the PLS regression analysis algorithm just like as PLS1. In this proposed method, multiple materials can discriminate with simpler processes than the previous method by generate a multiclass classifier to discriminat e multiple materials. As a result of an experiment, multiple materials were discriminated with almost the same accuracy as the previous method, and the processing speed was improved approximately five times faster than the previous method.

Smoke detectors use point detection, and depending on the position of a sensor relative to the source of a fire, there is the possibility of a warning being delayed. On the other hand, with security cameras placed in various locations these days, video smoke detection is thought to be capable of detecting a fire more quickly than smoke detectors. In this paper, with the aim of early detection of smoke using images from security cameras, fire tests with six types of fire source conditions and backgrounds that provided the maximum and minimum contrast with the color of smoke were conducted. For the captured video, learning and classification were carried out on a linear SVM that used HOG feature vectors as training data. And regarding the distance from the separating hyperplane to a sample, it showed that there is a relative change in the space where the smoke occurred. Furthermore, it was clarified that it is possible to detect smoke efficiently by setting and comparing two random points within the space.

This study aims to explore methods for the active, early detection of smoke from the initial stage of a fire or a smoldering fire. Methods to detect smoke and fires from images have been tested, but in the case of environments where the visible light provides insufficient illumination for surveillance, the necessary light levels and contrast cannot be obtained, giving rise to the possibility that the system will not perform adequately. It is shown that by using a depth camera to project a discrete pattern of near infrared light into an area, paired with a camera with spectral sensitivity tuned to the same band of wavelengths to detect when that pattern is attenuated, it is possible to sense changes in the concentration of particles such as smoke and mist. In the study, fire tests were carried out using three fire sources and, using a NIR projector and NIR camera, clusters of particles that indicates a rising plume of smoke were visualized. Then, smoke candidates using a particle filter were found, and it was shown that it is possible to detect smoke from the candidates tracked.

For industrial products and foods, it is essential to conduct a visual inspection to improve the quality of products. In recent years, automation by a neural network has been considered but learning a neural networks requires a lot of good and defective samples. However it is so difficult to ensure a lot of defective samples that neural networks cannot learn properly. In this paper, we aimed at discrimination of defects under conditions where there is a large number of good products and a small number of defective products. By combining AAE, which can extract features following any distribution and Hotelling's T-Square, which is an effective anomaly detection method when data follows a normal distribution, it is possible to discriminate defects under a small number of defective samples. We experimented on 2 dataset and showed the effectiveness.

In the production line, template matching is used as a technique for positioning parts. In template matching, it is a general problem to be high recognition accuracy and high speed processing. So far, to achieve this problem, a method of selecting several% of pixels effective for positioning from a template has been proposed. However, there is a problem that the method can not be applied unless the object has strong features such as edges. In this research, we propose a method to solve this problem by using Deep Neural Network. It is assumed that Deep Neural Network can detect not only strong features of objects but also weak features, because it can detect defects in visual inspection and identify animals. Therefore, we pay attention to the feature map of the feature extraction unit of Deep Neural Network and we make a hypothesis that features which are effective for class identification remain in the feature map. In this research, reference pixels are determined based on this hypothesis. As a result of experiment using 4000 images, we confirmed that the recognition rate of the proposed method is 97.7%, which is about 27% higher than the conventional pixel selection method.

Demand for teleoperation of construction machines at disaster sites is increasing to prevent the secondary disasters. In teleoperation, operaters can see the environment using cameras mounted on the construction machines. However, there is a problem that the camera images have blind spots occluded by obstacles such as the arm of the construction machine itself, which cause a decrease of work efficiency. This paper presents a method to generate images with few blind spots by seeing through foreground obstacles and visualizing background objects. The proposed method uses two RGB cameras and LiDAR to acquire texture data with the three-dimensional information of the environment. And then, it projects the background image acquired by one camera to the other camera by using the three-dimensional information, and integrates the two camera images into a see-through image. In the experiment with an actual construction machine, we succeeded in generating see-through images in real time.