Recent advances in deep learning have dramatically improved the performance of instance segmentation, which is the task of predicting object area in images. However, depending on the shape of the target object, precise detection may still be difficult. For example, linear objects such as Wires still pose challenges for accurate instance segmentation due to their unique characteristics about shape. Therefore, we propose an instance segmentation method to accurately detect linear objects. The proposed method focuses on the characteristics of linear objects: continuity and irregularity of shape. We attempt to accurately detect linear objects by using Smooth Loss, which evaluates continuity, and Edge Enhance Loss, which focuses on the correctness of contours. In addition, we propose an evaluation metrics using the distance between contours to evaluate the accuracy of contour prediction. The proposed instance segmentation method improves by around 12% the average performance of contour prediction on the iShape dataset.

In this paper, we propose preliminary action recognition which is an action anticipation method that can be used in real-time. Preliminary action recognition as a solution to action anticipation, which is the task of detecting the action of a person before the person performs the action. We define preliminary action class before action class that is anticipated by recognizing the preliminary actions specific to the action before it occurs, just as humans do. Conventional action anticipation methods have two problems for real-time use. First, it requires 10 to 20 seconds of video to anticipate 1 second ahead, and it can only anticipate every 10 seconds. Second, the video input in training always switches actions 1 second after the end of the video. This problem setting can be a leakage because there is no action break in real-time operation. Proposed method solves this problem and makes it possible to operate action prediction in real-time. In addition, since action classes and preliminary action classes tend to share a lot of similarities, it is difficult to distinguish between them. We propose a model that can discriminate the subtle differences in motion by increasing the latent distance between adjacent classes in terms of time.

In recent years, deep learning models have achieved great success in many fields, but their training relies on large labeled data. Annotation to obtain labeled samples is challenging, time-consuming, and expensive, and has been a major challenge in deep learning. In this study, we propose a method for unsupervised domain adaptation that can reduce annotation cost in terms of accuracy and time compared to conventional methods by using the uncertainty of the model in semantic segmentation. Unsupervised domain adaptation aims to adapt models trained on synthetic data to real-world data without the need for costly annotation of real images. Since the training data are images and correct labels created by the game engine, there is no need to manually annotate them. However, if the source (synthetic) data is trained using conventional supervised learning, the performance is significantly degraded because the domain is different from the target (real image) data.This study closes the domain gap between source and target by calculating the uncertainty in the target data from the model output on a pixel-by-pixel basis and minimizing this uncertainty as loss. Adding this uncertainty loss to conventional unsupervised domain adaptation methods results in a model that is more robust to the target domain and achieves state-of-the-art results.

In recent years, image and text retrieval in the zero-shot learning setting has increasingly become more accurate by utilizing pre-trained image and text embedding models. By adapting these models to the task of video retrieval, we were able to achieve the world's highest retrieval accuracy in the 2022 ad-hoc video search task of TRECVID, a video retrieval and evaluation benchmark conducted by the National Institute of Standards and Technology. This paper reports on an investigation and experiments to determine the effectiveness of a number of trained image and text embedding models currently available to the public for video retrieval tasks, using a large set of over 1.4 million test videos used in the TRECVID benchmark.

This paper proposes an object-aware skeleton-based anomaly detection method for surveillance videos. The previous skeleton-based anomaly detection approaches learn to reconstruct normal skeleton patterns solely from the skeleton information. However, such methods suffer from detecting object-related abnormal behavior, which has a similar skeleton pose to normal behavior (e.g., riding bicycles/motorcycles). To improve the detection accuracy of such anomalies, we propose incorporating the information of objects (bounding boxes and class labels) around humans. The object and skeleton information are jointly processed through an encoder-decoder RNN to reconstruct the information. We evaluate the proposed method on the HR-ShanghaiTech dataset and achieve an accuracy improvement of 3.1%, reaching 78.2% in the best model.

Anomaly detection aims to detect unusual patterns and samples in a training distribution. In this domain, many researchers have paid attention to anomaly detection models using ImageNet-pretrained weights. Among them, PaDiM is a promising approach that detects anomalies based on the feature distribution. While such approaches have achieved significant results, they tend to overlook global information due to the texture bias caused by ImageNet-pretrained convolutional models. Therefore, in this paper, we propose incorporating Fast Fourier Convolution, which can extract global information in the frequency domain, into PaDiM. This proposed model is named Fourier-Convolutional PaDiM (FC-PaDiM). Our FC-PaDiM is able to extract global features from frequency space and local features from feature space for more accurate anomaly detection. In our experiments, we demonstrated that our proposed FC-PaDiM allowed for extracting local and global features compared to PaDiM. Moreover, our additional analysis revealed the robustness of perturbations in frequency bands in the MVTecAD dataset.

In this study, the accuracy of food region extraction, which is used to obtain image pixels in the food regions from food images and remove pixels in the background regions such as table and plate, was improved. In the proposed method, state-of-the-art saliency detection, which is used to predict pixels that attract human gaze based on the deep neural network (DNN), and semiautomatic segmentation, which is used to iteratively refine food and background regions by using graph theory, are employed. The comparative experiment demonstrated that the proposed method significantly increased the mean F-measure, which is a comprehensive evaluation metric, over that of conventional saliency-based food region extraction by 9.15% by reducing the erroneous determination of the background as the food region. Furthermore, the F-measure was higher than that of UNet+, which is a DNN-based semantic segmentation trained on a well-known public image dataset. This paper comprehensively details the analysis of performance improvement.

With the growing demand for dietary control in health management, studies that estimate categories and ingredients from food images have been conducted. Especially, recent studies have achieved high recognition accuracy through multi-task learning of categories and ingredients. Although the accuracy for frequently used ingredients in each category (typical ingredients) is high, the accuracy for infrequently used ones (untypical ingredients) has room for improvement. This paper proposes a method to improve the estimation accuracy of untypical ingredients in multi-task learning by estimating typical and untypical ingredients in separate modules. Experimental results on three datasets show the effectiveness of the proposed method in terms of the accuracy for atypical ingredients.

We propose a method for extracting temporal features robust to headwear variations for person identification using the video sequences of body sway. When people put on headwear such as caps and helmets, their head shapes, observed from an overhead camera, change dramatically depending on the type of headwear. The existing method cannot obtain high accuracy of person identification in situations where the head shapes change because their features are directly affected by the headwear variations. We perform a learning-based low-pass filter for the time-series signal of head center positions representing body sway to extract our temporal features robust to the headwear variations. Experimental results show that our temporal features significantly improved the accuracy of person identification when the headwear variations occur, compared to the existing features.

This study aims to automate the sensory inspection of the wood grain. Sensory inspections rely on human perception, which requires considerable experience to master. Additionally, verbalizing and quantifying the inspection indices makes it difficult to unify the evaluation among inspectors. To overcome these challenges, we propose a method that models the cognitive mechanisms of skilled inspectors to automate sensory inspections and clarify potential factors of sensibility. We confirmed that our proposed method is more effective for the sensory inspection of wood grain images than conventional anomaly detection methods. Moreover, our experiments demonstrated the potential usefulness of our proposed method in the field of wood quality control.