On the 50th Anniversary of IIEEJ, this paper looks back the 50 years of the history of IIEEJ, including the history of image technology, market, and standardizations of these 50 years. The future surveys of these points are also described.

View full abstract

Since images are evaluated by human, image processing should be designed to reflect human visual characteristics and subjective evaluation. In this talk, human-centric image processing that emphasizes such human elements by machine learning is introduced, focusing on the research of the speaker. Especially methods to consider human preference and subjective evaluation by utilizing deep learning and interactive evolutionary computation are presented with examples of the result images.

View full abstract

In today's online commerce, the easy distribution of counterfeit products, such as food, brand-name products, and pharmaceuticals, has become a serious social problem in terms of economy and safety. To address this problem, we propose a system for determining the authenticity of 2D codes printed by inkjet printers; it leverages the micro-scale nature of the ink sequences in the codes, which are difficult to duplicate. The proposed system combines locally likely arrangement hashing, which performs a fast search for similar images, and Accelerated-KAZE, which performs high-precision feature matching, to determine the authenticity of images taken with a smartphone. The results of our study demonstrated the high effectiveness of our system in terms of accuracy, discriminability, and speed by validating it on a large dataset of 15,000 images. The system was also validated for industrial applications by verifying its scalability for low-cost industrial media printing and monochrome printing as well as its robustness to rotation, scale change, and smudging.

View full abstract

Aortic valve disease, one of the cardiac diseases, causes symptoms such as short breath, anginal pain, and heart failure due to valve malfunction. The aortic valve disease needs a surgery, called “aortic valvuloplasty”, which rarely causes the postoperative thrombogenesis and requires no medication. The surgery, however, is very difficult and requires preoperative planning and simulations. In this paper, we propose a particle based simulation using a model generated from X-ray CT data. In the simulation, blood flow from the left ventricle to the aorta is visualized, and the pressure changes in the aorta and the left ventricle are compared with those in the real data. After the left ventricle contracts, the aortic valve opens, the blood flows from the left ventricle to the aorta, and the left ventricular pressure increases. As the blood is ejected from the left ventricle to the aorta, the aortic pressure rises. Then, as the left ventricle relaxes, the left ventricular pressure decreases below the aortic pressure, and the aortic valve closes. As the result of the simulation considering the interplay between the left ventricle’s contraction and relaxation, and the valve’s opening and closing, we have confirmed that the pressure changes in the aorta and the left ventricle have become very similar to the real data.

View full abstract

When generating a virtual reality (VR) space using omnidirectional images, it is desirable to use images in which the camera photographer does not appear in the image. In this paper, we propose a method to exclude the photographer from an omnidirectional image by using multiple images taken by an omnidirectional camera. In the proposed method, the photographer first takes multiple omnidirectional images while rotating around the omnidirectional camera. Next, feature point matching is performed on the multiple omnidirectional images, and the appearances of all the omnidirectional images are unified using the amount of translation calculated from the feature point matching. Finally, the images are color-corrected and complemented to produce an omnidirectional image excluding the photographer.

View full abstract

In order to realize realistic and smooth movement and work in a 3D virtual space, one of the major challenges is how to realize an interface that can effectively give a sense of agency to movement and work. “Self-viewing”, which allows you to see your own body when you look at it in virtual space, is a very important factor for obtaining a sense of immersion. It is also considered to be a key point for obtaining a sense of agency. We are trying to build a unique interface that enables smooth movement in the virtual space by conducting experimental verification of the method and the main factors that give a sense of agency. We take up bicycle riding as a practical task in a 3D virtual space and we are developing an experimental interface which can provide a sense of agency during bicycle riding. The interface displays the avatar of the operator's hands and steering wheel synchronize with the steering of the bicycle in the field of view as an example of “self-viewing”. In this report, we describe an overview of the results of experiments on the correlation between avatar display methods, driver posture stability, driving control stability, sense of agency, etc. The results of a quantitative evaluation method for sense of agency, and further reports on the effectiveness of a subjective interface that utilizes such “self-viewing” are also described.

View full abstract

A self-written memo by Father Gustav Voss, the founder of Eiko Gakuen, was discovered and analyzed and considered. The memos were written in English, German, Romaji, and Japanese, but most were in Romaji. For that analysis, I tagged it with XML, prepared for future consideration, and tried to give an overview of Principal Foss's life in a matrix resume.

View full abstract

Due to the recent the COVID-19 pandemic, remote meetings have become commonplace. As a result, the chances of oneself participating in a remote conference have increased, and in videos such as news, the scene where an actual person has a dialogue with the person displayed on the monitor has also increased. So far, studies have been made on what size the person on the screen should be displayed in order to realize natural communication in a video conference. However, the case where there is an actual person next to the person projected on the screen has not been examined. In this study, in the above case, we examined the size of the person on the screen from the scale of size recognition and incongruity.

View full abstract

People perceive depth on a daily basis because of the presence of both eyes. The factors that contribute to depth perception include binocular disparity, motion disparity, and convergence. In our previous studies, we quantified the visual effects of size and the amount of information in the visual field, and conducted experiments to evaluate CG images displayed on a display. In these experiments, the relationship between the depth of the content and the sense of discomfort was demonstrated by using inverse-viewing (switching the left and right images) as a means of evaluation. In this study, as a new attempt of research on depth, we consider depth perception using inverse-viewing when walking in a virtual space. As an experiment, we created two types of rooms in a virtual space and subjectively evaluated the difference in depth brought about by the addition of information and movements in the virtual space in the forward-viewing state and the reverse-viewing state.

View full abstract

The use of real-world walking for movement in virtual reality (VR) space is an effective interaction technique for creating a natural and realistic VR experience. However, one drawback of using real-world walking is that users cannot move within a virtual environment that is larger than the tracking space. To solve this problem, research has been conducted to scale up or down the amount of movement and rotation in the VR space to give the user the illusion of moving through a large space, and adding tactile presentation has been used for more effective manipulation of spatial perception. However, from the viewpoint of casual enjoyment of VR experiences involving actual walking, it is still necessary to reduce the space required for tracking. In addition, a locomotion interface that enables unlimited walking by offsetting the amount of movement by actual walking has been studied, but it is not easy to use because it requires a large-scale device. In this study, we propose a method to realize infinite walking in a VR space by using a rotational walking using an easily accessible chair, and evaluate the sensation of this method.

View full abstract

A 3D point cloud is a representation of a 3D space with a large number of discrete coordinate data obtained by laser or photogrammetric surveying. The processing of 3D point clouds enables us to directly analyze features in 3D space, which is expected to have various applications. A contour tracking method for 3D point clouds has been proposed by Daniel et al. This method is intended for surface segmentation and surface meshing, and performs surface fitting on a set of 3D points that show geometric shapes, and traces contours on a set of points projected on the edges that exist at the boundaries between surfaces. However, this method is not necessarily suitable for contour tracking of edges of 3D point clouds that represent general natural shapes, such as those obtained at surveying sites. In this work, we propose a new method of contour tracking for edges of 3D point clouds representing natural shapes, and show its effectiveness on drawing map of the remains through experiments on actual remain data.

View full abstract

development of robots that can run independently in the same space as humans has been progressing. Most of the current navigation methods for autonomous robots use geometric information. While this method is easy to implement, it has some drawbacks, such as the need for accurate self-positioning and vulnerability to errors in sensor information. Humans do not have an exact self-position, but rather, they recognize an approximate location based on surrounding information and envision the path to the goal to reach the destination. Therefore, this study examines a navigation method that plans a route based on an abstract map, such as a hand-drawn map, and then travels independently to the destination.

View full abstract

This research aims to realize a communication system to improve the sense of presence in online entertainment and education, which has become widely used in recent years. In this paper, we examine and evaluate methods for creating and manipulating 3D avatars using depth cameras and smartphone cameras.

View full abstract

This study aims to realize a new online teaching material by XR to enhance learning effectiveness in online education, which is widely practiced in the Corona Disaster. This research aims to realize a new online educational material using XR to enhance the learning effect in online education, which is widely used in the Corona Disaster. In this paper, we use cooking as an example to construct a 3D model of an ingredient to be used as an educational material and evaluate a posting method that improves the realism of the model for users.

View full abstract

In Japan, the proportion of the elderly person exceeds 25% in 2015 and continues to increase. Then, the number of missing elderly due to dementia is also increasing. Many local governments are supporting the early detection by establishing watching service. However, further early detection is necessary because it takes time to detect by manual search based on pre-registered information on physical characteristics. Therefore, we aim to develop a technology that supports the early detection and notification to the searcher of the watching service by automatically detecting the target person from multiple surveillance cameras based on their pre-registered gait information when searching. Since occlusion caused by the static objects such as guardrails occurs in the surveillance camera images, it is necessary to develop a collation algorithm robust to occlusion. In this paper, we evaluated the accuracy of previous methods using face, whole-body and gait information. In addition, we propose a gait recognition that enables high accuracy by adaptively selecting features according to occlusion. As a result of evaluation using originally made occlusion images, the proposed method achieved the highest accuracy with an average of 83.4%. We will aim at the practical application by evaluating with the actual surveillance camera images.

View full abstract

Promotional videos by YouTubers are used for corporate advertisements and public relations by government and public organizations. For effective video promotion, the videos are needed to be viewed widely. The objective of this study is to clarify the requirements for the promotion videos to be viewed widely. For this purpose, we first focus on four representative channels and compare the number of views for the promotion videos and that for the other videos. Then, we analyzed the characteristics of the titles and contents of the videos with the higher and lower number of views, respectively.

View full abstract

This paper presents a method for measuring the working time of employees who work on a PC using a webcam. In the situation of an online meeting where multiple people join in a camera frame, it is difficult to monitor the employees applying conventional face recognition techniques because the target faces are too small in the frame and they freely change the face direction. The proposal captures a frame per min. to the PC user and transforms it into a black-and-white image recognizing the skin color. Then, the image is discriminated into either of working or not by utilizing run-lengths of every scanning line in the image. The working time is measured by counting continuous frames discriminated as working. In the experiments, the performance of the discrimination was examined with a movie where four people join in the camera frame of a single webcam, and it was confirmed that the proposal could perfectly discriminate all the frames.

View full abstract

This paper presents image features for evaluating the progression of gastric atrophy from gastric X-ray images. In the proposed method, after the diagnostic area is extracted, the areas where the gastric wall looks rough are extracted according to the diagnostic index for reading the atrophy. Then, the proposed features measure the number and the size of the rough areas in the diagnostic area. In the experiments, the performance of the proposed features were examined with 84 images. The experimental results showed that the features are effective well to evaluate the progression of gastric atrophy.

View full abstract

Over the last few decades, threats to urban safety have increased worldwide. Security solutions aim to detect, track, and identify people using many videos take by security cameras. Video is the most effective information for arresting criminals. Although the number of security camera installed and the opportunities to use them are increasing year by year, investigators manually collect data and analyze videos, which takes time and effort. Therefore, in recent years, a technique for identifying a person using AI has been desired. The purpose of this research is to develop a technique for accurately searching for a person from security cameras, utilizing the height and the size of belongings. The height of a person and the size of an object in the video can be measured based on the coordinates in the image if it is possible to acquire the camera parameters (external parameters such as installation position and installation angle, internal parameters representing optical camera characteristics). Previous research has proposed a method for estimating camera parameters, however, the method has not been evaluated for videos in a shooting conditions of security cameras. Therefore, the purpose of this study is to improve the conventional method by using videos that meets the shooting conditions of security camera from the viewpoint of the depression angle of camera and number of people shown in the video.

View full abstract

360-degree cameras are beginning to be widely used for recording and presenting a wide range of scenes such as street view, drive recorders, property introductions, and drone photography. However, most of them use fisheye lenses with large distortion, which often makes it difficult to apply general image processing algorithms. Therefore, although it has the advantage of being able to capture a wide range of scenes, it is not actively used as a three-dimensional shape measurement or various image processing sensors. In this study, I propose a method for detecting straight and parallel lines directly from 360-degree images, and a method for estimating the axial direction and camera orientation of the detected parallel lines. In addition, we simply calibrate the camera internal parameters using horizontal and vertical lines, and report the experimental results comparing the estimation errors of the camera attitude in two ways, calibrated and uncalibrated.

View full abstract

This paper proposes a new method for estimating the attributes of a pedestrian, e.g. an office worker or a student, from the pedestrian's clothing and, based on the estimation results, a method for efficiently navigating an autonomous mobile robot while safely avoiding pedestrians. The robot can estimate the organization the pedestrians belong to by learning in advance the clothing of pedestrians that frequently appear in the city in which the is travelling, e.g. the uniforms of certain high school students. Furthermore, the robot is given the direction in which the person in that organization is likely to be heading at the current time, so it uses this information to estimate the future walking trajectory of that pedestrian and design a running path that is less likely to prevent that pedestrian from walking. We use the artificial potential method to design the robot's running trajectory. Even if pedestrians with different attributes are detected in the robot's running path, our proposed robot can efficiently reach its destination while avoiding those pedestrians appropriately in real time. The effectiveness of the proposed method was confirmed by subjects' experiments with our prototype robot.

View full abstract

In recent years, with the spread of smartphones, the number of SNS users in younger and older age groups with low Internet literacy has been increasing. LINE, Twitter, and Instagram are some of the SNSs with high usage rates in Japan. Twitter is widely used to gather information on the latest news and current events. However, there are many cases in which tweets containing false or uncertain information are widely spread, and this has become a social problem. For example, there was a nationwide shortage of toilet paper due to the spread of false tweets about a new type of coronavirus. The spread of inaccurate information that can cause people to act incorrectly is a major problem for Twitter, which users increasingly use with low Internet literacy. To prevent users with low Internet literacy from mistakenly recognizing hoaxes as correct information, we are constructing a system to extract and present tweets considered hoaxes and tweets that correct hoaxes, especially for tweets related to the novel coronavirus, which can be life-threatening. In this study, as the first step of our attempt, we analyzed tweets containing "hoaxes" and "lies.

View full abstract

According to a survey conducted by the Ministry of Internal Affairs and Communications, Japan, social media usage increased rapidly from 20.3% in 2012, the following year of the Great East Japan Earthquake, to 86.9% in 2019. At the time of disasters such as the Kumamoto Earthquake (2016) and Northern Osaka Earthquake (2018), information such as safety confirmation and evacuation status reports were shared via SNS. In some cases, SNS could be used even when telephone and e-mail services were unavailable, and the presence of SNS is increasing as a means of information during emergencies, along with television and radio. Twitter and Facebook, which allow information to be shared with an unspecified number of people, receive an enormous number of postings immediately after a disaster. In fact, during the first week after the Kumamoto earthquake, approximately 26.1 million earthquake-related tweets were confirmed. With such a large amount of information circulating, it is challenging to provide helpful information to each disaster victim in a short period. Against this background, this study aims to extract disaster-related tweets from those posted on Twitter at the time of a disaster and to support the decision-making process of disaster victims who need local information.

View full abstract

In recent years, neural radiance fields (NeRF) has dramatically improved the performance of novel view synthesis using perspective images. However, no study has addressed the problem of applying NeRF to 360° images in ERP format. For 360° images in ERP format, NeRF’s general ray sampling strategy is ineffective due to the characteristics of spatial distortion in high latitude regions and a 360° wide viewing angle. Therefore, we propose two non-uniform ray sampling schemes, distortion-aware ray sampling and content-aware ray sampling, to make NeRF applicable to 360° images. We created an evaluation dataset using Replica and SceneCity models of indoor and outdoor scenes, respectively. In experiments, we show that our proposed method successfully builds NeRF for 360° images in terms of both accuracy and efficiency. Furthermore, we showed that NeRF++ combined with our proposal accurately synthesizes arbitrary views from a set of 360° images of real scenes.

View full abstract

Medical wireless telemeter is device that monitor patient biometric information using radio waves. For trouble-free use of medical wireless telemeter, it is important to manage the radio wave environment to ensure that there is no electromagnetic interference and that radio waves are received with sufficient strength. Generally, measurement and evaluation of the radio wave environment is done by measuring the received voltage using the simple spectrum analyzer function installed in the central monitor. The central monitor does not have a function to output the received voltage as a numerical value, and the investigator visually reads the received voltage from the graph displayed on the central monitor's display. On the other hand, there are a total of 480 channels available on the medical wireless telemeter. Visually reading and recording the received voltage of all channels is very labor intensive and has a large visual reading error. Therefore, we considered a method to obtain the received voltage of each channel by outputting the simple spectrum analyzer screen displayed on the display from the DVI terminal of the central monitor and analyzing the image.

View full abstract

This paper proposes a new method for estimating the depth of an observation space from a monocular image based on a catadioptric imaging system using an omnidirectional camera and a spherical mirror. This method uses a catadioptric imaging system with a curved mirror, which enables observation of a large space at a time, and uses only one omnidirectional camera as the sensor of the imaging system. Therefore, the method does not depend on learning and can acquire a wide range of 3D information with a compact device. Since 3D information is useful for 3D scene recognition, the technologies for acquiring and displaying 3D information are rapidly spreading in various fields. Laser light and multi-view images are known as sensing methods for 3D information, and estimation technologies based on deep learning continue to develop. However, each of these methods requires large-scale equipment or observation systems, or prior information. We compare our method with other methods and confirm its effectiveness through demonstration experiments using images taken in an environment constructed using CG.

View full abstract

This paper presents a method for recognizing the same tumors between CT images before and after a therapy for patients with hepatocellular carcinoma. This recognition is necessary for examining the effectiveness of the therapy with a computer-aided diagnosis. In the proposed method, after a contour of the liver region is corrected, 10 feature points are evenly prepared on the liver contour. And then, the distance between each feature point and each tumor area is measured to determine the location of the tumor area in the liver region. To a tumor area in a slice image before the therapy, the closest tumor area in the slice image after the therapy is selected as the same tumor. Both of the slice images are the same turn from the head of each liver. In the experiment, the accuracy of the proposed method was visually evaluated using CT images of 12 cancer patients and 323 couples of tumors were successfully recognized among 325 couples.

View full abstract

The quality of document images acquired by scanners is degraded by noise and lossy compression. Since the degradation of document images leads to deterioration of visibility and text recognition accuracy, color correction is an important process to improve the quality of document images. In this study, we propose a color correction method for two-color printed document images. The proposed method consists of hue correction and edge-preserving smoothing. The proposed method assumes that each pixel in a two-color printed document image has the same hue as the ink color or achromatic. Therefore, in an undegraded image, all pixels lie on a constant hue plane. The proposed method first finds the constant hue plane with the minimum error and corrects the hue by projecting each pixel onto the constant hue plane. Next, edge-preserving smoothing is performed to suppress small changes in intensity and saturation in flat areas and thus remove noise-induced degradation. In experiments, proposed color correction is performed on document images to verify the effectiveness of the proposed method. The aim of this study is not to achieve perfect color reproduction of images acquired by a scanner but to convert them to natural-looking images with little degradation by performing color correction processing.

View full abstract

In recent years, the number of endoscopic surgery has been increasing in Japan. Accordingly, the amount of works of surgeons other than operation has also been increasing. Surgeons has opportunity to explain to others about the surgery they performed. For example, in the case of laparoscopic cholecystectomy, the average operation time is about 70 minutes. Surgeons makes a short video which include the scene they want to mention about 2 minutes long for case conference in hospital and about 20 to 30 seconds long for medical conference presentation. The purpose of this study is to develop a video editing system which helps surgeons to select important scenes based on some artificial intelligence (AI) algorithms. In this presentation, we will introduce an overview of the AI used in the system and report on the current state of this system development.

View full abstract

[in Japanese]

View full abstract

An increasing number of municipalities are engaging in citizen-participatory / citizen　collaborative city planning, in which citizens and the government work together to solve various community issues. As part of such activities, there has been an increase in the number of cases where citizen reporting systems utilizing ICT devices, such as PCs and smartphones, have been introduced. For example, Chiba City in Chiba Prefecture, Japan began operating an application called Chiba-Repo in 2014, which allows residents to post issues in the city. Currently, a service called My City Report is being developed as an extension of Chiba-Repo. We have jointly developed a road condition reporting system with Hiratsuka City in Kanagawa Prefecture, Japan, based on a disaster information sharing system using Twitter implemented in our previous study. The system has been operating as an official system of Hiratsuka City, called “Michirepo,” since March 2019. This paper introduces the functions and features of Michirepo, a road reporting system using Twitter, and reports its operation for about three years since the service was launched.

View full abstract

Novel Coronavirus infections (COVID-19) have spread since 2020, and a variety of information on infectious diseases has been shared on Twitter. In Japan, the vaccination began in February 2021, however false information about the vaccine and posts (tweets) denying or correcting the false information have been shared since that time. In order to suppress social confusion, it is necessary to prevent the spread of false information. Therefore, it is crucial to clarify the characteristics of tweets denying or correcting false information. In this study, we focus on “infertility” and “microchip” as keywords related to false information about the vaccine, and collect tweets containing each keyword. Then we divide those tweets into those that deny or correct the false information and others, and compare the tweets with a large number of retweets. Moreover, we clarify the characteristics of the posters of the tweets that deny or correct the false information.

View full abstract

There is great demand for tools that enable people to explore various regions of the world. Google Street View (GSV), an interface that integrates images and map data, is a typical example of such a tool. Movie Map, on the other hand, is a tool that visualizes target areas using 360-degree videos from a walker's viewpoint. Unlike GSV, Movie Map can better convey the atmosphere of the place by using videos instead of images. However, current Movie Map is only used as an archive built from previously recorded videos. This study aims to add live features to Movie Map. The goal of this study is to improve Movie Map from a mere exploration tool to a tool with entertainment value that allows users to have realistic experiences. This study introduces the following two approaches for realizing this goal. (1) Embed live views in some areas of Movie Map. (2) Switch to 360-degree live video at specific spots in Movie Map.

View full abstract

In baseball, pitchers, at a wide range of levels are suffering from severe elbow injuries. To reduce the risk of developing elbow injuries, pitchers should find and grasp technical tips for minimizing the valgus stress. This technical improvement is not easily done primarily due to that the pitcher cannot self-recognize how large the valgus stress applied to the elbow joint during pitching. We present our first attempt to find a way to automatically estimate elbow joint valgus stress from the video-images taken by a smartphone. Pitching motions of 13 adults with various heights, weights and baseball experiences were recorded by a smartphone and a motion capture system simultaneously, and the elbow valgus stress of each pitch was computed for both data sets with an inverse dynamics approach. The comparisons of the corresponding two sets of valgus stresses showed that the average difference was 15.31%. In the future, the application of the smartphone including the user interface will be developed, and the servicing will be examined.

View full abstract

In recent years, many electric and electronic devices are operated by remote control devices or operating buttons on the device itself. However, these methods give the operator a somewhat constrained feeling. In order to improve these problems, researches have been conducted to make hand gestures an intuitive user interface and a virtual device that can operate devices. This allows users to use electronic and electric devices without being constrained by a sense of constraint. In this research, we focus on hand detection and shape recognition to operate devices using hand gestures. In this case, we aim to obtain motion stability by limiting the area to be recognized using a monocular camera and focusing on the operator of the device. Using mouse operation as an example, we set up three hand gestures and use a cascade classifier and Yolov5 to perform detection. We prepare datasets for each of the three handshapes to be used for gestures, and perform training. At this time, we compare the differences in detection results depending on the conditions of the datasets, and also evaluate the datasets.

View full abstract

Hyperspectral cameras are VNIR sensors that can detect the electromagnetic spectrum incident from an object with high wavelength resolution and have recently been expected to be applied in various fields. In this study, we applied hyperspectral cameras to the construction field and investigated the possibility of extracting areas of interest and estimating the approximate amount of moisture contained in the embankment. Hyperspectral data of fill under construction taken with the cooperation of actual field workers were used for the data. For the extraction of regions of interest, we confirmed that if the average teacher spectrum of the target region could be prepared, it would be possible to extract regions that are like the spectral waveform. Next, regarding the estimation of moisture content, we created a prediction model consisting of ensemble learning such as XGBoost using teacher data labeled by moisture content for the extracted fill and obtained a classification accuracy of 94%. These results suggest that hyperspectral cameras have potential applications in the construction field.

View full abstract

The method of using a drone to reconstruct a large-scale scene in 3D is a widely used method in 3D urban modeling. The specific process is to acquire an aerial multi-viewpoint image in the target city using a UAV (Unmanned Aerial Vehicle) and restore the 3D city point cloud model by matching the feature points. The aerial multi-viewpoint image obtained in this way requires sufficient overlap between adjacent images. Large-scale urban modeling takes a lot of time and effort because the UAV needs to fly to capture as much scene information as possible to reduce the effects of occlusion and noises. In this paper, we propose a layer-based point cloud complementation method that can extract the shape of an urban building and complement a scene with occlusion such as walls using only a small number of aerial ortho-mosaic views. The accuracy of the complementary result was verified by the city model acquired in the virtual reality space.

View full abstract

[in Japanese]

View full abstract

In recent years, computational photography technology can achieve expressions by computational calculation. As part of this technology, it is now possible to synthesize multiple camera images into 360-degree images. Ricoh has developed the world's first consumer 360-degree camera, THETA, which generates 360-degree images by combining two low-parallax fisheye lenses through the stitching. This paper will explain the advances in this technology. In addition, business use of 360-degree cameras is expanding, particularly in the real estate and construction industries, taking advantage of their ability to capture images without blind spots. These services consist of image processing and recognition processing.

View full abstract

In recent years, computational photography technology can achieve expressions by computational calculation. As part of this technology, it is now possible to synthesize multiple camera images into 360-degree images. Ricoh has developed the world's first consumer 360-degree camera, THETA, which generates 360-degree images by combining two low-parallax fisheye lenses through the stitching. 360-degree images are widely used as advertisements of real estate. However, taking 360-degree images is time-consuming work for real estate agents because they also take conventional photos. Therefore, this paper will explain auto-cropping which reduces the amount of time for taking conventional photos by cropping conventional images from a 360-degree image.

View full abstract

Two-color pyrometer technique with a high-speed camera was conducted　to measure the combustion flame temperature on the solid fuel of a hybrid rocket. The combustion flame above 2600 K formed at a chamber pressure of 2.0 MPa was analyzed using the Hottel & Broughton method based on the red and green luminance recorded by a high-speed camera. The study revealed high-speed phenomena on the fuel surface, which had previously been obscured. It is also presented that image analysis by the two-color pyrometer method is a powerful technique for measuring the high temperature formed on the solid fuel surface.

View full abstract

The morphing method is very useful in animation field. The morphing method is divided into mesh morphing and field morphing. The proposed method is one kind of the field morphing. The vectors are used for specify the features. In most cases, boundaries of object image are curved, so curves instead of vectors are useful. The features are specified by the distance from the vectors. Our method uses Bezier curves for specifies the features of images. The distance from the curve is defined by Bezier function, and the finding the closest point is easy by using the control points of the distance function.

View full abstract

Replica materials are often exhibited in museums. To make 3 dimensional replica models, 3D printers can be used effectively in limited schedule. Examples of speech articulators’ models for exhibition purposes are explained.

View full abstract

Historic buildings around the world are often outdoors and are fading away every moment. We propose a 3D shape measurement method using a drone and LIDAR as one of the methods to preserve such historical buildings. The use of a drone is easy and can be used in high places, and LIDAR can generate accurate 3D models. In this proposal, a model is set up in the laboratory to simulate outdoor work. OPEN3D is used for point cloud processing and 3D model generation, and a comparison between the LiDAR 3D model and the photogrammetric 3D model will be made.

View full abstract

When creating a three-dimensional point cloud from photographs, a method of creating a three-dimensional point cloud that includes a cube by placing a cube as an indicator in advance is being considered. However, detecting cube regions from a huge amount of point cloud data is computationally expensive. In this study, we propose a method to extract only the cube regions quickly and accurately by a two-step filtering process using the color information assigned to the acquired point cloud, and demonstrate the effectiveness of this method.

View full abstract

3D point cloud processing is a field of computer vision and has been applied in various fields in recent years. When reconstructing a 3D point cloud from a sequence of video images, if a moving object exists in the frame, it remains in the reconstructed 3D model, and the accuracy of the reconstruction of the entire point cloud is degraded. In this study, we examined a method of detecting moving objects using optical flow and Detectron2, masking the moving objects, and verified the effectiveness of this method.

View full abstract

Analysis of the 3D structure of wires in electrical cables is a necessary technique for verifying various characteristics of cables and nondestructive inspection using X-ray CT is used for this purpose. However, due to the limited resolution of X-ray CT images, each wire in the images is blurred, making it difficult to accurately extract the 3D structure of thousands of wires from thousands of CT images taken in the longitudinal direction of the cable. In this study, we propose a method for estimating the 3D structure of all wires in a cable from a 3D X-ray CT image of the cable. The proposed method first detects wires in each image using Faster R-CNN. It then tracks each wire in a set of cross-sectional images collected in the longitudinal direction of the cable to estimate the structure as a 3D trajectory of wires. Faster R-CNN is trained using our synthetic image dataset that simulates the wire region. It achieves wire detection without the need of annotating the training data. The results showed that wires can be detected with an error of about 1%, and 40% to 80% of the wires can be tracked continuously over 500 frames.

View full abstract

By using barn images taken by a camera installed on the ceiling of the barn, the system automatically detects mounting behavior, which is a clue to detecting signs of estrus in dairy cows, through image recognition.It is difficult to collect actual mounting data because mounting behavior occurs only rarely.In this paper, we learn a two-class classification of whether or not an image is a mounting image by using the overlapped image of two dairy cow images as a pseudo-mounting image.

View full abstract