This paper proposes a method for registering X-ray images with its 3D CT model by estimating 3D point clouds from X-ray images and their corresponding points on the image. Many conventional methods generate a simulated X-ray image from a 3D CT model and optimize the pose by using the similarity metrics between the simulated X-ray and the input X-ray image. On the other hand, deep learning approaches that predict pose information need a canonical coordinate system defined manually on the pre-operative CT to properly utilize the estimated pose. Therefore, we devise a fully automatic registration pipeline that is independent of coordinate system, by recovering 3D point clouds from X-ray images, estimating the corresponding points on the images, and aligning them with the given 3D CT model.

Image enhancement of jewelry is a difficult task because of the shape of the jewelry, its color, background elements such as shadows and glass stands, as well as the blurring of the boundary between the jewelry and the background and unique light reflections. Our preliminary results indicate that CycleGAN is effective in correcting jewelry images and that background elements in jewelry images adversely affect jewelry image correction. In this study, we propose a method to correct jewelry images with strong background elements. The results show that the target consistency of TC-ShadowGAN is effective not only in removing the background but also correcting the jewelry area in the image. In addition, data augmentation with Balanced Consistency Regularization (BCR) and Dense Consistency Regularization (DCR) are applied to increase the accuracy of the correction of the jewelry area.

Semantic segmentation is an important technique in various applications, such as autonomous driving, medical imaging, and industrial inspection. Depth estimation, as one of the important components of scene understanding, can be used to obtain effective depth information while utilizing only RGB images. In recent years, such depth information has been used as an auxiliary feature to facilitate the semantic segmentation task. This study proposes a Simultaneous Fusion Network(SF-Net) that simultaneously learns semantic segmentation and depth estimation tasks based on a monocular camera image. The features are first extracted and strengthened by injecting contextual information using semantic labels through the feature reinforcement module and then learned simultaneously by analyzing the imaging process to establish the relationship between the size and depth of the objects in the image. A new loss function is represented by the geometric relationship. Furthermore, a feature fusion module is constructed to perform image feature fusion on the common parts of depth estimation and semantic segmentation tasks. By learning simultaneously, the accuracy of semantic segmentation can be improved by utilizing the depth information obtained from depth estimation inference. We conducted experiments using the Cityscapes dataset and the NYUDv2 dataset and verified the effectiveness of the proposed method.

In this paper, we propose a novel bit depth enhancement (BDE) model that considers semantic contextual information by incorporating spatial feature transform (SFT) layers into the BDE model. In the proposed method, we adopt the pixel-wise class probability maps of the input image obtained by semantic segmentation as prior information for SFT. The SFT layer transforms from the input feature maps into the modulated feature maps by considering the contextual information modulated using affine parameters generated from the prior information. The proposed method considers the pixel-wise details by a proposed network that preserves spatial dimensions and the contextual information by incorporating the SFT layers conditioned with semantic information. Moreover, the proposed method adopts a perceptual loss function to recover the visually natural luminance changes by considering the contextual information. The experimental results show that the proposed BDE method achieves better performance compared with existing DNN-based BDE methods for restoring 8-bit depth from 3,4, and 6-bit depths. In addition, we investigate how to provide the contextual information to the BDE model, and show that providing it through the SFT layer is effective compared with other providing methods.

Digital video signal processing for varieties of purposes including storage as well as transmission is currently playing a very important role in our daily life. NEC Corporation has been actively doing R&D on video coding since its dawn and incessantly making every effort for implementation of practical codecs, terminals, and systems. They include Delta Modulation for digitization, HO-DPCM for high quality TV transmission, and interframe coding for a broad spectrum of audiovisual services. The author believes it is quite beneficial to introduce a combined history of coding algorithm improvements and practical codecs based on them, since they will provide the overview of the technical history from theoretical and also practical aspects, and definitely there is no such review published so far. This survey paper consists of two parts and reviews the corporate R&D efforts with respect to video coding algorithm improvements and our contribution to progress in digital TV/video transmission services over the world for three decades since the mid-1960s. Part -1 (Chapter 1 and 2) mainly handles continued improvement of Delta Modulation and coding algorithms on intraframe and interframe prediction as well as entropy coding. Part - 2 (from Chapter 3 to 6) mainly shows effectiveness and significance of video coding through various application examples of codecs/terminals as well as standardization activities in which we were deeply involved. In addition, Part-2 includes a brief history of NEC’s CODECs in conjunction with continued algorithm improvements and several examples of practical use in businesses.

Digital video signal processing for varieties of purposes including storage as well as transmission is currently playing a very important role in our daily life. NEC Corporation has been actively doing R&D on video coding since its dawn and incessantly making every effort for implementation of practical codecs, terminals, and systems. They include Delta Modulation for digitization, HO-DPCM for high quality TV transmission, and interframe coding for a broad spectrum of audiovisual services. The author believes it is quite beneficial to introduce a combined history of coding algorithm improvements and practical codecs based on them, since they will provide the overview of the technical history from theoretical and also practical aspects, and definitely there is no such review published so far. This survey paper consists of two parts and reviews the corporate R&D efforts with respect to video coding algorithm improvements and our contribution to progress in digital TV/video transmission services over the world for three decades since the mid-1960s. Part -1 (Chapter 1and 2) mainly handles continued improvement of Delta Modulation and coding algorithms on intraframe and interframe prediction as well as entropy coding. Part -2 (from Chapter 3 to 6) mainly shows effectiveness and significance of video coding through various application examples of codecs/terminals as well as standardization activities in which we were deeply involved. In addition, Part-2 includes a brief history of NEC’s CODECs in conjunction with continued algorithm improvements and several examples of practical use in businesses.