"Super-resolution" is not only a key word with its own active research area but is also used in sales messages for new consumer products such as HDTV. Of the many proposals for super-resolution image reconstruction, the total variation (TV) regularization method seems to be the most successful approach due to its sharp edge preservation and no artifacts. The TV regularization method still has two problems. One is the large computational time, and the other is insufficient texture interpolation. In this paper, we propose a system that solves these problems. In our system, the number of TV regularization processes is smaller than that of the conventional method, and the learning-based method is introduced in place of texture interpolation. The learning-based method is another super-resolution approach. This paper proposes combining the TV regularization and learning-based methods. The experimental results show that our approach performs well and reduces computational time while being robustness to the input noise.

View full abstract

We describe an efficient access control method for digital videos that controls access to video frames based on one-way hash functions. The method offers effective key management and delivery as well as conventional scalability-aware access control methods for coded still images. It uses hash chain-based key generation, thereby limiting the number of managed keys to one, and the user receives only one key as well. This method controls access to a video sequence based on frame rates, and it is applicable to access control of videos based on movie ratings. For access control with two controlled subjects, our method reduces the number of keys to the theoretical lower limit.

View full abstract

Keyword-based image retrieval (KBIR) from WWW image database enables users to obtain a lot of images corresponding their query keywords. However, when users need images that have severely limited features, KBIR is an inefficient option. Content-based image retrieval (CBIR) is proposed for solving this problem, but it requires that users prepare their query images, which is also time consuming. We have developed a new easy-to-use method to create precise query images only from the keywords. In our method, the query keywords are divided into those for KBIR and those for representing specific features. Then, original features of images from KBIR are modified into the extracted features. From these synthesized images, users can easily choose the query image that best represents what they want with relevance feedback. The experimental results show that our method enables users to obtain proper query images more easily than conventional methods.

View full abstract

We describe a method for clipping a rectangular region from an image by minimizing the weighted intersection of two color histograms that are constructed with pixels inside and outside the rectangular region. In the clipped image, the main object in the original image is relatively zoomed up. Experimental results showed that this proposed method can clip the object regions from images and remove the background regions. The proposed clipping method is also applicable to videos.

View full abstract

In accordance with the recent improvement in the quality of image displays, digital image compression artifacts are more visible than ever. Moreover, a lot of studies have been done to remove the artifacts such as blocky noise and mosquito noise. Among them, the total variation (TV) regularization approach proposed by Alter is considered to be one of the most successful. In this approach, the TV is regularized under constrained conditions, making it possible to efficiently remove the artifacts included by quantizing DCT coefficients.
In this paper, unlike Alter's approach, an image is decomposed into a structure component and a texture component using the ROF TV regularization, and blocky noise and mosquito noise are moved in the texture component. Then, by filtering it using the deblocking edge filter, blocky noise can be removed. Furthermore, by controlling the selective filters using edge information obtained from the structure component, mosquito noise can be removed. Also, the reconstructed image is obtained to compose a filtered texture component and a structure component. An advantage the proposed method has over Alter's approach is it removes the artifacts without removing small texture signals. The experimental results show that the proposed method produces fine images subjectively and objectively. Also, the proposed method can be applied for not only JPEG-compressed images but also DCT-based compressed images such as MPEG and H.264.

View full abstract

We have developed a method to enhance small cloth stains, such as weak coffee stains, that are slight in color and gray level. The proposed method is composed of four steps. First, we obtain two color images observed under two types of light (near-ultraviolet LEDs and white LEDs). Second, we create the RGB components of the color images to be uncorrelated Using principal component analysis. Third, we transform the uncorrelated components into independent components to obtain more enhanced images. Finally, we reduce the texture structure from the independent component image by Using sparse coding in order to recognize the cloth stain more easily. Experimental results demonstrate that the proposed method is effective for enhancing weak cloth stains in texture images.

View full abstract

The out-of-focus blurred JPEG images are degraded by being out-of-focus as well as blocking artifacts and mosquito noise caused by quantization on block DCT domain. The purpose of this paper is to sharpen out-of-focus JPEG images without enhancing the blocking artifacts and the mosquito noise. This proposed method is based on the edge-adaptive restoration method: the regularizing operator depends on the edge orientation, and the regularizing parameter depends on the local activity. The variance of the quantization on block DCT domain is taken into consideration.

View full abstract

View synthesis using depth maps is a crucial application for Free-viewpoint TV (FTV). The depth estimation based on stereo matching is error-prone, leading to noticeable artifacts in the synthesized new views. To provide high-quality virtual views for FTV, we innovatively introduce a probabilistic framework that constrains the reliability of each synthesized pixel by Maximizing Likelihood (ML). Our spatial adaptive reliability is provided by incorporating Gamma hyper-prior and the synthesis error approximation using reference crosscheck¹⁾. Furthermore, we formulate view synthesis in the framework of Maximum a Posterior (MAP). For the outputs, two versions of the synthesized view are generated: the solution with ML criterion and the solution with MAP criterion, solved by straightforward interpolation and graph cuts, respectively. We experimentally demonstrate the effectiveness of both solutions with MPEG standard test sequences. The results show that the proposed method outperforms state-of-the-art depth based view synthesis methods, both in terms of subjective artifact reduction and objective PSNR improvement.

View full abstract

In this paper, we propose a semi-automatic depth estimation algorithm for Free-viewpoint TV (FTV). The proposed method is an extension of an automatic depth estimation method whereby additional manually created data is input for one or multiple frames. Automatic depth estimation methods generally have difficulty obtaining good depth results around object edges and in areas with low texture. The goal of our method is to improve the depth in these areas and reduce view synthesis artifacts in Depth Image Based Rendering. High-quality view synthesis is very important in applications such as FTV and 3DTV. We define three types of manual input data providing disparity initialization, object segmentation information, and motion information. This data is input as images, which we refer to as manual disparity map, manual edge map, and manual static map, respectively. For evaluation, we used MPEG multi-view videos to demonstrate that our algorithm can significantly improve the depth maps and, as a result, reduce view synthesis artifacts.

View full abstract

We propose a robust background subtraction method for multi-view images. Our method uses an approach for integrating multi-view images in which the background region is determined using voxel information rather than each camera image itself. We introduce a likelihood of background to each pixel of camera images, and derive integrated likelihood in the voxel space. The background region is determined on the basis of minimization of energy functions of the likelihood. Furthermore, the proposed method also applies a robust refining process, in which each silhouette is modified on the basis of projections of a 3D-model to each viewpoint and a 3D-model is reconstructed using modified silhouettes. Experimental results show the proposed method to be more effective than the existing methods.

View full abstract

Ultra-high resolution video is expected to be one of the next-generation high quality video formats. Current video coding technology H.264 achieves the maximum amount of coding efficiency from among the existing coding standards. However, even H.264 is not sufficient enough for ultra-high resolution video distribution service due to a lack of coding efficiency. Therefore, an enhanced video coding technology is strongly required to improve the coding efficiency. In the past works, several approaches for extending the macroblock (MB) size have been proposed, and these researches showed that extended MB size technology improved the coding efficiency, especially for high resolution video. However, a clear description of the performance improvement mechanism has yet to be presented. We analytically consider the coding characteristics of an extended MB size scheme for ultra-high resolution video in this paper. We clarified the coding gain mechanism of this technology based on a R-D characteristics analysis. Furthermore, we analytically confirmed that the extended MB size scheme was most effective at high-resolution and low-bitrate video coding. Finally, we conducted a coding experiment and found that the maximum bit reduction ratio reached approximately 15% using an optimal coding control for the extended MB size scheme.

View full abstract

This paper proposes a novel coding method that reduces correlation of 4:4:4 chroma format in high-resolution images more than HDTV. The proposed method compensates for Intra prediction errors by applying the linear prediction to a certain Intra prediction error. The adaptive inter-channel prediction uses a coefficient that minimizes the MSE of the Intra prediction errors in each channel. The experimental results show that combining the Intra prediction and inter-channel prediction has better coding efficiency than the conventional method.

View full abstract

We previously developed a method for improving a target image quality based on web images. The target image is conventionally improved by using similar blocks for reference that are generated from web images. However, if bordering blocks have the same reference blocks, the improved image might have an unusual pattern. Therefore, we propose a method for selecting reference blocks based on web images for image quality improvement. Experimental results showed that use of the proposed method improves image quality.

View full abstract

Recent advancements in digital archiving technologies have enabled us to store greater amounts of priceless digital data related to cultural properties. However, accessing this information by using conventional database systems is too difficult for general users. To solve this, this paper proposes an associating, retrieval and display system for archeological information using huge 3D models of cultural heritage as interfaces to access information. To define target regions on 3D models, users can quickly select specific regions from huge models by using an interface that combines a splitting method by graph-cut and a lasso tool. In the display process, we achieved highly interactive rendering of huge 3D models by using an efficient 3D rendering algorithm on the basis of multi-resolution meshes.

View full abstract

In real environments, the detection and motion of an interesting moving object against a complex background in computer vision are very important. However, many undesirable factors that can prevent stable detection arise in a scene. The factors are, for example, occlusion, lighting change and jittery background. In addition to these, an infinite number of randomly moving particle-like patterns can make moving objects more difficult to detect. These particles have ambiguous edges and no definite shape. In a more complicated scene, we assume that such particles cannot be smoothed out by a simple preprocessing such as a low-pass filtering. Thus, they can distribute optical flow of moving objects in time and space. To deal with these issues for matching between frames, it is first assumed that particle-like patterns and moving objects have properties of fluidity and rigidity, respectively. The image brightness change and motion smoothness between frames can constrain the estimated optical flow of a moving object. However, a local large brightness change caused by the above factors violates such constraints. Thus, on the basis of a statistical approach, a robust optical flow estimation method has been developed by adding a locally parallel flow contraint. Estimated flow is used to count the number of multiple moving objects with a clustering method, which is applied starting with a large number of centorids. Centorids are iteratively merged within a distance until converged. Experimental results show that our method outperforms previous methods, thus validating our proposed method.

View full abstract

Holographic video display requires high-speed calculation. Our previous study, showed that changing the hologram type from Fresnel to image generates holograms very quickly. In the image hologram, the object is located very close to the hologram, or superposed. Therefore, for the single object point, required computation area for the image hologram becomes much smaller than the entire hologram. In this paper, we have investigated applying the distance look-up table method for the image hologram. Since image points are located close to the hologram, the required look-up table size also becomes much smaller than that of the Fresnel hologram. From the experimental results, generation speed of 19 frames per second has been obtained for a full color image hologram that has a resolution of 1,400 x 1,050 and 10,000 object points.

View full abstract

Photography has an expression technique that lets a person see the main subject closely by shading off background to put the main focus on the main subject, such as a person. However, its depth of field is deep and cannot satisfy the demand of the user to adequately shade off the background of the subject because almost all digital cameras have an imaging element only 1/5 size of that a film camera. Therefore, in this study, a method was developed to change the depth of field of digital camera images to that of film cameras by Using firmware.

View full abstract