Deep demosaicking considering inter-channel correlation and self-similarity

Abstract

The effectiveness of utilizing inter-channel correlation and self-similarity for demosaicking has been reported in many literatures. On the other hand, many convolutional neural network (CNN)-based demosaicking techniques have also been proposed to achieve state-of-the-art accuracy. In CNN-based demosaicking, one of the most important issue is how to consider the correlations using neural network. In this paper, we propose a novel CNN-based demosaicking method that considers an effective combination of both inter-channel correlation and self-similarity. Specifically, we apply the CNN to predict the color differences R-G and B-G, then the demosaicked image is obtained from the predicted color differences and the input color filter array (CFA) image. At the same time, our network considers the self-similarity in the color difference domain by applying non-local attention for high-level feature map. Experimental results show that our method provides the better accuracy and visual performance compared with conventional demosaicking methods. In addition, the versatility of the proposed framework is demonstrated by experiments with images sampled by various CFA patterns.