This paper proposes a measuring method of facial expression and emotion by using CCA (canonical correlation analysis). The CCA analyzes correlation between two kinds of quantities of facial expression images and subjectively evaluated values of valence and arousal. We introduce gazing property of humans for observing facial expression images into the proposed method and inspect its efficiency. We also propose a method for performing CCA of singular matrices by using intermediate variables. We also employ a kernel CCA method with Gaussian kernel functions and verify its effectiveness by comparing it with linear CCA.