A neural network as a unified model for explaining image features for processing various types of “shitsukan”

Abstract

Natural visual stimuli contain rich “shitsukan”, such as glossiness, translucency, and material of an object. Explaining various types of shitsukan in a unified framework is difficult because in general visual perception involves numerous features. Here we tried to explain visual features for shitsukan perception by analyzing the experimental data of shitsukan discrimination tasks. We assumed that participants responded based on the image features of the stimuli. The features were calculated by a deep neural network (DNN) optimized for image classification, in which more complex and abstract features are represented in the higher layers. The features in the middle layer best explained the participants’ responses, suggesting that relatively complex features are used for shitsukan perception. We also found that the effective features depends on the type of the shitsukan. These results suggest the effectiveness of a DNN for explaining visual features for shitsukan perception.