In the research field of Human-agent interaction, the uncanny valley is the crucial issue to realize co-existence of human and artificial agents. It is referred to as the phenomenon that human can feel repulsive against the agents whose appearance is considerably humanlike. There has been just theoretical based but not verifiable model providing an explanation for how it occurs. We hypothesized that when human observes that humanlike agent, s/he perceives it as both human and non-human, and the contradiction between the perceptions causes him or her to elicit negative response against it. We conducted the experiment where the participants were asked to judge whether face of agents or a person was depicted as that of a real person, with their eye tracked and their gaze direction estimated. The results indicated that observers had two-step information processing to the agent. Above all, we proposed a model generating the human negative response against humanlike agents, taking into consideration of the function of the brain regions such as amygdala, prefrontal cortex, hippocampus, and striatum. To verify the model, the transition of the emotional value (namely, positive or negative) was simulated on the basis of a qualitative description for the model. It is suggested that the model be proposed which is verifiable with many findings in the field of neuroscience.