The present study investigates the design of interaction of agent and learners' affective states during an activity on a large scale web-base tutoring task. We especially focus on how learners' and agents characteristics such as gender influence emotional states of the learner. Also, we analyze how those emotional states are influenced impressions towards the agent. The types of emotions were analyzed based on Russell's(1980) two-dimensional circumplex model. Results of two experiments consistently revealed that male students were sensitive to the pleasantness bipolar and female students were sensitive to the activation bipolar. Further investigations also show that female learners with arousal states evaluated the agent positively. Male learners with pleasure feelings evaluated the agent positively in some degree.