In the present paper, we propose a selection method of retrieval words on an apparel material retrieval system using impression-words. In old retrieval systems, since technical terms were used chiefly as retrieval words, beginners could not use the systems satisfactorily. However because of a decrease in experts, a retrieval system that any user can operate is demanded recently. To develop the retrieval system, which uses an impression-word as the retrieval word, we propose a refinement method of effective retrieval words. The retrieval words were decided through two refinement processes. First, we collected 1, 184 raw words used daily from three kinds of famous newspapers, then we selected 50 fine impression-words from them by the use of a frequency value. Next, we classified the fine impression-words into 15 appropriate impression fields that were decided based on the contents of those articles. To determine the effective retrieval words, we refined the words by factor analysis and coefficient of variation, then we obtained 22 refined retrieval words. A prototype retrieval system for apparel material database has been constructed to confirm the utility of the proposed method. The evaluation experiment has shown that the system had a high retrieval hit rate and high user interface performance. Accordingly, the utility of our proposed method was proven.