This study looks at one way to improve the effectiveness of a Japanese vocabulary simplification system for leaners from non-kanji backgrounds. The system uses the Simple Japanese Replacement List to rewrite difficult vocabulary automatically. However, until now there has been no verification of whether the replaced items were actually simpler for JFL/JSL learners. To verify this, an experiment was conducted with learners from non-kanji backgrounds on the effects of replacing difficult nominal verbs with easy Japanese. The results show that the effect of simplification varies with the type of replaced word(s). These results indicate a need to modify the replacement list.