Observing numerical results obtained by Granular Element Method, we discuss the incremental stress-strain behavior of granular materials. We assume a periodic boundary for the numerical experiments to avoid disturbances caused by boundary controlling. Stress-probe tests starting with the intermediate data of a bi-axial compression test give a set of incremental stress-strain relationships. The result indicates that the flow direction for plastic loading is not constant as is assumed in the classical plasticity theory and depends on the direction of the stress increment. Comparing the numerical results with a proposed flow rule, we conclude that the origin of the direction dependency is the volumetric component of plastic strain.