Abstract
In failure problem of soil, localization of deformation, progressive failure and unstable failure is dominated. Those phenomena affect peak load and post-peak behavior. In this research, we direct our attention to unstable failure and suggest a condition for beginning of unstable failure by using J-integral. As a result of finite element analysis which is applied for the suggested condition, we can reproduce different failure phenomena for rock shear test in various vertical load. And it is shown we can reproduce an unstable failure in rock shear test and progressive failure test.