Applying a conventional image inpainting method only once does not always generate plausible textures, and this problem is usually solved by repeatedly applying image inpainting with manual reselection of failure regions. This paper proposes a method that automatically detects failure regions in a resulting image using a convolutional neural network.