This paper describes a dexterous and robust manipulation system with a multi-fingered hand. Dynamic interactions between the finger-tips and the grasped object together with those between the grasped object and the environment easily cause manipulation errors and task execution errors. Pose errors of the manipulated object is toosmall to be detected in the servo cycle of the finger control. However such small errors will grow rapidly to cause task execution error unless compensation is correctly made. We have developed a task evaluation module which observes the pose change of the object in a longer time span depending on the motion primitive of the multi-fingered hand. Error-recovery strategies are developed and successfully executed based on the task evaluation. Result of experiment is finally shown.