Abstract
Modal instability due to spillover is investigated for a flexible vibration control system that employs a modal filter as a state estimator. First the characteristic equation of the system equipped with actuators, sensors and a modal filter is derived in a determinantal form. Then it is shown that the characteristic equation can be expanded. By applying the perturbation technique to the expanded characteristic equation, a criterion is obtained to predict whether a residual mode become stable or unstable. Using the criterion, it is mathematically proved that the control system equipped with one actuator and one sensor always has unstable modes if they are non-colocated.