The Effect of Knowledge Editing Methods on Confidence Calibration

Abstract

As the scale of language models continues to grow, research on knowledge editing methods has become increasingly active, as these methods can update knowledge without retraining the entire model. However, since knowledge editing directly modifies the token prediction probabilities acquired during pretraining, it may cause inconsistencies between the model’s token probabilities and the token distribution learned from the training data. As an adverse effect of this inconsistency, the model’s confidence, which represents how strongly it believes its outputs are correct, often diverges from accuracy. For example, the model’s confidence becomes higher relative to accuracy in Reinforcement Learning from Human Feedback (RLHF)．In this study, we applied confidence calibration, which measures the degree of deviation between confidence and accuracy, to language models before and after knowledge editing to investigate the effect of knowledge editing on the model’s confidence. By applying three different knowledge editing methods to a total of ten language models, we found that particularly in tasks requiring semantic understanding, if editing is successful, the increase in token prediction probability after editing was smaller than the actual improvement in accuracy. This indicates that, unlike post-training such as RLHF, successful knowledge editing tends to cause underconfidence, where the model’s confidence is lower than its accuracy. Moreover, when knowledge editing was applied to models that had already undergone RLHF, the opposing tendencies of the two processes appeared to offset each other, suggesting a potential improvement in confidence calibration.