Towards Reverse-engineering the Kanizsa Illusion: A Mechanistic Study of Vision Transformer

Abstract

The formation of illusory contours has been associated with predictive processing, yet its detailed mechanism remain unclear. In this study, we show that the Kanizsa illusion can also be observed in Vision Transformers, a class of feedforward neural networks, thereby challenging the conventional understanding of their formation. To elucidate the underlying mechanism, we introduce a novel mechanistic interpretability method leveraging a diffusion model to track how predictions evolve across transformer layers. Finally, We discuss the universality of mechanisms between models and biological systems and the potential of our approach to contribute to a deeper understanding of the illusory contour formation in biological systems.