Fundamental Concepts of Self-Attention in Generative AI

Abstract

With the rapid development of generative AI, the Transformer architecture―centered on Self-Attention―has emerged as a fundamental technology across various fields, including natural language processing and image generation. This paper provides a step-by-step explanation of the mathematical foundations of Self-Attention, focusing on the roles of Query, Key, and Value, the computation and normalization of similarity scores using the Softmax function, and the generation of output vectors through weighted averaging. In addition, the architectural design and theoretical principles of Attention mechanisms within Transformers are reviewed to clarify the central role of Self-Attention in generative AI. Through the use of equations and illustrative figures, this work aims to support intuitive understanding and to contribute to the foundational knowledge required for future applications in the field of medical image analysis and visualization.