Generation and editing of facade images from illustrations controlling Attention in the diffusion model.

Abstract

In determining facade design, designers spend a great deal of effort considering multiple ideas. On the other hand, image generation technologies have made significant progress in recent years, and especially SDEdit can generate high-quality and creative images from illustrations. However, this method makes it difficult to connect the relationship between colors and text within an illustration, making it difficult to generate and edit facade design images composed of various materials as intended. Therefore, we propose a facade design generation and editing method that links the relationship between color and text. Specifically, the Attention mechanism used in diffusion model changes the importance of words depending on the colors in the illustration, and switches the reference text depending on the editing point. From the verification, it was indicated that the proposed method reflects the pre-specified color and text information, and can generate and edit images more in line with the user’s intentions.