Editing Façade Design with Stable Diffusion

Abstract

The process of facade design by designers involves creating a base plan for their own design to begin with, and then sublimating it by incorporating the client’s request and other factors. Current image editing methods are mostly GAN-based, in which latent representations are acquired and edited using an inversion model. In recently, however, research using Diffusion Models has come to the fore. Specifically, Latent Diffusion Model (or Stable Diffusion based on it), which targets latent space, is easy to handle and accurate. In this paper, we proposed a system that uses Stable Diffusion together with Attention-based Layout conditioning and CLIP’s output conversion model to achieve detailed and target image editing with only simple prompts. Through the evaluation by actual designers, we confirmed the usefulness of the proposed system and future work.