Diffusion models are a
generative AI architecture that learns to generate images, audio, or video by adding noise to real data and then training to reverse this process. It's like learning to restore a damaged photograph step by step until recovering the original image.
These models work through a two-phase process. First, during training, they take real images and gradually add noise until converting them into pure noise, as if we progressively blurred a photograph until it becomes unrecognizable. Then, the model learns to do the reverse process: removing the noise step by step (a process called denoising) to recover sharp images.
Once trained, the model can start from random noise and gradually transform it into a coherent image following the instructions you give it through
prompts. It's similar to a sculptor starting from a shapeless block and refining it to reveal the desired figure, but in this case guided by textual descriptions and applying progressive denoising.
Diffusion models have revolutionized visual content generation and are the foundation of popular tools like Stable Diffusion, DALL-E, and Midjourney. Their ability to generate high-quality and diverse content has made them the standard for creative applications, design, photo editing, and multimedia content generation. They are also being successfully applied to audio, video, and 3D model generation.
They are currently the dominant architecture for image generation, playing an equivalent role to
transformers in language processing.