DALL-E
OpenAI's text-to-image models. The first (January 2021) generated images token by token with a Transformer; DALL-E 2 (April 2022) switched to diffusion guided by CLIP.
Also DALL·E · DALL-E 2
Ramesh et al., OpenAI, 2021
Type 'an armchair in the shape of an avocado' and get pictures of exactly that, something no one had ever drawn before.
Where it fits
+All relationships as a list
Is a kind of / part of
Uses / built on
How it works
On 5 January 2021, OpenAI introduced DALL-E, named after Salvador Dalí and Pixar’s WALL-E. It treated an image like a sentence: a discrete autoencoder turned images into a grid of tokens, and a 12-billion-parameter Transformer learned to generate those tokens after a caption, just like a GPT generating text.
On 6 April 2022, DALL-E 2 switched approach. It used CLIP: a “prior” model turned the caption into a CLIP image embedding, and a diffusion model decoded that embedding into a high-resolution picture. Images became far more realistic.
Together with Stable Diffusion, it made text-to-image generation a mainstream tool, and raised the questions about artists’ rights, misinformation and consent that the field still debates.
Everyday use cases
- Case 01
How you've probably met it
Image generation inside OpenAI's products, and the viral "avocado armchair" images that introduced many people to text-to-image AI.
Strengths, weaknesses and when to use it
Strengths
- Showed that combining concepts never seen together was possible from text alone.
Weaknesses
- Early versions struggled with text in images, counting and complex compositions.
When to use itAs a landmark in text-to-image generation.