Segment Anything
Meta's promptable segmentation model: click on anything in an image and it returns that object's outline, even for things it was never specifically trained on.
Also SAM · SA-1B
Kirillov et al., Meta AI, 2023
Point at something in a photo, anything at all, and it carefully traces around exactly that thing for you.
Where it fits
+All relationships as a list
How it works
Classic segmentation models learn a fixed list of classes. Segment Anything, released by Meta AI in April 2023, instead learns a general notion of “an object” and lets you say which one you mean with a prompt: a click, a box or a rough mask.
It has three parts:
- A heavy image encoder (a Vision Transformer) that processes the image once.
- A small prompt encoder for your clicks or boxes.
- A fast mask decoder that combines both and outputs masks in a fraction of a second, so you can click around interactively.
The training data was as important as the model: the SA-1B dataset of about 1.1 billion masks on 11 million images was built with a “data engine” in which the model helped annotators and, in the final stage, generated masks fully automatically.
Everyday use cases
- Case 01
How you've probably met it
Click-to-select tools in photo and video editors, and labeling tools that pre-draw masks for human annotators.
Strengths, weaknesses and when to use it
Strengths
- Works on new kinds of objects without retraining (zero-shot).
- Fast interactive use once an image is encoded.
Weaknesses
- Returns masks, not names: it does not say what the object is.
When to use itFor interactive cut-outs and for speeding up dataset labeling.
Go deeper
Sources
Facts checked as of 2026-10-12