SAM 3: segmenting by concept, not by click

Both SAM and SAM 2 answer one prompt at a time: click an object, get its mask. That’s efficient for one object, tedious for counting or segmenting every matching instance in a crowded scene — a click per phage, a click per particle. SAM 3 changes the prompt itself: instead of pointing at a single instance, you give it a concept — a short phrase or an example image — and it segments every matching instance across the whole image at once.

From “this one” to “all of these”

A point or box prompt in SAM/SAM 2 is inherently single-instance — the model has no way to know you mean “this object” versus “every object like it” unless you click each one. SAM 3’s concept prompts (a text phrase like “phage particle,” or an example crop) sidestep that entirely: the model interprets the prompt as a category description and returns masks for every instance matching it in the image, not just the region nearest the prompt. That’s a genuinely different task, closer to open-vocabulary detection than to interactive point-based segmentation.

SAM 3D: the other half of the release

Alongside concept segmentation, SAM 3 added SAM 3D, single-image 3D reconstruction — inferring a 3D shape from one 2D image rather than requiring multiple views or a depth sensor. It’s a separate capability bundled into the same release rather than a component of concept segmentation itself, but it points at the same underlying direction: fewer, more natural prompts producing richer structured output.

Why concept-level prompting matches how a microscopist actually works

“Segment every phage in this field” is a fundamentally more natural request than “click every phage” — and at real experimental densities, clicking every instance isn’t practical at all. Concept-level, multi-instance prompting is a closer match to how quantification tasks in microscopy are actually phrased, even though (as with the original SAM) performance on scientific imaging modalities the model wasn’t trained on still needs direct validation before being trusted for a real measurement.

Takeaways

• The prompt changed from “point at one object” to “describe a concept,” and the output changed correspondingly from one mask to every matching instance in the image.
• This is closer to open-vocabulary detection than to the original SAM’s interactive, single-instance segmentation — a different task, not just a bigger model.
• SAM 3D (single-image 3D reconstruction) shipped in the same release but is a separate capability, not a component of concept segmentation.
• As with SAM and µSAM, out-of-domain performance on scientific imaging still needs to be validated directly rather than assumed from natural-image results.

Source

Meta AI, Segment Anything Model 3: segment anything with concepts, 19 November 2025 — announcement.

Continue through the concepts

Related notes

SAM 2: keeping a mask's identity across a video

How a streaming-memory architecture let promptable segmentation extend from single images to video, tracking an object's identity through occlusion and motion in real time.

µSAM: closing the gap between SAM and real microscopy images

SAM was trained on natural photographs and performs noticeably worse on fluorescence and electron microscopy out of the box. µSAM fine-tunes it for exactly those modalities and ships it as a usable napari tool.

Segment Anything (SAM): segmentation you can prompt instead of retrain

How SAM turned segmentation into a promptable task, why its billion-mask training set exists, and why that idea mattered enough to spawn a whole branch of microscopy-specific models.

AlexNet: the result that ended the debate about deep learning

How a convolutional network trained on two GPUs cut ImageNet error by a margin large enough to reset computer vision, and why every microscopy segmentation model on this site descends from it.