SAM 2

SAM 2 is a promptable image and video segmentation foundation model that Meta FAIR open-sourced in 2024-07 under Apache 2.0. Click a point, draw a box or hand it a mask, and it lifts the target out of any frame and tracks it frame by frame along the video. Positioning matters: this is a research model plus open weights plus a free demo, not a subscription online tool.

Core capabilities

• Unified images and video: an image is treated as a single-frame video, and image segmentation is more accurate than the original SAM and 6x faster
• Prompt-based refinement: points, boxes and masks are all valid inputs, and one extra hint on any frame corrects the tracking result
• Occlusion awareness: an occlusion head predicts whether the target is present in that frame, so tracking resumes once the target reappears
• Zero-shot generalization: unseen objects and scenes work without training; the paper evaluates on 17 zero-shot video datasets
• Four model sizes: 38.9M to 224.4M parameters, 91.2 to 39.5 FPS on an A100, so accuracy and speed are yours to choose
• Complete training assets: the SA-V dataset ships with it, at 51,000 videos, 643,000 spatio-temporal masks and 196 hours of footage

Where it fits Interactive pre-annotation for data-labeling teams, rotoscoping and object removal in video post-production, prototyping for robot perception and video analytics, and a foundation for vision research. For hard real-time line inspection use YOLO-seg; this model only answers “which target does this pixel belong to”, not “what is this”.

Why it stands out Its ecosystem is the real moat: native support in Hugging Face Transformers, plus integrations with Ultralytics, Roboflow, CVAT and SageMaker. The weights fall under Apache 2.0 as well, so closed-source commercial use is allowed; the successor SAM 3 has moved to a more restrictive custom license.

Editorial take Two years after release it is still the default starting point for video segmentation, with more than twenty thousand GitHub stars and over ten thousand monthly Hugging Face downloads, but the default branch has been largely frozen since 2024-12 and new capabilities have moved to SAM 3. There is no official API and no hosted inference, so you bring your own GPU. For convenience use the free demo (personal, non-commercial only, single video up to 70 MB); to build a product, download the weights and self-host.