ultralytics 8.2.70 Segment Anything Model 2 (SAM 2) (#14813)
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
This commit is contained in:
Laughing
GitHub
parent
80f699ae21
commit
8648572809
36 changed files with 3276 additions and 77 deletions
|
|
@ -113,6 +113,8 @@ The following table details the available SAM 2 models, their pre-trained weight
|
|||
|
||||
| Model Type | Pre-trained Weights | Tasks Supported | Inference | Validation | Training | Export |
|
||||
| ----------- | ------------------------------------------------------------------------------------- | -------------------------------------------- | --------- | ---------- | -------- | ------ |
|
||||
| SAM 2 tiny | [sam2_t.pt](https://github.com/ultralytics/assets/releases/download/v8.2.0/sam2_t.pt) | [Instance Segmentation](../tasks/segment.md) | ✅ | ❌ | ❌ | ❌ |
|
||||
| SAM 2 small | [sam2_s.pt](https://github.com/ultralytics/assets/releases/download/v8.2.0/sam2_s.pt) | [Instance Segmentation](../tasks/segment.md) | ✅ | ❌ | ❌ | ❌ |
|
||||
| SAM 2 base | [sam2_b.pt](https://github.com/ultralytics/assets/releases/download/v8.2.0/sam2_b.pt) | [Instance Segmentation](../tasks/segment.md) | ✅ | ❌ | ❌ | ❌ |
|
||||
| SAM 2 large | [sam2_l.pt](https://github.com/ultralytics/assets/releases/download/v8.2.0/sam2_l.pt) | [Instance Segmentation](../tasks/segment.md) | ✅ | ❌ | ❌ | ❌ |
|
||||
|
||||
|
|
|
|||
|
|
@ -13,8 +13,4 @@ keywords: Ultralytics, MaskDecoder, MLP, machine learning, transformer architect
|
|||
|
||||
## ::: ultralytics.models.sam.modules.decoders.MaskDecoder
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam.modules.decoders.MLP
|
||||
|
||||
<br><br>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
description: Discover the Ultralytics Sam module for object segmentation. Learn about its components, such as image encoders and mask decoders, in this comprehensive guide.
|
||||
keywords: Ultralytics, Sam Module, object segmentation, image encoder, mask decoder, prompt encoder, AI, machine learning
|
||||
description: Discover the Ultralytics SAM module for object segmentation. Learn about its components, such as image encoders and mask decoders, in this comprehensive guide.
|
||||
keywords: Ultralytics, SAM Module, object segmentation, image encoder, mask decoder, prompt encoder, AI, machine learning
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam/modules/sam.py`
|
||||
|
|
@ -11,6 +11,6 @@ keywords: Ultralytics, Sam Module, object segmentation, image encoder, mask deco
|
|||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam.modules.sam.Sam
|
||||
## ::: ultralytics.models.sam.modules.sam.SAMModel
|
||||
|
||||
<br><br>
|
||||
|
|
|
|||
36
docs/en/reference/models/sam2/build.md
Normal file
36
docs/en/reference/models/sam2/build.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
description: Discover detailed instructions for building various Segment Anything Model 2 (SAM 2) architectures with Ultralytics.
|
||||
keywords: Ultralytics, SAM 2 model, Segment Anything Model 2, SAM, model building, deep learning, AI
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/build.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/build.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/build.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/build.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.build.build_sam2_t
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.build.build_sam2_s
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.build.build_sam2_b
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.build.build_sam2_l
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.build._build_sam2
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.build.build_sam2
|
||||
|
||||
<br><br>
|
||||
16
docs/en/reference/models/sam2/model.md
Normal file
16
docs/en/reference/models/sam2/model.md
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
---
|
||||
description: Explore the SAM 2 (Segment Anything Model 2) interface for real-time image segmentation. Learn about promptable segmentation and zero-shot capabilities.
|
||||
keywords: Ultralytics, SAM 2, Segment Anything Model 2, image segmentation, real-time segmentation, zero-shot performance, promptable segmentation, SA-1B dataset
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/model.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/model.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/model.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/model.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.model.SAM2
|
||||
|
||||
<br><br>
|
||||
16
docs/en/reference/models/sam2/modules/decoders.md
Normal file
16
docs/en/reference/models/sam2/modules/decoders.md
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
---
|
||||
description: Explore the MaskDecoder and MLP modules in Ultralytics for efficient mask prediction using transformer architecture. Detailed attributes, functionalities, and implementation.
|
||||
keywords: Ultralytics, MaskDecoder, MLP, machine learning, transformer architecture, mask prediction, neural networks, PyTorch modules
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/modules/decoders.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/decoders.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/decoders.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/modules/decoders.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.decoders.MaskDecoder
|
||||
|
||||
<br><br>
|
||||
28
docs/en/reference/models/sam2/modules/encoders.md
Normal file
28
docs/en/reference/models/sam2/modules/encoders.md
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
---
|
||||
description: Discover the Ultralytics SAM 2 module for object segmentation. Learn about its components, such as image encoders and mask decoders, in this comprehensive guide.
|
||||
keywords: Ultralytics, SAM 2 Module, object segmentation, image encoder, mask decoder, prompt encoder, AI, machine learning
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/modules/encoders.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/encoders.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/encoders.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/modules/encoders.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.encoders.MemoryEncoder
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.encoders.ImageEncoder
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.encoders.FpnNeck
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.encoders.Hiera
|
||||
|
||||
<br><br>
|
||||
20
docs/en/reference/models/sam2/modules/memory_attention.md
Normal file
20
docs/en/reference/models/sam2/modules/memory_attention.md
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
---
|
||||
description: Explore detailed documentation of various SAM 2 encoder modules such as MemoryAttentionLayer, MemoryAttention, available in Ultralytics' repository.
|
||||
keywords: Ultralytics, SAM 2 encoder, MemoryAttentionLayer, MemoryAttention
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/modules/memory_attention.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/memory_attention.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/memory_attention.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/modules/memory_attention.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.memory_attention.MemoryAttentionLayer
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.memory_attention.MemoryAttention
|
||||
|
||||
<br><br>
|
||||
16
docs/en/reference/models/sam2/modules/sam2.md
Normal file
16
docs/en/reference/models/sam2/modules/sam2.md
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
---
|
||||
description: Discover the Ultralytics SAM 2 module for object segmentation. Learn about its components, such as image encoders and mask decoders, in this comprehensive guide.
|
||||
keywords: Ultralytics, SAM 2 Module, object segmentation, image encoder, mask decoder, prompt encoder, AI, machine learning
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/modules/sam2.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/sam2.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/sam2.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/modules/sam2.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2.SAM2Model
|
||||
|
||||
<br><br>
|
||||
56
docs/en/reference/models/sam2/modules/sam2_blocks.md
Normal file
56
docs/en/reference/models/sam2/modules/sam2_blocks.md
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
---
|
||||
description: Explore detailed documentation of various SAM 2 modules such as MaskDownSampler, CXBlock, and more, available in Ultralytics' repository.
|
||||
keywords: Ultralytics, SAM 2 encoder, DropPath, MaskDownSampler, CXBlock, Fuser, TwoWayTransformer, TwoWayAttentionBlock, RoPEAttention, MultiScaleAttention, MultiScaleBlock. PositionEmbeddingSine, do_pool
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/modules/sam2_blocks.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/sam2_blocks.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/sam2_blocks.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/modules/sam2_blocks.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.DropPath
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.MaskDownSampler
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.CXBlock
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.Fuser
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.TwoWayAttentionBlock
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.TwoWayTransformer
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.RoPEAttention
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.MultiScaleAttention
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.MultiScaleBlock
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.PositionEmbeddingSine
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.sam2_blocks.do_pool
|
||||
|
||||
<br><br>
|
||||
44
docs/en/reference/models/sam2/modules/utils.md
Normal file
44
docs/en/reference/models/sam2/modules/utils.md
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
---
|
||||
description: Explore the detailed API reference for Ultralytics SAM 2 models.
|
||||
keywords: Ultralytics, SAM 2, API Reference, models, window partition, data processing, YOLO
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/modules/utils.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/utils.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/modules/utils.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/modules/utils.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.select_closest_cond_frames
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.get_1d_sine_pe
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.init_t_xy
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.compute_axial_cis
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.reshape_for_broadcast
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.apply_rotary_enc
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.window_partition
|
||||
|
||||
<br><br><hr><br>
|
||||
|
||||
## ::: ultralytics.models.sam2.modules.utils.window_unpartition
|
||||
|
||||
<br><br>
|
||||
16
docs/en/reference/models/sam2/predict.md
Normal file
16
docs/en/reference/models/sam2/predict.md
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
---
|
||||
description: Explore Ultralytics SAM 2 Predictor for advanced, real-time image segmentation using the Segment Anything Model 2 (SAM 2). Complete implementation details and auxiliary utilities.
|
||||
keywords: Ultralytics, SAM 2, Segment Anything Model 2, image segmentation, real-time, prediction, AI, machine learning, Python, torch, inference
|
||||
---
|
||||
|
||||
# Reference for `ultralytics/models/sam2/predict.py`
|
||||
|
||||
!!! Note
|
||||
|
||||
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/predict.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/sam2/predict.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/models/sam2/predict.py) 🛠️. Thank you 🙏!
|
||||
|
||||
<br>
|
||||
|
||||
## ::: ultralytics.models.sam2.predict.SAM2Predictor
|
||||
|
||||
<br><br>
|
||||
Loading…
Add table
Add a link
Reference in a new issue