Docs updates: Add Explorer to tab, YOLOv5 in Guides and Usage in Quickstart (#7438)

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: Haixuan Xavier Tao <tao.xavier@outlook.com>
This commit is contained in:
Ayush Chaurasia 2024-01-10 04:20:26 +05:30 committed by GitHub
parent 53150a925b
commit a92adf8231
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
30 changed files with 227 additions and 105 deletions

View file

@ -6,7 +6,7 @@ keywords: Ultralytics, Explorer GUI, semantic search, vector similarity search,
# Explorer GUI
Explorer GUI is like a playground build using (Ultralytics Explorer API)[api.md]. It allows you to run semantic/vector similarity search, SQL queries and even search using natural language using our ask AI feature powered by LLMs.
Explorer GUI is like a playground build using [Ultralytics Explorer API](api.md). It allows you to run semantic/vector similarity search, SQL queries and even search using natural language using our ask AI feature powered by LLMs.
### Installation

View file

@ -6,7 +6,13 @@ keywords: Ultralytics Explorer, CV Dataset Tools, Semantic Search, SQL Dataset Q
# Ultralytics Explorer
Ultralytics Explorer is a tool for exploring CV datasets using semantic search, SQL queries and vector similarity search. It is also a Python API for accessing the same functionality.
<p>
<img width="1709" alt="Screenshot 2024-01-08 at 7 19 48PM (1)" src="https://github.com/AyushExel/assets/assets/15766192/e536b0eb-6bce-43fe-b800-3e79510d2e5b">
</p>
Ultralytics Explorer is a tool for exploring CV datasets using semantic search, SQL queries, vector similarity search and even using natural language. It is also a Python API for accessing the same functionality.
### Installation of optional dependencies
@ -33,8 +39,3 @@ yolo explorer
!!! note "Note"
Ask AI feature works using OpenAI, so you'll be prompted to set the api key for OpenAI when you first run the GUI.
You can set it like this - `yolo settings openai_api_key="..."`
Example
<p>
<img width="1709" alt="Screenshot 2024-01-08 at 7 19 48PM (1)" src="https://github.com/AyushExel/assets/assets/15766192/e536b0eb-6bce-43fe-b800-3e79510d2e5b">
</p>

View file

@ -10,7 +10,12 @@ Ultralytics provides support for various datasets to facilitate computer vision
## 🌟 New: Ultralytics Explorer 🌟
Create embeddings for your dataset, search for similar images, run SQL queries and perform semantic search. You can get started with our GUI app or build your own using the API. Learn more [here](explorer/index.md).
Create embeddings for your dataset, search for similar images, run SQL queries, perform semantic search and even search using natural language! You can get started with our GUI app or build your own using the API. Learn more [here](explorer/index.md).
<p>
<img width="1709" alt="Screenshot 2024-01-08 at 7 19 48PM (1)" src="https://github.com/AyushExel/assets/assets/15766192/e536b0eb-6bce-43fe-b800-3e79510d2e5b">
</p>
- Try the [GUI Demo](explorer/index.md)
- Learn more about the [Explorer API](explorer/index.md)

View file

@ -23,47 +23,47 @@ Object blurring with [Ultralytics YOLOv8](https://github.com/ultralytics/ultraly
from ultralytics import YOLO
from ultralytics.utils.plotting import Annotator, colors
import cv2
model = YOLO("yolov8n.pt")
names = model.names
cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))
# Blur ratio
blur_ratio = 50
# Video writer
video_writer = cv2.VideoWriter("object_blurring_output.avi",
cv2.VideoWriter_fourcc(*'mp4v'),
fps, (w, h))
while cap.isOpened():
success, im0 = cap.read()
if not success:
print("Video frame is empty or video processing has been successfully completed.")
break
results = model.predict(im0, show=False)
boxes = results[0].boxes.xyxy.cpu().tolist()
clss = results[0].boxes.cls.cpu().tolist()
annotator = Annotator(im0, line_width=2, example=names)
if boxes is not None:
for box, cls in zip(boxes, clss):
annotator.box_label(box, color=colors(int(cls), True), label=names[int(cls)])
obj = im0[int(box[1]):int(box[3]), int(box[0]):int(box[2])]
blur_obj = cv2.blur(obj, (blur_ratio, blur_ratio))
im0[int(box[1]):int(box[3]), int(box[0]):int(box[2])] = blur_obj
cv2.imshow("ultralytics", im0)
video_writer.write(im0)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
video_writer.release()
cv2.destroyAllWindows()

View file

@ -24,50 +24,50 @@ Object cropping with [Ultralytics YOLOv8](https://github.com/ultralytics/ultraly
from ultralytics.utils.plotting import Annotator, colors
import cv2
import os
model = YOLO("yolov8n.pt")
names = model.names
cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))
crop_dir_name = "ultralytics_crop"
if not os.path.exists(crop_dir_name):
os.mkdir(crop_dir_name)
# Video writer
video_writer = cv2.VideoWriter("object_cropping_output.avi",
cv2.VideoWriter_fourcc(*'mp4v'),
fps, (w, h))
idx = 0
while cap.isOpened():
success, im0 = cap.read()
if not success:
print("Video frame is empty or video processing has been successfully completed.")
break
results = model.predict(im0, show=False)
boxes = results[0].boxes.xyxy.cpu().tolist()
clss = results[0].boxes.cls.cpu().tolist()
annotator = Annotator(im0, line_width=2, example=names)
if boxes is not None:
for box, cls in zip(boxes, clss):
idx += 1
annotator.box_label(box, color=colors(int(cls), True), label=names[int(cls)])
crop_obj = im0[int(box[1]):int(box[3]), int(box[0]):int(box[2])]
cv2.imwrite(os.path.join(crop_dir_name, str(idx)+".png"), crop_obj)
cv2.imshow("ultralytics", im0)
video_writer.write(im0)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
video_writer.release()
cv2.destroyAllWindows()

View file

@ -39,17 +39,13 @@ Introducing [Ultralytics](https://ultralytics.com) [YOLOv8](https://github.com/u
Explore the YOLOv8 Docs, a comprehensive resource designed to help you understand and utilize its features and capabilities. Whether you are a seasoned machine learning practitioner or new to the field, this hub aims to maximize YOLOv8's potential in your projects
# 🌟 New: Ultralytics Explorer 🌟
Create embeddings for your dataset, search for similar images, run SQL queries and perform semantic search. You can get started with our GUI app or build your own using the API. Learn more [here](datasets/explorer/index.md).
## Where to Start
- **Install** `ultralytics` with pip and get up and running in minutes &nbsp; [:material-clock-fast: Get Started](quickstart.md){ .md-button }
- **Predict** new images and videos with YOLOv8 &nbsp; [:octicons-image-16: Predict on Images](modes/predict.md){ .md-button }
- **Train** a new YOLOv8 model on your own custom dataset &nbsp; [:fontawesome-solid-brain: Train a Model](modes/train.md){ .md-button }
- **Tasks** YOLOv8 tasks like segment, classify, pose and track &nbsp; [:material-magnify-expand: Explore Tasks](tasks/index.md){ .md-button }
- **Explore** datasets with advanced semantic and SQL search &nbsp; [:material-magnify-expand: Run Explorer](datasets/explorer/index.md){ .md-button }
- **NEW 🚀 Explore** datasets with advanced semantic and SQL search &nbsp; [:material-magnify-expand: Explore a Dataset](datasets/explorer/index.md){ .md-button }
<p align="center">
<br>

View file

@ -43,6 +43,10 @@ keywords: Ultralytics, YOLO, Configuration, cfg2dict, handle_deprecation, merge_
<br><br>
## ::: ultralytics.cfg.handle_explorer
<br><br>
## ::: ultralytics.cfg.parse_key_value_pair
<br><br>

View file

@ -7,7 +7,7 @@ keywords: Ultralytics, YOLO, distance calculation, object tracking, data visuali
!!! Note
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/distance_calculation.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/distance_calculation.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/solutions/heatmap.py) 🛠️. Thank you 🙏!
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/distance_calculation.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/distance_calculation.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/solutions/distance_calculation.py) 🛠️. Thank you 🙏!
<br><br>

View file

@ -7,7 +7,7 @@ keywords: Ultralytics YOLO, speed estimation software, real-time vehicle trackin
!!! Note
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/speed_estimation.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/speed_estimation.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/solutions/object_counter.py) 🛠️. Thank you 🙏!
This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/speed_estimation.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/speed_estimation.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/solutions/speed_estimation.py) 🛠️. Thank you 🙏!
<br><br>

View file

@ -240,6 +240,53 @@ Benchmark mode is used to profile the speed and accuracy of various export forma
[Benchmark Examples](../modes/benchmark.md){ .md-button }
## Explorer
Explorer API can be used to explore datasets with advanced semantic, vector-similarity and SQL search among other features. It also searching for images based on their content using natural language by utilizing the power of LLMs. The Explorer API allows you to write your own dataset exploration notebooks or scripts to get insights into your datasets.
!!! Example "Semantic Search Using Explorer"
=== "Using Images"
```python
from ultralytics import Explorer
# create an Explorer object
exp = Explorer(data='coco128.yaml', model='yolov8n.pt')
exp.create_embeddings_table()
similar = exp.get_similar(img='https://ultralytics.com/images/bus.jpg', limit=10)
print(similar.head())
# Search using multiple indices
similar = exp.get_similar(
img=['https://ultralytics.com/images/bus.jpg',
'https://ultralytics.com/images/bus.jpg'],
limit=10
)
print(similar.head())
```
=== "Using Dataset Indices"
```python
from ultralytics import Explorer
# create an Explorer object
exp = Explorer(data='coco128.yaml', model='yolov8n.pt')
exp.create_embeddings_table()
similar = exp.get_similar(idx=1, limit=10)
print(similar.head())
# Search using multiple indices
similar = exp.get_similar(idx=[1,10], limit=10)
print(similar.head())
```
[Explorer](../datasets/explorer/index.md){ .md-button }
## Using Trainers
`YOLO` model class is a high-level wrapper on the Trainer classes. Each YOLO task has its own trainer that inherits from `BaseTrainer`.

View file

@ -170,12 +170,14 @@ nav:
- Classify: tasks/classify.md
- Pose: tasks/pose.md
- OBB: tasks/obb.md
- Guides:
- guides/index.md
- Models:
- models/index.md
- Datasets:
- datasets/index.md
- Guides:
- guides/index.md
- NEW 🚀 Explorer:
- datasets/explorer/index.md
- Languages:
- 🇬🇧&nbsp English: https://docs.ultralytics.com/
- 🇨🇳&nbsp 简体中文: https://docs.ultralytics.com/zh/
@ -188,7 +190,14 @@ nav:
- 🇵🇹&nbsp Português: https://docs.ultralytics.com/pt/
- 🇮🇳&nbsp हिन्दी: https://docs.ultralytics.com/hi/
- 🇸🇦&nbsp العربية: https://docs.ultralytics.com/ar/
- Quickstart: quickstart.md
- Quickstart:
- quickstart.md
- Usage:
- CLI: usage/cli.md
- Python: usage/python.md
- Callbacks: usage/callbacks.md
- Configuration: usage/cfg.md
- Advanced Customization: usage/engine.md
- Modes:
- modes/index.md
- Train: modes/train.md
@ -219,7 +228,7 @@ nav:
- RT-DETR (Realtime Detection Transformer): models/rtdetr.md
- Datasets:
- datasets/index.md
- Explorer:
- NEW 🚀 Explorer:
- datasets/explorer/index.md
- Explorer API: datasets/explorer/api.md
- Explorer Dashboard: datasets/explorer/dashboard.md
@ -263,6 +272,11 @@ nav:
- DOTA8: datasets/obb/dota8.md
- Multi-Object Tracking:
- datasets/track/index.md
- NEW 🚀 Explorer:
- datasets/explorer/index.md
- Explorer API: datasets/explorer/api.md
- Explorer Dashboard Demo: datasets/explorer/dashboard.md
- VOC Exploration Example: datasets/explorer/explorer.ipynb
- Guides:
- guides/index.md
- YOLO Common Issues: guides/yolo-common-issues.md
@ -290,6 +304,31 @@ nav:
- VisionEye Mapping: guides/vision-eye.md
- Speed Estimation: guides/speed-estimation.md
- Distance Calculation: guides/distance-calculation.md
- YOLOv5:
- yolov5/index.md
- Quickstart: yolov5/quickstart_tutorial.md
- Environments:
- Amazon Web Services (AWS): yolov5/environments/aws_quickstart_tutorial.md
- Google Cloud (GCP): yolov5/environments/google_cloud_quickstart_tutorial.md
- AzureML: yolov5/environments/azureml_quickstart_tutorial.md
- Docker Image: yolov5/environments/docker_image_quickstart_tutorial.md
- Tutorials:
- Train Custom Data: yolov5/tutorials/train_custom_data.md
- Tips for Best Training Results: yolov5/tutorials/tips_for_best_training_results.md
- Multi-GPU Training: yolov5/tutorials/multi_gpu_training.md
- PyTorch Hub: yolov5/tutorials/pytorch_hub_model_loading.md
- TFLite, ONNX, CoreML, TensorRT Export: yolov5/tutorials/model_export.md
- NVIDIA Jetson Nano Deployment: yolov5/tutorials/running_on_jetson_nano.md
- Test-Time Augmentation (TTA): yolov5/tutorials/test_time_augmentation.md
- Model Ensembling: yolov5/tutorials/model_ensembling.md
- Pruning/Sparsity Tutorial: yolov5/tutorials/model_pruning_and_sparsity.md
- Hyperparameter evolution: yolov5/tutorials/hyperparameter_evolution.md
- Transfer learning with frozen layers: yolov5/tutorials/transfer_learning_with_frozen_layers.md
- Architecture Summary: yolov5/tutorials/architecture_description.md
- Roboflow Datasets: yolov5/tutorials/roboflow_datasets_integration.md
- Neural Magic's DeepSparse: yolov5/tutorials/neural_magic_pruning_quantization.md
- Comet Logging: yolov5/tutorials/comet_logging_integration.md
- Clearml Logging: yolov5/tutorials/clearml_logging_integration.md
- Integrations:
- integrations/index.md
- Comet ML: integrations/comet.md
@ -303,37 +342,6 @@ nav:
- Neural Magic: integrations/neural-magic.md
- TensorBoard: integrations/tensorboard.md
- Amazon SageMaker: integrations/amazon-sagemaker.md
- Usage:
- CLI: usage/cli.md
- Python: usage/python.md
- Callbacks: usage/callbacks.md
- Configuration: usage/cfg.md
- Advanced Customization: usage/engine.md
- YOLOv5:
- yolov5/index.md
- Quickstart: yolov5/quickstart_tutorial.md
- Environments:
- Amazon Web Services (AWS): yolov5/environments/aws_quickstart_tutorial.md
- Google Cloud (GCP): yolov5/environments/google_cloud_quickstart_tutorial.md
- AzureML: yolov5/environments/azureml_quickstart_tutorial.md
- Docker Image: yolov5/environments/docker_image_quickstart_tutorial.md
- Tutorials:
- Train Custom Data: yolov5/tutorials/train_custom_data.md
- Tips for Best Training Results: yolov5/tutorials/tips_for_best_training_results.md
- Multi-GPU Training: yolov5/tutorials/multi_gpu_training.md
- PyTorch Hub: yolov5/tutorials/pytorch_hub_model_loading.md
- TFLite, ONNX, CoreML, TensorRT Export: yolov5/tutorials/model_export.md
- NVIDIA Jetson Nano Deployment: yolov5/tutorials/running_on_jetson_nano.md
- Test-Time Augmentation (TTA): yolov5/tutorials/test_time_augmentation.md
- Model Ensembling: yolov5/tutorials/model_ensembling.md
- Pruning/Sparsity Tutorial: yolov5/tutorials/model_pruning_and_sparsity.md
- Hyperparameter evolution: yolov5/tutorials/hyperparameter_evolution.md
- Transfer learning with frozen layers: yolov5/tutorials/transfer_learning_with_frozen_layers.md
- Architecture Summary: yolov5/tutorials/architecture_description.md
- Roboflow Datasets: yolov5/tutorials/roboflow_datasets_integration.md
- Neural Magic's DeepSparse: yolov5/tutorials/neural_magic_pruning_quantization.md
- Comet Logging: yolov5/tutorials/comet_logging_integration.md
- Clearml Logging: yolov5/tutorials/clearml_logging_integration.md
- HUB:
- hub/index.md
- Quickstart: hub/quickstart.md
@ -357,6 +365,11 @@ nav:
- build: reference/data/build.md
- converter: reference/data/converter.md
- dataset: reference/data/dataset.md
- explorer:
- explorer: reference/data/explorer/explorer.md
- gui:
- dash: reference/data/explorer/gui/dash.md
- utils: reference/data/explorer/utils.md
- loaders: reference/data/loaders.md
- split_dota: reference/data/split_dota.md
- utils: reference/data/utils.md
@ -436,10 +449,10 @@ nav:
- tasks: reference/nn/tasks.md
- solutions:
- ai_gym: reference/solutions/ai_gym.md
- distance_calculation: reference/solutions/distance_calculation.md
- heatmap: reference/solutions/heatmap.md
- object_counter: reference/solutions/object_counter.md
- speed_estimation: reference/solutions/speed_estimation.md
- distance_calculation: reference/solutions/distance_calculation.md
- trackers:
- basetrack: reference/trackers/basetrack.md
- bot_sort: reference/trackers/bot_sort.md