HongChuang/ultralytics-ascend
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 3

ultralytics 8.0.238 Explorer Ask AI feature and fixes (#7408)

Browse source 
Co-authored-by: Kayzwer <68285002+Kayzwer@users.noreply.github.com>
Co-authored-by: uwer <uwe.rosebrock@gmail.com>
Co-authored-by: Uwe Rosebrock <ro260@csiro.au>
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Laughing-q <1182102784@qq.com>
Co-authored-by: Muhammad Rizwan Munawar <chr043416@gmail.com>
Co-authored-by: AdamP <adamp87hun@gmail.com>
This commit is contained in:
Glenn Jocher 2024-01-08 23:36:29 +01:00 • committed by GitHub
parent e76754eab0
commit 783033fa6b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
19 changed files with 387 additions and 76 deletions
Download patch file Download diff file Expand all files Collapse all files

70
docs/en/datasets/explorer/explorer.ipynb
View file

@ -109,7 +109,10 @@
"metadata": {},
"source": [
"You can use the also plot the similar samples directly using the `plot_similar` util\n",
"<img width=\"689\" alt=\"Screenshot 2024-01-06 at 9 46 48 PM\" src=\"https://github.com/AyushExel/assets/assets/15766192/70e1a4c4-6c67-4664-b77a-ad27b1fba8f8\">\n"
"<p>\n",
"\n",
" <img src=\"https://github.com/AyushExel/assets/assets/15766192/a3c9247b-9271-47df-aaa5-36d96c5034b1\" />\n",
"</p>\n"
]
},
{
@ -139,17 +142,74 @@
"metadata": {},
"source": [
"<p>\n",
"<img width=\"766\" alt=\"Screenshot 2024-01-06 at 10 05 10 PM\" src=\"https://github.com/AyushExel/assets/assets/15766192/faa9c544-d96b-4528-a2ea-95c5d8856744\">\n",
"<img src=\"https://github.com/AyushExel/assets/assets/15766192/8e011195-b0da-43ef-b3cd-5fb6f383037e\">\n",
"\n",
"</p>"
]
},
{
"cell_type": "markdown",
"id": "0cea63f1-71f1-46da-af2b-b1b7d8f73553",
"metadata": {},
"source": [
"## 2. Ask AI: Search or filter with Natural Language\n",
"You can prompt the Explorer object with the kind of data points you want to see and it'll try to return a dataframe with those. Because it is powered by LLMs, it doesn't always get it right. In that case, it'll return None.\n",
"<p>\n",
"<img width=\"1131\" alt=\"Screenshot 2024-01-07 at 2 34 53 PM\" src=\"https://github.com/AyushExel/assets/assets/15766192/c4a69fd9-e54f-4d6a-aba5-dc9cfae1bc04\">\n",
"\n",
"</p>\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "92fb92ac-7f76-465a-a9ba-ea7492498d9c",
"metadata": {},
"outputs": [],
"source": [
"df = exp.ask_ai(\"show me images containing more than 10 objects with at least 2 persons\")\n",
"df.head(5)"
]
},
{
"cell_type": "markdown",
"id": "f2a7d26e-0ce5-4578-ad1a-b1253805280f",
"metadata": {},
"source": [
"for plotting these results you can use `plot_query_result` util\n",
"Example:\n",
"```\n",
"plt = plot_query_result(exp.ask_ai(\"show me 10 images containing exactly 2 persons\"))\n",
"Image.fromarray(plt)\n",
"```\n",
"<p>\n",
" <img src=\"https://github.com/AyushExel/assets/assets/15766192/2cb780de-d05b-4412-a526-7f7f0f10e669\">\n",
"\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b1cfab84-9835-4da0-8e9a-42b30cf84511",
"metadata": {},
"outputs": [],
"source": [
"# plot\n",
"from ultralytics.data.explorer import plot_query_result\n",
"from PIL import Image\n",
"\n",
"plt = plot_query_result(exp.ask_ai(\"show me 10 images containing exactly 2 persons\"))\n",
"Image.fromarray(plt)"
]
},
{
"cell_type": "markdown",
"id": "35315ae6-d827-40e4-8813-279f97a83b34",
"metadata": {},
"source": [
"## 2. Run SQL queries on your Dataset!\n",
"## 3. Run SQL queries on your Dataset!\n",
"Sometimes you might want to investigate a certain type of entries in your dataset. For this Explorer allows you to execute SQL queries.\n",
"It accepts either of the formats:\n",
"- Queries beginning with \"WHERE\" will automatically select all columns. This can be thought of as a short-hand query\n",
@ -179,7 +239,7 @@
"metadata": {},
"source": [
"Just like similarity search, you also get a util to directly plot the sql queries using `exp.plot_sql_query`\n",
"<img width=\"771\" alt=\"Screenshot 2024-01-06 at 9 48 08 PM\" src=\"https://github.com/AyushExel/assets/assets/15766192/332f5acd-3a4e-462d-a281-5d5effd1886e\">\n"
"<img src=\"https://github.com/AyushExel/assets/assets/15766192/f8b66629-8dd0-419e-8f44-9837969ba678\">\n"
]
},
{
@ -419,7 +479,7 @@
"metadata": {},
"source": [
"You should see something like this\n",
"<img width=\"897\" alt=\"Screenshot 2024-01-06 at 9 50 48 PM\" src=\"https://github.com/AyushExel/assets/assets/15766192/5d3f0e35-2ad4-4a67-8df7-3a4c17867b72\">\n"
"<img src=\"https://github.com/AyushExel/assets/assets/15766192/649bc366-ca2d-46ea-bfd9-3097cf575584\">\n"
]
},
{