Deployment - Improved Small Object Detection
Contents
Deployment - Improved Small Object Detection#
! git clone https://github.com/ultralytics/yolov5.git
Cloning into 'yolov5'...
remote: Enumerating objects: 14717, done.
remote: Counting objects: 100% (40/40), done.
remote: Compressing objects: 100% (33/33), done.
remote: Total 14717 (delta 19), reused 18 (delta 7), pack-reused 14677
Receiving objects: 100% (14717/14717), 13.54 MiB | 17.98 MiB/s, done.
Resolving deltas: 100% (10161/10161), done.
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
%cd yolov5
%pip install -qr requirements.txt
!pip install -U sahi yolov5
/content/yolov5
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Requirement already satisfied: sahi in /usr/local/lib/python3.7/dist-packages (0.11.1)
Requirement already satisfied: yolov5 in /usr/local/lib/python3.7/dist-packages (6.2.3)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from sahi) (6.0)
Requirement already satisfied: pybboxes==0.1.5 in /usr/local/lib/python3.7/dist-packages (from sahi) (0.1.5)
Requirement already satisfied: pillow>=8.2.0 in /usr/local/lib/python3.7/dist-packages (from sahi) (9.3.0)
Requirement already satisfied: click==8.0.4 in /usr/local/lib/python3.7/dist-packages (from sahi) (8.0.4)
Requirement already satisfied: shapely>=1.8.0 in /usr/local/lib/python3.7/dist-packages (from sahi) (1.8.5.post1)
Requirement already satisfied: opencv-python>=4.2.0.32 in /usr/local/lib/python3.7/dist-packages (from sahi) (4.6.0.66)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from sahi) (2.23.0)
Requirement already satisfied: terminaltables in /usr/local/lib/python3.7/dist-packages (from sahi) (3.1.10)
Requirement already satisfied: tqdm>=4.48.2 in /usr/local/lib/python3.7/dist-packages (from sahi) (4.64.1)
Requirement already satisfied: fire in /usr/local/lib/python3.7/dist-packages (from sahi) (0.4.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from click==8.0.4->sahi) (4.13.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from pybboxes==0.1.5->sahi) (1.21.6)
Requirement already satisfied: torch>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from yolov5) (1.12.1+cu113)
Requirement already satisfied: pandas>=1.1.4 in /usr/local/lib/python3.7/dist-packages (from yolov5) (1.3.5)
Requirement already satisfied: seaborn>=0.11.0 in /usr/local/lib/python3.7/dist-packages (from yolov5) (0.11.2)
Requirement already satisfied: torchvision>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from yolov5) (0.13.1+cu113)
Requirement already satisfied: tensorboard>=2.4.1 in /usr/local/lib/python3.7/dist-packages (from yolov5) (2.9.1)
Requirement already satisfied: psutil in /usr/local/lib/python3.7/dist-packages (from yolov5) (5.4.8)
Requirement already satisfied: boto3>=1.19.1 in /usr/local/lib/python3.7/dist-packages (from yolov5) (1.26.5)
Requirement already satisfied: ipython in /usr/local/lib/python3.7/dist-packages (from yolov5) (7.9.0)
Requirement already satisfied: scipy>=1.4.1 in /usr/local/lib/python3.7/dist-packages (from yolov5) (1.7.3)
Requirement already satisfied: matplotlib>=3.2.2 in /usr/local/lib/python3.7/dist-packages (from yolov5) (3.2.2)
Requirement already satisfied: thop>=0.1.1 in /usr/local/lib/python3.7/dist-packages (from yolov5) (0.1.1.post2209072238)
Requirement already satisfied: botocore<1.30.0,>=1.29.5 in /usr/local/lib/python3.7/dist-packages (from boto3>=1.19.1->yolov5) (1.29.5)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from boto3>=1.19.1->yolov5) (1.0.1)
Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from boto3>=1.19.1->yolov5) (0.6.0)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/local/lib/python3.7/dist-packages (from botocore<1.30.0,>=1.29.5->boto3>=1.19.1->yolov5) (1.25.11)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/dist-packages (from botocore<1.30.0,>=1.29.5->boto3>=1.19.1->yolov5) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.2.2->yolov5) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.2.2->yolov5) (1.4.4)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.2.2->yolov5) (3.0.9)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib>=3.2.2->yolov5) (4.1.1)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.1.4->yolov5) (2022.6)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.30.0,>=1.29.5->boto3>=1.19.1->yolov5) (1.15.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->sahi) (2022.9.24)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->sahi) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->sahi) (3.0.4)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (1.50.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (0.38.1)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (1.3.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (0.4.6)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (1.35.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (0.6.1)
Requirement already satisfied: protobuf<3.20,>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (3.17.3)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (3.4.1)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (1.0.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (1.8.1)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard>=2.4.1->yolov5) (57.4.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard>=2.4.1->yolov5) (4.2.4)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard>=2.4.1->yolov5) (4.9)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard>=2.4.1->yolov5) (0.2.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.4.1->yolov5) (1.3.1)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->click==8.0.4->sahi) (3.10.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard>=2.4.1->yolov5) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.4.1->yolov5) (3.2.2)
Requirement already satisfied: termcolor in /usr/local/lib/python3.7/dist-packages (from fire->sahi) (2.1.0)
Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (5.1.1)
Requirement already satisfied: backcall in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (0.2.0)
Requirement already satisfied: pexpect in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (4.8.0)
Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (2.0.10)
Requirement already satisfied: pygments in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (2.6.1)
Requirement already satisfied: jedi>=0.10 in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (0.18.1)
Requirement already satisfied: pickleshare in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (0.7.5)
Requirement already satisfied: decorator in /usr/local/lib/python3.7/dist-packages (from ipython->yolov5) (4.4.2)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /usr/local/lib/python3.7/dist-packages (from jedi>=0.10->ipython->yolov5) (0.8.3)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython->yolov5) (0.2.5)
Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.7/dist-packages (from pexpect->ipython->yolov5) (0.7.0)
import sys
import cv2
from PIL import Image as Img
from IPython.display import Image, display, HTML
from sahi.utils.yolov5 import (
download_yolov5s6_model,
)
# import required functions, classes
from sahi import AutoDetectionModel
from sahi.utils.cv import read_image
from sahi.utils.file import download_from_url
from sahi.predict import get_prediction, get_sliced_prediction, predict
import shutil
import numpy as np
import pandas as pd
import plotly.express as px
from plotly.offline import init_notebook_mode
def configure_plotly_browser_state():
import IPython
display(IPython.core.display.HTML('''
<script src="/static/components/requirejs/require.js"></script>
<script>
requirejs.config({
paths: {
base: '/static/base',
plotly: 'https://cdn.plot.ly/plotly-latest.min.js?noext',
},
});
</script>
'''))
Slicing Aided Hyper Inference#
Slicing Aided Hyper Inference (SAHI) ist eine Python-Bibliothek die das Problem der Small Objects Detection auf großen Bildern zu lösen versucht. Dies erfolgt über eine Sliced Prediction, also eine Prediction auf Tiles. Dabei werden jedoch doppelte Zählunge von erkannten Objekten vermieden. [Akyon et al., 2021]
Nachfolgend werden die Gewichte des besten Modells geladen und auf die Bilder aus dem Proof of Concept angewendet. Der Barplot zeigt, dass deutlich mehr Fluggerät erkannt wurde - auch für die anderen Flugplätze.
SAHI bietet unterschiedliche Methoden für den Zugriff auf die Ergebnisse. Mit to_coco_predictions(image_id) können Listen mit Dictionary je Detektion abgerufen werden. Dies ermöglicht das Erstellen des Barplots aus dem Proof of Concept.
yolov5_model_path = '/content/drive/MyDrive/dataset/colab_exports/Zeile31_XLmod_Background_Tiled_TrainingData2/weights/best.pt'
_images = list()
_images.append("/content/drive/MyDrive/dataset/tests/MCAS_Iwakuni_1_100m.png")
_images.append("/content/drive/MyDrive/dataset/tests/MCAS_Iwakuni_2_100m.png")
_images.append("/content/drive/MyDrive/dataset/tests/RUS_Severomorsk.jpg")
_images.append("/content/drive/MyDrive/dataset/tests/DEU_Ansbach_Katterbach.jpg")
detection_model = AutoDetectionModel.from_pretrained(
model_type='yolov5',
model_path=yolov5_model_path,
confidence_threshold=0.4,
#device="cpu",
device="cuda:0"
)
result_list = list()
for image in _images:
results = get_sliced_prediction(
image,
detection_model,
slice_height = 256,
slice_width = 256,
overlap_height_ratio = 0.2,
overlap_width_ratio = 0.2
)
loc = image.split("/")[-1].split(".")[0]
result_list_cache = results.to_coco_annotations()
results.export_visuals(export_dir=f'/content/drive/MyDrive/dataset/tests/sahi_exp/{loc}')
for i in result_list_cache:
i['image_id'] = loc
result_list.extend(result_list_cache)
Performing prediction on 50 number of slices.
Performing prediction on 50 number of slices.
Performing prediction on 414 number of slices.
Performing prediction on 120 number of slices.
df = pd.DataFrame.from_dict(result_list)
df.tail()
| image_id | bbox | score | category_id | category_name | segmentation | iscrowd | area | |
|---|---|---|---|---|---|---|---|---|
| 70 | DEU_Ansbach_Katterbach | [504.81485748291016, 1118.430519104004, 46.627... | 0.781747 | 1 | helicopter | [] | 0 | 1504 |
| 71 | DEU_Ansbach_Katterbach | [178.6126708984375, 1426.4920501708984, 53.600... | 0.776490 | 1 | helicopter | [] | 0 | 4044 |
| 72 | DEU_Ansbach_Katterbach | [255.3604393005371, 1440.1481018066406, 56.805... | 0.575088 | 1 | helicopter | [] | 0 | 2839 |
| 73 | DEU_Ansbach_Katterbach | [257.0429878234863, 1410.9247741699219, 53.042... | 0.514110 | 1 | helicopter | [] | 0 | 2641 |
| 74 | DEU_Ansbach_Katterbach | [205.39718171954155, 1427.8233489990234, 21.20... | 0.449921 | 1 | helicopter | [] | 0 | 957 |
# Zusätzlicher Screenshot für Iwakuni
df['location'] = np.where(df['image_id'].str.contains('100m'),'MCAS_Iwakuni',df['image_id'])
df['location'] = df['location'].str.replace('_',' ')
configure_plotly_browser_state()
init_notebook_mode(False) # To show plotly plots when notebook is exported to html
fig = px.bar(
df.groupby(['location','category_name']).count().reset_index(),
x='location',
y='category_id',
color='category_name',
barmode='group'
)
fig.update_layout(
xaxis_title="Standorte",
yaxis_title="Anzahl",
legend_title="Kategorie",
title='Anzahl an Fluggerät nach Standort und Kategorie'
)
newnames = {'plane':'Flugzeuge', 'helicopter': 'Helikopter'}
fig.for_each_trace(lambda t: t.update(name = newnames[t.name],
legendgroup = newnames[t.name],
hovertemplate = t.hovertemplate.replace(t.name, newnames[t.name])
)
)
fig.show()
Fig. 43 Eigene Darstellung: Barplot Anzahl Detektionen mit SAHI#
Image("/content/drive/MyDrive/dataset/tests/sahi_exp/MCAS_Iwakuni_1_100m/prediction_visual.png")
Kartendaten Copyright 2022: Google
Bilder Copyright 2022: CNES / Airbus, Maxer Technologies, Planet.com
Fig. 44 Output zu MCAS Iwakuni Bild 1 (100m)#
df[df['image_id']=='MCAS_Iwakuni_1_100m']
| image_id | bbox | score | category_id | category_name | segmentation | iscrowd | area | location | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | MCAS_Iwakuni_1_100m | [784.9304809570312, 352.1981506347656, 64.6701... | 0.899122 | 0 | plane | [] | 0 | 4819 | MCAS Iwakuni |
| 1 | MCAS_Iwakuni_1_100m | [1821.3680515289307, 891.7122192382812, 66.797... | 0.897323 | 0 | plane | [] | 0 | 4421 | MCAS Iwakuni |
| 2 | MCAS_Iwakuni_1_100m | [736.2460327148438, 645.2518920898438, 63.1655... | 0.884831 | 0 | plane | [] | 0 | 4391 | MCAS Iwakuni |
| 3 | MCAS_Iwakuni_1_100m | [596.0985717773438, 419.3920593261719, 67.4025... | 0.884480 | 0 | plane | [] | 0 | 5336 | MCAS Iwakuni |
| 4 | MCAS_Iwakuni_1_100m | [755.6351928710938, 548.8868408203125, 61.5064... | 0.878369 | 0 | plane | [] | 0 | 4277 | MCAS Iwakuni |
| 5 | MCAS_Iwakuni_1_100m | [577.803466796875, 517.09912109375, 66.5289916... | 0.872563 | 0 | plane | [] | 0 | 5247 | MCAS Iwakuni |
| 6 | MCAS_Iwakuni_1_100m | [724.25537109375, 740.0423583984375, 61.101623... | 0.871069 | 0 | plane | [] | 0 | 4143 | MCAS Iwakuni |
| 7 | MCAS_Iwakuni_1_100m | [764.1305541992188, 451.3215637207031, 66.4436... | 0.864177 | 0 | plane | [] | 0 | 4711 | MCAS Iwakuni |
| 8 | MCAS_Iwakuni_1_100m | [1053.2021484375, 296.310791015625, 30.7189941... | 0.774200 | 0 | plane | [] | 0 | 865 | MCAS Iwakuni |
| 9 | MCAS_Iwakuni_1_100m | [1920.8675537109375, 337.9620361328125, 73.077... | 0.764337 | 0 | plane | [] | 0 | 5003 | MCAS Iwakuni |
| 10 | MCAS_Iwakuni_1_100m | [1944.7564697265625, 280.8858947753906, 37.818... | 0.468951 | 0 | plane | [] | 0 | 1270 | MCAS Iwakuni |
| 11 | MCAS_Iwakuni_1_100m | [1945.3134765625, 280.3576354980469, 38.140258... | 0.511455 | 1 | helicopter | [] | 0 | 1293 | MCAS Iwakuni |
| 12 | MCAS_Iwakuni_1_100m | [1922.3946533203125, 336.11224365234375, 72.19... | 0.454081 | 1 | helicopter | [] | 0 | 5241 | MCAS Iwakuni |