A deep learning object detector framework written in Python for supporting Land Search and Rescue Missions.

Last update: Dec 22, 2022

Overview

AIR: Aerial Inspection RetinaNet for supporting Land Search and Rescue Missions

AIR is a deep learning based object detection solution to automate the aerial drone footage inspection task frequently carried out during search and rescue (SAR) operations with drone units. It provides a fast, convenient and reliable way to augment aerial, high-resolution image inspection for clues about human presence by highlighting relevant image regions with bounding boxes, as done in the image below. With the assistance of AIR, SAR missions with aerial drone searches can likely be carried out much faster before, and with considerably higher success rate.

This code repository is based on the master's thesis work by Pasi Pyrrö from Aalto University, School of Science with the title "AIR: Aerial Inspection RetinaNet for Land Search and Rescue Missions". This thesis work was funded by Accenture.

Key features

Supports both video and images
Highly accurate deep learning detector custom built for this application
Decent processing speed despite the high-resolution images and a very deep neural network used
Supports object tracking with kalman filter
Implements the novel MOB bounding box postprocessing algorithm
Implements the SAR-APD evaluation algorithm for real-world, value-oriented metric reporting in SAR aerial person detection applications
Plethora of options for customization (e.g., in bounding box postprocessing, image preprocessing, speed vs accuracy trade-offs, etc.)

Results on HERIDAL test set

In the table below are listed the results of the AIR detector and other notable state-of-the-art methods on the HERIDAL test set. Note that the FPS metric is a rough estimate based on the reported inference speed and hardware setup by the original authors, and should be taken as mostly directional. We did not test any of the other methods ourselves.

Method	Precision	Recall	AP	FPS
Mean shift segmentation method [1]	18.7	74.7	-	-
Saliency guided VGG16 [2]	34.8	88.9	-	-
Faster R-CNN [3]	67.3	88.3	86.1	1
Two-stage multimodel CNN [4]	68.9	94.7	-	0.1
SSD [4]	4.3	94.4	-	-
AIR with NMS (ours)	90.1	86.1	84.6	1

It turns out AIR achieves state-of-the-art results in precison and inference speed while having comparable recall to the strongest competitors!

You can check out the full details of AIR evaluation in this Wandb report.

Bounding Box Aggregation options

AIR implements both NMS and MOB algorithms for bounding box prediction postprocessing. The image above shows the main difference: MOB (c) merges bounding boxes (a) instead of eliminating them like NMS (b). Thus, choosing MOB can produce visually more pleasing predictions. Moreover, MOB is less sensitive to the choice of condifence score threshold, making it more robust under unseen data. AIR also comes with a custom SAR-APD evaluation scheme that truthfully ranks MOB-equipped object detector performance (as standard object detection metrics, such as VOC2012 AP, do not like MOB very much).

Should you choose MOB postprocessing and SAR-APD evaluation, you should see similar evaluation results (e.g., after running bash evaluate.sh)

All metrics increased by over 4 percentage points from NMS & VOC2012 results with no loss of prediction visual quality, neat!

Hardware Requirements

x86 or x64 processor architecture (at least for the Docker installation, and if trying to install natively on ARM64 architecture, expect a lot of compiling python packages from source)
NVIDIA® GPU with CUDA® capability 3.5 or higher (recommended)
~8 GB of RAM or more

Software Requirements

If using containers:

Docker or Singularity container system

If using native installation:

Python 3.6
pip and setuptools
FFmpeg (recommended)
Linux or Mac OS (not tested on Windows)

Quick install instructions using Docker

Clone this repo
Either start docker CPU environment by running: bash start-air-cpu-env.sh docker
Or start docker GPU environment by running: bash start-air-gpu-env.sh docker
Inside docker container, build the AIR project by running: python setup.py build_ext --inplace

Quick install instructions using Singularity

Clone this repo
You might wanna specify your SINGULARITY_CACHEDIR env variable prior to installation, if so edit the shell scripts used in the next steps
Either start docker CPU environment by running: bash start-air-cpu-env.sh singularity
Or start docker GPU environment by running: bash start-air-gpu-env.sh singularity
Go to mounted directory by typing cd AIR
Build the AIR project by running: python setup.py build_ext --inplace

Quick native install instructions

Clone this repo
To build the AIR project for CPU, run the command: /usr/bin/python3 -m pip install air-detector[cpu]
To build the AIR project for GPU, run the command: /usr/bin/python3 -m pip install air-detector[gpu]

Download data and the trained model

Save the demo image folder under data/images
Save the demo videos under data/videos
Save the trained model in models/ folder
- In the project root, convert the training model to inference model by running: /bin/bash convert-model.sh
Optionally, you can download and extract the whole HERIDAL dataset in keras-retinanet format to data/datasets

Quick demos

Once everything is set up (installation and asset downloads), you might wanna try out these cool and simple demos to get the hang of using the AIR detector.

Running inference on HERIDAL test image folder

In the project root, run the command: /bin/bash infer.sh
Check data/predictions/dauntless-sweep-2_resnet152_pascal-enclose-inference/ folder for the output images

Evaluating AIR performance on HERIDAL test set (can be slow on CPU)

In the project root, run the command: /bin/bash evaluate.sh
Check data/predictions/dauntless-sweep-2_resnet152_pascal-enclose-sar-apd-eval/ folder for the output images

SAR-APD video detection with image output

In the project root, run the command: /bin/bash convert-model.sh
Then start the inferencing by running: /usr/bin/python3 video_detect.py -c mob_cpu_images
Check data/videos/Ylojarvi-gridiajo-two-guys-moving_air_output/ folder for the output images

SAR-APD video detection with video output

In the project root, run the command: /bin/bash convert-model.sh
Then start the inferencing by running: /usr/bin/python3 video_detect.py -c mob_gpu_tracking
Check data/videos/Ylojarvi-gridiajo-two-guys-moving_air_output_compressed.mov output video

Wandb support

AIR uses experiment tracking software by Weight & Biases
Wandb enables you to view the whole public experimentation record for AIR
AIR experiments can be controlled with the following environment variables:
- WANDB_MODE="disabled"
  - disables all wandb functionality, required if not logged in to wandb
- WANDB_MODE="dryrun"
  - wandb logging works as normal, but nothing is uploaded to cloud
- WANDB_API_KEY=<your_api_key>
  - needed to interact with wandb web UI, you might want to put this in your ~/.bashrc or ~/.zshrc so that it's automatically included into the Docker envs. On most linux based systems, you can achieve this by running this shell command with your Wandb API key:
```
echo "export WANDB_API_KEY=<your_api_key>" >> "~/.${SHELL/\/bin\//}rc"; exec $SHELL
```
- WANDB_DIR=~/wandb
  - select where local log files are stored
- WANDB_PROJECT="air-testing"
  - select your wandb cloud project
- WANDB_ENTITY="ML-Mike"
  - select your wandb account username
- WANDB_RUN_GROUP="cool-experiments"
  - Semantic group within a project to log experiments runs to
More info in wandb documentation (not all env variables are supported though)

Project folder structure

airutils: air utility functions that augment the standard keras-retinanet framework to incorporate aerial person detection (APD) support to it.
config: configuration files for video_detect.py script (easier to use than the CLI in most cases)
data: input and output data for the AIR detector, divided into subfolders:
- /datasets (input for training and evaluation)
- /images (input for general inference)
- /predictions (output of the AIR detector)
- /videos (input for video_detect.py)
keras-retinanet: a fork of the keras-retinanet repository with some AIR modifications
models: contains trained object detection models (both trainable and inference models)

General toubleshooting

Try setting the AIR_VERBOSE=1 enviroment variable to see full TF logs
If using docker, make sure you're allocating enough memory for the container (like at least 8 GB)
If some script simply gets "killed", it's a clear indicator that you have too little memory allocated
Tracker options in video_detect.py might need to be recalibrated for each use case for best performance
If you edit any of the bash scripts in the root folder (e.g., evaluate.sh), make sure there are no whitespace after linebreaks '\', bash can be picky about these things... Also avoid commenting out any command line parameters in those scripts, just delete the whole line outright
The actual python scripts corresponding to the bash scripts are in keras_retinanet/keras_retinanet/bin folder and they can be called directly (or you can try the installed console scripts (e.g., air-evaluate) if you ran pip install .) with approproate parameters (examples can be found in those bash scripts)
Using the command line parameter -h or --help usually helps (pun intended)
If you find a bug or have a complicated question, open a new issue here

Acknowledgements

Kudos to Aalto Science-IT project for providing the compute hardware for training and testing the AIR detector
Big thanks to contributors of keras-retinanet, and their marvellous programming efforts that have allowed AIR to exists in the first place!
Huge credit to the authors of the HERIDAL dataset [2], for which AIR equally well owes its very existence
Also thanks to Ari Nissinen from Finnish Voluntary Rescue Service (VAPEPA) for providing video material used in development and testing of the AIR detector
Lastly, thanks to Hassan Naseri from Accenture for instructing with the thesis work and providing valuable insights throughout the whole AIR project

How to cite

@MastersThesis{pyrro2021air, 
	title={{AIR:} {Aerial} Inspection RetinaNet for Land Search and Rescue Missions}, 
	author={Pyrr{\"o}, Pasi and Naseri, Hassan and Jung, Alexander},
	school={Aalto University, School of Science},
	year={2021}
}

Licenses

AIR is licensed under the Apache 2.0 License
HERIDAL data is licensed under the Creative Commons Attribution 3.0 Unported (CC BY 3.0) License
Demo videos and the trained model are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) License

References

[1] TURIC, H., DUJMIC, H., AND PAPIC, V. Two-stage segmentation of aerial images for search and rescue. Information Technology and Control 39, 2 (2010).

[2] BOŽIC-ŠTULIC, D., MARUŠIC, Ž., AND GOTOVAC, S. Deep learning approach in aerial imagery for supporting land search and rescue missions. International Journal of Computer Vision 127, 9 (2019), 1256–1278.

[3] MARUŠIC, Ž., BOŽIC-ŠTULIC, D., GOTOVAC, S., AND MARUŠIC, T. Region proposal approach for human detection on aerial imagery. In 2018 3rd International Conference on Smart and Sustainable Technologies (SpliTech) (2018), IEEE, pp. 1–6.

[4] VASIC, M. K., AND PAPIC, V. Multimodel deep learning for person detection in aerial images. Electronics 9, 9 (2020), 1459.

Comments

cudnn failed to initialyze

Hello i tried to install the docker-gpu images, i've download the weights and the data, but when i launch /bin/bash infer.sh i get this response :

[email protected]:/home# /bin/bash infer.sh 
Using TensorFlow backend.
2022-03-01 23:46:15.554674: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-03-01 23:46:15.578708: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2499950000 Hz
2022-03-01 23:46:15.579340: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x561a790 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-03-01 23:46:15.579364: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2022-03-01 23:46:15.580946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2022-03-01 23:46:15.636357: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-03-01 23:46:15.636553: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x53a4460 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-03-01 23:46:15.636569: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce GTX 1650 Ti with Max-Q Design, Compute Capability 7.5
2022-03-01 23:46:15.636762: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-03-01 23:46:15.636888: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA GeForce GTX 1650 Ti with Max-Q Design major: 7 minor: 5 memoryClockRate(GHz): 1.2
pciBusID: 0000:01:00.0
2022-03-01 23:46:15.637043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-03-01 23:46:15.637856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-03-01 23:46:15.638563: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-03-01 23:46:15.638766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-03-01 23:46:15.639774: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-03-01 23:46:15.640478: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-03-01 23:46:15.642728: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-03-01 23:46:15.642815: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-03-01 23:46:15.642967: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-03-01 23:46:15.643040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-03-01 23:46:15.643072: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-03-01 23:46:15.643622: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-03-01 23:46:15.643634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2022-03-01 23:46:15.643639: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2022-03-01 23:46:15.643714: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-03-01 23:46:15.643833: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-03-01 23:46:15.643934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1650 Ti with Max-Q Design, pci bus id: 0000:01:00.0, compute capability: 7.5)
Loading model, this may take a second...
tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
Running inference on image folder: /home/data/images/test
Running network: N/A% (0 of 4) |                                                                                                                                     | Elapsed Time: 0:00:00 ETA:  --:--:--2022-03-01 23:46:40.231228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-03-01 23:46:40.548887: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2022-03-01 23:46:40.554431: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
Traceback (most recent call last):
  File "keras_retinanet/keras_retinanet/bin/infer.py", line 215, in <module>
    main()
  File "keras_retinanet/keras_retinanet/bin/infer.py", line 192, in main
    profile=args.profile
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
    max_inflation_factor=max_inflation_factor
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 134, in run_inference_on_image
    boxes, scores, labels = model.predict_on_batch(np.expand_dims(image, axis=0))[:3]
  File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1580, in predict_on_batch
    outputs = self.predict_function(ins)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/backend.py", line 3476, in __call__
    run_metadata=self.run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1472, in __call__
    run_metadata_ptr)
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
  (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
	 [[{{node conv1/convolution}}]]
	 [[filtered_detections/map/while/LoopCond/_9551]]
  (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
	 [[{{node conv1/convolution}}]]
0 successful operations.
0 derived errors ignored.

inside the container I checked under the cuda repository and noticed that the cudnn librairies where not inside , so i downloaded it and copy it inside cuda/include and cuda/lib64 but i've got the same result. I am on ubuntu 18.04 i have a GeForce GTX 1650 , nvidia-smi return cuda 11.6 nvida-driver 510 Can you help me on that ? Thank you

help wanted

opened by mrba59 7

Box detect group of person

Hi, I've runned inference on my eval dataset, the results are really good and i congrat you on that it's really impressive. I just have one question. The inference detect 15 person on this image , as you can see some box contains more that one person but they are detected as one, maybe there is a parameter taking account group of person ? Is there any chance to return one box for each person ?

opened by mrba59 3
Dataset unavailable

Hi, nice work! Unfortunately i can't reach out the HERIDAL download page. If i try from here: http://ipsar.fesb.unist.hr/HERIDAL%20database.html It prompts an error.

Is there any chance you have an offline dump you can share with me somehow?

Thanks

opened by lodm94 1

ModuleNotFoundError: No module named 'keras.api'

When I ran !/bin/bash convert-model.sh in Google Colab, it resulted in:

2022-03-22 09:32:42.586979: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
Traceback (most recent call last):
  File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 103, in <module>
    main()
  File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 79, in main
    model = models.load_model(args.model_in, backbone_name=args.backbone)
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/__init__.py", line 87, in load_model
    return keras.models.load_model(filepath, custom_objects=backbone(backbone_name).custom_objects, compile=compile)
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/__init__.py", line 57, in backbone
    from .resnet import ResNetBackbone as b
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/resnet.py", line 18, in <module>
    from keras.utils import get_file
ImportError: cannot import name 'get_file' from 'keras.utils' (/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py)

After some googling I found out that I should install kashgari, so I ran

pip install kashgari

And when I came back to !/bin/bash convert-model.sh, I had:

Using TensorFlow backend.
Traceback (most recent call last):
  File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 33, in <module>
    from ..utils.config import read_config_file, parse_anchor_parameters
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/config.py", line 19, in <module>
    import keras
  File "/usr/local/lib/python3.7/dist-packages/keras/__init__.py", line 3, in <module>
    from . import utils
  File "/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py", line 26, in <module>
    from .vis_utils import model_to_dot
  File "/usr/local/lib/python3.7/dist-packages/keras/utils/vis_utils.py", line 7, in <module>
    from ..models import Model
  File "/usr/local/lib/python3.7/dist-packages/keras/models.py", line 12, in <module>
    from .engine.training import Model
  File "/usr/local/lib/python3.7/dist-packages/keras/engine/__init__.py", line 8, in <module>
    from .training import Model
  File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 14, in <module>
    from . import training_utils
  File "/usr/local/lib/python3.7/dist-packages/keras/engine/training_utils.py", line 17, in <module>
    from .. import metrics as metrics_module
  File "/usr/local/lib/python3.7/dist-packages/keras/metrics.py", line 1850, in <module>
    BaseMeanIoU = tf.keras.metrics.MeanIoU
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/lazy_loader.py", line 58, in __getattr__
    module = self._load()
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/lazy_loader.py", line 41, in _load
    module = importlib.import_module(self.__name__)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'keras.api'

Am I doing something wrong? Is there a way to address this error?

help wanted

opened by dinarakhay 1

Bump opencv-python from 4.1.0.25 to 4.2.0.32
⚠️ Dependabot is rebasing this PR ⚠️

Rebasing might not happen immediately, so don't worry if this takes some time.

Note: if you make any changes to this PR yourself, they will take precedence over the rebase.

Bumps opencv-python from 4.1.0.25 to 4.2.0.32.

Release notes

Sourced from opencv-python's releases.

4.2.0.32

opencv-python: https://pypi.org/project/opencv-python/

opencv-contrib-python: https://pypi.org/project/opencv-contrib-python/

opencv-python-headless: https://pypi.org/project/opencv-python-headless/

opencv-contrib-python-headless: https://pypi.org/project/opencv-contrib-python-headless/

OpenCV version 4.2.0.

Changes:

macOS environment updated from xcode8.3 to xcode 9.4

macOS uses now Qt 5 instead of Qt 4

Nasm version updated to Docker containers

multibuild updated

Fixes:

don't use deprecated brew tap-pin, instead refer to the full package name when installing #267

replace get_config_var() with get_config_vars() in setup.py #274

add workaround for DLL errors in Windows Server #264

4.1.2.30

opencv-python: https://pypi.org/project/opencv-python/

opencv-contrib-python: https://pypi.org/project/opencv-contrib-python/

opencv-python-headless: https://pypi.org/project/opencv-python-headless/

opencv-contrib-python-headless: https://pypi.org/project/opencv-contrib-python-headless/

OpenCV version 4.1.2.

Changes:

Python 3.8 builds added to the build matrix

Support for Python 3.4 builds dropped (Python 3.4 is in EOL)

multibuild updated

minor build logic changes

Docker images rebuilt

Notes:

Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

4.1.1.26

opencv-python: https://pypi.org/project/opencv-python/

opencv-contrib-python: https://pypi.org/project/opencv-contrib-python/

opencv-python-headless: https://pypi.org/project/opencv-python-headless/

opencv-contrib-python-headless: https://pypi.org/project/opencv-contrib-python-headless/

OpenCV version 4.1.1.

Changes:

... (truncated)

Commits

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump pillow from 6.2.1 to 8.3.2
⚠️ Dependabot is rebasing this PR ⚠️

Rebasing might not happen immediately, so don't worry if this takes some time.

Note: if you make any changes to this PR yourself, they will take precedence over the rebase.

Bumps pillow from 6.2.1 to 8.3.2.

Release notes

Sourced from pillow's releases.

8.3.2

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.2.html

Security

CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

Fix 6-byte OOB read in FliDecode [wiredfool]

Python 3.10 wheels

Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

Fixed regressions

Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

Updates for ImagePalette channel order #5599 [radarhere]

Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

8.3.1

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.1.html

Changes

Catch OSError when checking if fp is sys.stdout #5585 [@radarhere]

Handle removing orientation from alternate types of EXIF data #5584 [@radarhere]

Make Image.array take optional dtype argument #5572 [@t-vi]

8.3.0

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html

Changes

Use snprintf instead of sprintf #5567 [@radarhere]

Limit TIFF strip size when saving with LibTIFF #5514 [@kmilos]

Allow ICNS save on all operating systems #4526 [@newpanjing]

De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [@gofr]

Do not use background or transparency index for new color #5564 [@radarhere]

Simplified code #5315 [@radarhere]

Replaced xml.etree.ElementTree #5565 [@radarhere]

... (truncated)

Changelog

Sourced from pillow's changelog.

8.3.2 (2021-09-02)

CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

Fix 6-byte OOB read in FliDecode [wiredfool]

Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

Updates for ImagePalette channel order #5599 [radarhere]

Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

8.3.1 (2021-07-06)

Catch OSError when checking if fp is sys.stdout #5585 [radarhere]

Handle removing orientation from alternate types of EXIF data #5584 [radarhere]

Make Image.array take optional dtype argument #5572 [t-vi, radarhere]

8.3.0 (2021-07-01)

Use snprintf instead of sprintf. CVE-2021-34552 #5567 [radarhere]

Limit TIFF strip size when saving with LibTIFF #5514 [kmilos]

Allow ICNS save on all operating systems #4526 [baletu, radarhere, newpanjing, hugovk]

De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [gofr, radarhere]

Replaced xml.etree.ElementTree #5565 [radarhere]

... (truncated)

Commits

8013f13 8.3.2 version bump

23c7ca8 Update CHANGES.rst

8450366 Update release notes

a0afe89 Update test case

9e08eb8 Raise ValueError if color specifier is too long

bd5cf7d FLI tests for Oss-fuzz crash.

94a0cf1 Fix 6-byte OOB read in FliDecode

cece64f Add 8.3.2 (2021-09-02) [CI skip]

e422386 Add release notes for Pillow 8.3.2

08dcbb8 Pillow 8.3.2 supports Python 3.10 [ci skip]

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump opencv-contrib-python from 4.1.0.25 to 4.2.0.32
⚠️ Dependabot is rebasing this PR ⚠️

Rebasing might not happen immediately, so don't worry if this takes some time.

Note: if you make any changes to this PR yourself, they will take precedence over the rebase.

Bumps opencv-contrib-python from 4.1.0.25 to 4.2.0.32.

Release notes

Sourced from opencv-contrib-python's releases.

4.2.0.32

opencv-python: https://pypi.org/project/opencv-python/

opencv-contrib-python: https://pypi.org/project/opencv-contrib-python/

opencv-python-headless: https://pypi.org/project/opencv-python-headless/

opencv-contrib-python-headless: https://pypi.org/project/opencv-contrib-python-headless/

OpenCV version 4.2.0.

Changes:

macOS environment updated from xcode8.3 to xcode 9.4

macOS uses now Qt 5 instead of Qt 4

Nasm version updated to Docker containers

multibuild updated

Fixes:

don't use deprecated brew tap-pin, instead refer to the full package name when installing #267

replace get_config_var() with get_config_vars() in setup.py #274

add workaround for DLL errors in Windows Server #264

4.1.2.30

opencv-python: https://pypi.org/project/opencv-python/

opencv-contrib-python: https://pypi.org/project/opencv-contrib-python/

opencv-python-headless: https://pypi.org/project/opencv-python-headless/

opencv-contrib-python-headless: https://pypi.org/project/opencv-contrib-python-headless/

OpenCV version 4.1.2.

Changes:

Python 3.8 builds added to the build matrix

Support for Python 3.4 builds dropped (Python 3.4 is in EOL)

multibuild updated

minor build logic changes

Docker images rebuilt

Notes:

Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

4.1.1.26

opencv-python: https://pypi.org/project/opencv-python/

opencv-contrib-python: https://pypi.org/project/opencv-contrib-python/

opencv-python-headless: https://pypi.org/project/opencv-python-headless/

opencv-contrib-python-headless: https://pypi.org/project/opencv-contrib-python-headless/

OpenCV version 4.1.1.

Changes:

... (truncated)

Commits

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1

Unable to get working with GPU: OOM and/or hangs

First of all, thanks for putting this together! I'm trying to integrate it into my volunteer search & rescue team's RPAS/drone workflow, and we're all very excited about the initial results.

While the results under CPU mode are great, they are a bit slow for our purposes (using 4000x2250px images). Our team computer has a 3060 GPU (Windows 11, i7 CPU, 32GB RAM), which we hope can speed up processing, but we're unable to get it working in GPU mode.

When running the default GPU Docker install, we get OOM errors:

[email protected]:/home# /bin/bash infer.sh
Using TensorFlow backend.
2022-06-16 02:32:07.062753: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-06-16 02:32:07.068259: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2687995000 Hz
2022-06-16 02:32:07.069773: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x57d5a40 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-06-16 02:32:07.069800: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2022-06-16 02:32:07.072249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2022-06-16 02:32:08.733246: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:32:08.733462: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x570f500 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-06-16 02:32:08.733513: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3060 Laptop GPU, Compute Capability 8.6
2022-06-16 02:32:08.738143: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:32:08.738184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA GeForce RTX 3060 Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.702
pciBusID: 0000:01:00.0
2022-06-16 02:32:08.738376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-06-16 02:32:08.739257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-06-16 02:32:08.740589: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-06-16 02:32:08.740821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-06-16 02:32:08.742371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-06-16 02:32:08.743383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-06-16 02:32:08.745999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-06-16 02:32:08.746592: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:32:08.747027: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:32:08.747050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-06-16 02:32:08.747094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-06-16 02:32:08.747495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-06-16 02:32:08.747514: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2022-06-16 02:32:08.747532: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2022-06-16 02:32:08.748081: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:32:08.748109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1387] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2022-06-16 02:32:08.748516: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:32:08.748577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
Loading model, this may take a second...
2022-06-16 02:36:14.483391: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 1073741824 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:14.483439: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 1073741824
2022-06-16 02:36:14.576914: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 966367744 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:14.576962: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 966367744
2022-06-16 02:36:14.671003: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 869731072 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:14.671047: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 869731072
2022-06-16 02:36:14.770382: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 782758144 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:14.770424: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 782758144
2022-06-16 02:36:14.864264: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 704482304 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:14.864312: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 704482304
2022-06-16 02:36:14.957833: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 634034176 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:14.957889: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 634034176
2022-06-16 02:36:15.053452: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 570630912 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.053498: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 570630912
2022-06-16 02:36:15.150759: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 513568000 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.150802: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 513568000
2022-06-16 02:36:15.243685: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 462211328 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.243726: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 462211328
2022-06-16 02:36:15.335976: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 415990272 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.336028: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 415990272
2022-06-16 02:36:15.427373: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 374391296 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.427417: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 374391296
2022-06-16 02:36:15.520526: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 336952320 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.520570: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 336952320
2022-06-16 02:36:15.612661: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 303257088 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.612720: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 303257088
2022-06-16 02:36:15.704694: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 272931584 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.704741: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 272931584
2022-06-16 02:36:15.798110: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 245638656 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.798153: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 245638656
2022-06-16 02:36:15.889580: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 221074944 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.889628: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 221074944
2022-06-16 02:36:15.984319: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 198967552 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:15.984362: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 198967552
2022-06-16 02:36:16.077288: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 179070976 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:16.077332: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 179070976
2022-06-16 02:36:16.169940: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 161164032 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:16.169981: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 161164032
2022-06-16 02:36:16.264281: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 145047808 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2022-06-16 02:36:16.264328: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 145047808
tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
Running inference on image folder: /home/data/images/test
Running network: N/A% (0 of 138) |                                          | Elapsed Time: 0:00:00 ETA:  --:--:--2022-06-16 02:36:21.094051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-06-16 02:48:27.323880: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal   : Value 'sm_86' is not defined for option 'gpu-name'

Relying on driver to perform ptx compilation. This message will be only logged once.
2022-06-16 02:48:27.773454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-06-16 02:49:29.518638: W tensorflow/core/common_runtime/bfc_allocator.cc:305] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
Traceback (most recent call last):
  File "keras_retinanet/keras_retinanet/bin/infer.py", line 215, in <module>
    main()
  File "keras_retinanet/keras_retinanet/bin/infer.py", line 192, in main
    profile=args.profile
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
    max_inflation_factor=max_inflation_factor
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 168, in run_inference_on_image
    max_inflation_factor=max_inflation_factor)
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 365, in merge_boxes_per_label
    iou_threshold, max_iterations, top_k, max_inflation_factor, merge_mode)
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 276, in merge_overlapping_boxes
    X = 1. - compute_overlap(boxes, boxes)
  File "compute_overlap.pyx", line 27, in compute_overlap.compute_overlap
    cdef np.ndarray[double, ndim=2] overlaps = np.zeros((N, K), dtype=np.float64)
MemoryError: Unable to allocate array with shape (100000, 100000) and data type float64
.....

I've tried some extreme configuration options to lessen the memory burden, but we still get the OOM errors:

    --image_min_side 100 
    --image_max_side 150 
    --image_tiling_dim 8

I found that by setting compile=False in the below line of keras_retinanet\keras_retinanet\bin\infer.py, I avoid getting the Cuda OOM errors, but the process just hangs and eventually errors out the same. model = models.load_model(args.model, backbone_name=args.backbone, compile=False)

Complete output with compile=False:

[email protected]:/home# /bin/bash infer.sh
Using TensorFlow backend.
2022-06-16 02:09:13.780103: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-06-16 02:09:13.786143: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2687995000 Hz
2022-06-16 02:09:13.787804: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5b70d30 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-06-16 02:09:13.787838: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2022-06-16 02:09:13.790658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2022-06-16 02:09:15.386150: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:09:15.386309: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x58fbfd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-06-16 02:09:15.386346: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3060 Laptop GPU, Compute Capability 8.6
2022-06-16 02:09:15.389332: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:09:15.389373: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA GeForce RTX 3060 Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.702
pciBusID: 0000:01:00.0
2022-06-16 02:09:15.389623: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-06-16 02:09:15.390896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-06-16 02:09:15.392376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-06-16 02:09:15.392657: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-06-16 02:09:15.395000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-06-16 02:09:15.396610: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-06-16 02:09:15.399907: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-06-16 02:09:15.400535: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:09:15.401007: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:09:15.401035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-06-16 02:09:15.401082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-06-16 02:09:15.401523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-06-16 02:09:15.401542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2022-06-16 02:09:15.401581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2022-06-16 02:09:15.402103: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:09:15.402132: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1387] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2022-06-16 02:09:15.402663: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-06-16 02:09:15.402747: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
Loading model, this may take a second...
tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
Running inference on image folder: /home/data/images/test
Running network: N/A% (0 of 138) |                                               | Elapsed Time: 0:00:00 ETA:  --:--:--2022-06-16 02:13:11.539626: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-06-16 02:25:21.996183: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal   : Value 'sm_86' is not defined for option 'gpu-name'

Relying on driver to perform ptx compilation. This message will be only logged once.
2022-06-16 02:25:22.378005: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-06-16 02:26:24.908769: W tensorflow/core/common_runtime/bfc_allocator.cc:305] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
Traceback (most recent call last):
  File "keras_retinanet/keras_retinanet/bin/infer.py", line 216, in <module>
  File "keras_retinanet/keras_retinanet/bin/infer.py", line 193, in main
    )
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
    max_inflation_factor=max_inflation_factor
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 168, in run_inference_on_image
    max_inflation_factor=max_inflation_factor)
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 365, in merge_boxes_per_label
    iou_threshold, max_iterations, top_k, max_inflation_factor, merge_mode)
  File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 276, in merge_overlapping_boxes
    X = 1. - compute_overlap(boxes, boxes)
  File "compute_overlap.pyx", line 27, in compute_overlap.compute_overlap
    cdef np.ndarray[double, ndim=2] overlaps = np.zeros((N, K), dtype=np.float64)
MemoryError: Unable to allocate array with shape (100000, 100000) and data type float64

In case it helps:

[email protected]:/home# nvidia-smi
Thu Jun 16 02:29:41 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.73.08    Driver Version: 512.96       CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
| N/A   47C    P0    24W /  N/A |      0MiB /  6144MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Any suggestions would be very much appreciated, as I think I'm fully out of ideas and we're really keen to make this work!

opened by duncanenman 2

Releases(0.0.2)

0.0.2(Jan 14, 2022)

More stable version of the AIR detector with various bug fixes.

Full Changelog: https://github.com/Accenture/AIR/compare/0.0.1...0.0.2
Source code(tar.gz)
Source code(zip)
0.0.1(Nov 9, 2021)

The first official release of the AIR detector.

Full Changelog: https://github.com/Accenture/AIR/commits/0.0.1
Source code(tar.gz)
Source code(zip)

A deep learning object detector framework written in Python for supporting Land Search and Rescue Missions.

Related tags

Overview

AIR: Aerial Inspection RetinaNet for supporting Land Search and Rescue Missions

Key features

Results on HERIDAL test set

Bounding Box Aggregation options

Hardware Requirements

Software Requirements

Quick install instructions using Docker

Quick install instructions using Singularity

Quick native install instructions

Download data and the trained model

Quick demos

Running inference on HERIDAL test image folder

Evaluating AIR performance on HERIDAL test set (can be slow on CPU)

SAR-APD video detection with image output

SAR-APD video detection with video output

Wandb support

Project folder structure

General toubleshooting

Acknowledgements

How to cite

Licenses

References

Comments

4.2.0.32

4.1.2.30

4.1.1.26

8.3.2

Security

Python 3.10 wheels

Fixed regressions

8.3.1

Changes

8.3.0

Changes

8.3.2 (2021-09-02)

8.3.1 (2021-07-06)

8.3.0 (2021-07-01)

4.2.0.32

4.1.2.30

4.1.1.26

Releases(0.0.2)

0.0.2(Jan 14, 2022)

0.0.1(Nov 9, 2021)

Owner

Accenture

Export CenterPoint PonintPillars ONNX Model For TensorRT

Code for "Adversarial Training for a Hybrid Approach to Aspect-Based Sentiment Analysis

M2MRF: Many-to-Many Reassembly of Features for Tiny Lesion Segmentation in Fundus Images

Official repository of Semantic Image Matting

Code for IntraQ, PyTorch implementation of our paper under review

public repo for ESTER dataset and modeling (EMNLP'21)

Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)

Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds (Local-Lip)

Improving Query Representations for DenseRetrieval with Pseudo Relevance Feedback:A Reproducibility Study.

Jingju baseline - A baseline model of our project of Beijing opera script generation

Quantile Regression DQN a Minimal Working Example, Distributional Reinforcement Learning with Quantile Regression

QuadTree Attention for Vision Transformers (ICLR2022)

RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

Python package provinding tools for artistic interactive applications using AI

YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with ONNX, TensorRT, ncnn, and OpenVINO supported.

A simple configurable bot for sending arXiv article alert by mail

UI2I via StyleGAN2 - Unsupervised image-to-image translation method via pre-trained StyleGAN2 network

3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks

This repository contains the code needed to train Mega-NeRF models and generate the sparse voxel octrees

The implementation of "Optimizing Shoulder to Shoulder: A Coordinated Sub-Band Fusion Model for Real-Time Full-Band Speech Enhancement"