Multi attribute object detection on rare aircraft data sets: experimental process using yolov5

 Click on the above “ Xiaobai studies vision ”, Optional plus " Star standard " or “ Roof placement ”
 Heavy dry goods , First time delivery 

Reading guide

How to use multiple features of an object to improve the ability of object detection , Use YOLOv5 Carry out the experiment of multi-attribute object detection .

We released RarePlanes Data sets and results of baseline experiments . today , We try to further demonstrate the multiple features of the data set and its unique uses . We trained a target detection model , It can not only identify the aircraft , You can also identify their characteristics , Such as the number of engines 、 Wing shape, etc , And built a tutorial , So you can do this experiment yourself .

In this series of tutorials , We will introduce from beginning to end in RarePlanes Training on data sets YOLOv5 The whole machine learning process of the model .

Quickly review ：RarePlanes Data sets are created by CosmiQ Works and AI.Reverie It is created by combining remote sensing data mainly from airports with comprehensively generated data . Then these images are divided into 5 Features 、10 Attributes and 33 Sub attributes are classified . Each aircraft is marked by a diamond from the nose to the wingtip and then to the tail , To maintain the ratio of width to length , then , Different aircraft features are marked on each label .

The following is the aircraft classification tree used in the dataset .

Model (YOLOv5)

Before we start , First, introduce the background . We try semantic segmentation and object detection . Final , We decided to use YOLOv5 Do object detection , In hindsight , That's right , The segmentation method is difficult to separate similar objects close to each other .

YOLO The network has shown excellent performance in various tasks .

You Only Look Once version 5 (YOLOv5), Just like its predecessor , It is an object detection network . It divides the input image into grids , Then the confidence of bounding box and class probability matrix of each grid box are output . Then filter these outputs , Remove overlap and low confidence detection from the final prediction . These bounding boxes are then transmitted to a neural network for detection . Use YOLO Grid suggestion method ( instead of R-CNN The larger area used in the style network suggests that the network ), Prediction is much faster , allow YOLOv5 Working in real time . We choose to use Ultralytics Of YOLOv5 Realization , Because it's very simple , Using this model to create pipes is much simpler than using similar methods .

Preprocessing / preparation

RarePlanes The dataset contains tiled Images , these tiled The image is in PS-RGB_tiled Local selection around the instance found in the directory . We suggest training these images first , Because they can improve the training speed . Download the picture , It must be organized according to the following structure ：

YOLOv5 Data hierarchy

Use RarePlanes Data sets , You can provide many options for the features you want to detect . for example , You can check the position of the plane 、 A single attribute or a unique combination of attributes of an aircraft . If we want to detect a unique combination of attributes , The first step in preprocessing is to create a custom class .“role”,“num_engines”,“propulsion”,“canards”,“num_tail_fins”,“wing_position”,“wing_type”,“faa_wingspan_class” Any combination of can be used to create a custom class . In this tutorial , We choose to combine... In a custom class “num_engines” and “propulsion”, Because we hope to promote the model inference of these two classes by forcing the model to try to identify these two related attributes .

Then you must add a list of these custom classes to YOLO specific data .yaml In file , This file includes the file path to the training and test images 、 Number of classes and list of classes .

The final step is to create... From the plain film image YOLO label , These images are text files separated by spaces , Contains the class type of each box 、 Location and size . Create these labels for each image , Create a border around each target , For training and model evaluation .

Training / Reasoning

Here are for training and running reasoning pipeline The foundation of . It tells the script where to find the images and training tags we created above .

Training

Use the following command , Use 2 individual NVIDIA Titan XP GPUs Training takes about 4-5 Hours .

Reasoning and scoring scripts are also pre built in this YOLOv5 In the implementation , Can be used as an initial measure of performance . Just point the function to the trained weight , We can do it all in less than two minutes 2700 Run inferences from multiple images .

Preliminary inference ：

Use detect.py Test output examples of different custom classes .

Preliminary score ：

Use a simple single line bash Command to run these scripts . However , These results are not the most accurate , Because they include repeated predictions and partial predictions . Run another round of non maximum suppression , Remove duplicate data , Splice prediction and in tiled Score them on your images . Now? , Let's see how we're doing .

result

Aircraft classified by engine number and power type at Palm Beach County Park Airport

Aircraft classified by engine number and power type at Salt Lake City International Airport

Shanghai Pudong International Airport according to the number of engines + Number of aircraft divided by power type

The color of the class is used as the legend of the above picture

Use F1 Measure ,IoU by 0.5, It turns out that 90 On the aircraft data set in the s F1 The score is very stable . It is worth noting that , The model can identify the location and number of engines , There is no need to train the data set to provide specific annotations for the engine . The number of engines is associated with each aircraft instance , Not the engine itself .

Besides , For less common aircraft , The result is obviously not so good , This may be due to the lack of suitable samples for the model . Before , We discussed how to use synthetic data to enhance these rare classes ( Or rare aircraft ), To improve the performance of specific classes .

summary

Robust machine learning relies heavily on high-quality data sets . Though with AlexNet And the invention of convolutional neural networks , Performance has been significantly improved , But the prediction mechanism lacks real proof . Final , The model depends on “ notice ” Enough scenarios similar to the test scenario , So as to make an accurate prediction ( It can be hundreds or thousands of scenes ). With these , Diversified 、 Organized 、 Well marked data sets can create effective models , But it should be noted that , You don't necessarily need a lot of data . diversification 、 High quality data can often create similar performance models with less data , Even just use 3% The data from 2/3 Performance model of .

However , As a data scientist , His role is not to provide as much data as possible to the model , Instead, generate the most accurate prediction to solve some problems . under these circumstances , We do this by creating meaningful custom classes , But in other cases , This may mean excluding less relevant features , wait . Creating custom classes can improve performance , Because it forces the model to consider the specific properties of the aircraft . for example , Use a combination of power type and number of engines , We see that the classification of the two attributes has been improved . Creating bias is an inherent part of this process .

All in all , Methods like this can be applied across domains , From obvious national security applications to health-related applications , For example, automatic detection of specific cell types in tissue scanning , Its accuracy is similar to that of manual counting , If not higher . Rare aircraft types can also test the value of synthetic data , Improve detection technology , Or evaluate zero sample or few sample learning , Achieve significant progress in the field of computer vision . We hope to introduce similar application centered research and tutorials , To promote the development of this field .


download 1：OpenCV-Contrib Chinese version of extension module

stay 「 Xiaobai studies vision 」 Official account back office reply ： Extension module Chinese course , You can download the first copy of the whole network OpenCV Extension module tutorial Chinese version , cover Expansion module installation 、SFM Algorithm 、 Stereo vision 、 Target tracking 、 Biological vision 、 Super resolution processing And more than 20 chapters .

download 2：Python Visual combat project 52 speak

stay 「 Xiaobai studies vision 」 Official account back office reply ：Python Visual combat project , You can download the Image segmentation 、 Mask detection 、 Lane line detection 、 Vehicle count 、 Add Eyeliner 、 License plate recognition 、 Character recognition 、 Emotional tests 、 Text content extraction 、 face recognition etc. 31 A visual combat project , Help fast school computer vision .

download 3：OpenCV Actual project 20 speak

stay 「 Xiaobai studies vision 」 Official account back office reply ：OpenCV Actual project 20 speak , You can download the 20 Based on OpenCV Realization 20 individual Actual project , Realization OpenCV Learn advanced .

Communication group

Welcome to join the official account reader group to communicate with your colleagues , There are SLAM、 3 d visual 、 sensor 、 Autopilot 、 Computational photography 、 testing 、 Division 、 distinguish 、 Medical imaging 、GAN、 Wechat groups such as algorithm competition （ It will be subdivided gradually in the future ）, Please scan the following micro signal clustering , remarks ：” nickname + School / company + Research direction “, for example ：” Zhang San  +  Shanghai Jiaotong University  +  Vision SLAM“. Please note... According to the format , Otherwise, it will not pass . After successful addition, they will be invited to relevant wechat groups according to the research direction . Do not Send ads within the group , Or you'll be invited out , Thanks for your understanding ~