Detectron: train your own data set -- convert your own data format to coco format

2021/12/6 to update ： See here for the complete code ：

turn COCO Format code https://download.csdn.net/download/qq_15969343/85088683

2021/7/6 to update ： still windows It's more comfortable to use ！ Welcome to use faster , Algorithm more Detectron2：

WIN10 The installation is based on Pytorch1.8.1、Cuda11.1.1 Of Detectron2（0.1 or 0.4.1）

The following is the original ：

---

Detectron series ：

1. Detectron:assert len(cfg_list) % 2 == 0 AssertionError

2. Detectron： Test with your trained model

1. Antecedents feed    

    Fackbook Open source detection framework Detectron It has been open source for some time ：

    https://github.com/facebookresearch/Detectron

     But I used to use Keras and Tensorflow, No contact caffe2, So I haven't tried ,~~~ And I haven't used it before linux System , Basically don't understand commands ~~, Fortunately, I can use the browser , If you encounter problems, check , It can be regarded as stepping on all the pits , Finally, there was no danger , With powerful Detectron La .

    About installation caffe2 and Detectron I'm writing when I'm free ~！    This article mainly talks about how to make the annotation format required by the framework ！ But what? ~~~ Because customizing your own dataset is cumbersome , Recommended Detectron Self contained COCO Dataset name , And convert your data set into COCO The format of the dataset , Of course, if you're big , Ignore this ~~ Now? , you 're right , The time is now ~~~ Let's share how to generate this COCO Format ~~~

2.COCO Dataset format  

     First , adopt COCO - Common Objects in Context（COCO Official website ） I understand coco The format of the data set is as follows ：

     For target detection , We need to pay attention to BBOX The format is as follows ：

     that , We just need to convert our data format to the above ~

3. Transform your own data set

  3.1 Data placement

     We need to create a folder , Used for storing pictures and marking data , The specific placement method is as follows ：

    ├── annos.txt
    ├── annotations
    ├── classes.txt
    └── images

    among ,annos Place the original annotation file of the dataset , May be txt, perhaps csv Format ;classes.txt Place the category name you marked , One category per line , No background ;images Place the original image file of the dataset .annotations Prepare to place with COCO Annotation file in dataset format . Next, we need to generate our own annotation file COCO Format label file .

   3.2 Convert your own data

    My own data is shown below ：

     Each line corresponds to one BBOX Mark ：filename,label, x_min, y_min, x_max, y_max , Now let's start the conversion :

    We use os extract images The name of the picture in the folder , And will BBox Read it all ：

import json
import os
import cv2

#  The root path , It contains images( Picture folder ),annos.txt(bbox mark ),classes.txt( Category label ), as well as annotations Folder ( If not, it will be created automatically , Used to save the last json)
root_path = 'E:\dogcat\\f_train\data\\get_json\\'
#  Used to create training sets or validation sets 
phase = 'val'
#  The boundary between training set and verification set 
split = 8000

#  Open the category tab 
with open(os.path.join(root_path, 'classes.txt')) as f:
  classes = f.read().strip().split()

#  Create category labels and numbers id Correspondence of 
for i, cls in enumerate(classes, 1):
  dataset['categories'].append({'id': i, 'name': cls, 'supercategory': 'mark'})

#  Read images The picture name of the folder 
indexes = [f for f in os.listdir(os.path.join(root_path, 'images'))]

#  Determine whether to establish a training set or a verification set 
if phase == 'train':
  indexes = [line for i, line in enumerate(_indexes) if i <= split]
elif phase == 'val':
  indexes = [line for i, line in enumerate(_indexes) if i > split]

#  Read Bbox Information 
with open(os.path.join(root_path, 'annos.txt')) as tr:
    annos = tr.readlines()

And then , The above data is converted to COCO The required

for k, index in enumerate(indexes):

    #  use opencv Read the picture , Get the width and height of the image 
    im = cv2.imread(os.path.join(root_path, 'images/') + index)
    height, width, _ = im.shape

    #  Add image information to dataset in 
    dataset['images'].append({'file_name': index,
                              'id': k,
                              'width': width,
                              'height': height})

For a graph with multiple boxes, judge ：

for ii, anno in enumerate(annos):
    parts = anno.strip().split()

    #  If the name of the image and the name of the tag are aligned , Then add a mark 
    if parts[0] == index:
        #  Category 
        cls_id = parts[1]
        # x_min
        x1 = float(parts[2])
        # y_min
        y1 = float(parts[3])
        # x_max
        x2 = float(parts[4])
        # y_max
        y2 = float(parts[5])
        width = max(0, x2 - x1)
        height = max(0, y2 - y1)
        dataset['annotations'].append({
            'area': width * height,
            'bbox': [x1, y1, width, height],
            'category_id': int(cls_id),
            'id': i,
            'image_id': k,
            'iscrowd': 0,
            # mask,  The rectangle is four vertices clockwise from the top left corner 
            'segmentation': [[x1, y1, x2, y1, x2, y2, x1, y2]]
        })

Then save the results ：

#  The folder where the results are saved 
folder = os.path.join(root_path, 'annotations')
if not os.path.exists(folder):
  os.makedirs(folder)
json_name = os.path.join(root_path, 'annotations/{}.json'.format(phase))
with open(json_name, 'w') as f:
  json.dump(dataset, f)

View results ：