Detailed explanation of pipline of mmdetection

---

Preface

  Part 1 It introduces mmdet How to build dataset General idea of . This chapter introduces in more detail mmdet How to read data .

1、CustomDataset Class instantiation

  stay mmdet/datasets/custom.py Classes are defined in CustomDataset, For most datasets Dataset All implementations of must inherit this class , such as coco Data sets . Different Dataset Class initialization is much the same , Therefore, this article takes the instantiation of the parent class as the starting point , First introduced CustomDataset Class Initialize configuration file and Initialization process ：

# config The configuration file 
    train=dict(
        type=dataset_type,
        imgset=data_root + 'ImageSets/trainval.txt',
        classwise=False,
        ann_file=data_root + 'FullDataSet/Annotations/',
        img_prefix=data_root + 'FullDataSet/AllImages/',
        pipeline=train_pipeline),

@DATASETS.register_module()
class CustomDataset(Dataset):
    CLASSES = None
    def __init__(self,
                 ann_file,                 # anno The path of 
                 pipeline,                 #  It's a dictionary {'loadImg','loadAnno'}
                 classes=None,            
                 data_root=None,           #  Dataset root :  such as /home/dataset/coco
                 img_prefix='',            #  Directory of images  : /images
                 seg_prefix=None,
                 proposal_file=None,
                 test_mode=False,
                 filter_empty_gt=True):    #  Filter the empty gt
        # processing pipeline
        self.pipeline = Compose(pipeline)  #  Data processing pipeline 

  In instantiation Dataset after , When traversing a read data set , Would call CustomDataset Of getitem Method , So here we put , Here's a little extra attention results This dictionary :

    def __getitem__(self, idx):
        if self.test_mode:
            return self.prepare_test_img(idx)
        while True:
            data = self.prepare_train_img(idx)  #  With train For example, the process 
            if data is None:
                idx = self._rand_another(idx)
                continue
            return data
            
    def prepare_train_img(self, idx):
        img_info = self.data_infos[idx]
        ann_info = self.get_ann_info(idx)
        results = dict(img_info=img_info, ann_info=ann_info) # result Added two messages 
        if self.proposals is not None:
            results['proposals'] = self.proposals[idx]
        self.pre_pipeline(results)                           #  And to results A new key is passed in 
        return self.pipeline(results)                        #  take results To pipeline 
        
    def pre_pipeline(self, results):
        """Prepare results dict for pipeline"""
        results['img_prefix'] = self.img_prefix              #  Added fields such as image prefix 
        results['seg_prefix'] = self.seg_prefix
        results['proposal_file'] = self.proposal_file
        results['bbox_fields'] = []
        results['mask_fields'] = []
        results['seg_fields'] = []

  Post here results An example of .

2、Pipline

  In the composite Dictionary results after ,CustomDaaset Inside getitem Methods by self.piplien(results) To pipeline . That is, the picture often seen on the official website ：

  The green field indicates to result Which is newly added key, The orange field indicates the current pipline May change key Of value. With Resize For example （ The green notes are very clear ）：

    def __call__(self, results):
        """Call function to resize images, bounding boxes, masks, semantic segmentation map. Args: results (dict): Result dict from loading pipeline. Returns: dict: Resized results, 'img_shape', 'pad_shape', 'scale_factor', 'keep_ratio' keys are added into result dict. """

        if 'scale' not in results:
            if 'scale_factor' in results:
                img_shape = results['img'].shape[:2]
                scale_factor = results['scale_factor']  #  Go to result add to scale_factor Field 
                assert isinstance(scale_factor, float)
                results['scale'] = tuple(               #  add to scale Field 
                    [int(x * scale_factor) for x in img_shape][::-1])
            else:
                self._random_scale(results)
        else:
            assert 'scale_factor' not in results, (
                "scale and scale_factor cannot be both set.")

        self._resize_img(results)
        self._resize_bboxes(results)
        self._resize_masks(results)
        self._resize_seg(results)
        return results

  Because of the key It may be used later , So we are building our own Pipeline Be sure to carefully check the dictionary you have modified or added key and value, Because once you overwrite or modify the contents of the original dictionary by mistake , The code may not report an error , If appear bug, It is more difficult to check .

3 、DefaultFormatBundle

&emspl I'm seeing this pipline The time is rather misty , therefore , I'll take it out and analyze it alone , Post the code first ：

from mmcv.parallel import DataContainer as DC    # DataContainer For the time being, it can be understood as a container 

@PIPELINES.register_module()
class DefaultFormatBundle(object):
    """ #  Will be result In the following key Press 1 --> 2 Packed in the order of  - img: (1)transpose, (2)to tensor, (3)to DataContainer (stack=True) - proposals: (1)to tensor, (2)to DataContainer - gt_bboxes: (1)to tensor, (2)to DataContainer - gt_bboxes_ignore: (1)to tensor, (2)to DataContainer - gt_labels: (1)to tensor, (2)to DataContainer - gt_masks: (1)to tensor, (2)to DataContainer (cpu_only=True) - gt_semantic_seg: (1)unsqueeze dim-0 (2)to tensor, (3)to DataContainer (stack=True) """

    def __call__(self, results):
        """Call function to transform and format common fields in results. Args: results (dict): Result dict contains the data to convert. Returns: dict: The result dict contains the data that is formatted with default bundle. """

        if 'img' in results:
            img = results['img']
            # add default meta keys
            results = self._add_default_meta_keys(results)
            if len(img.shape) < 3:
                img = np.expand_dims(img, -1)
            img = np.ascontiguousarray(img.transpose(2, 0, 1))
            results['img'] = DC(to_tensor(img), stack=True)      # img Turn into tensor Put in DC
        for key in ['proposals', 'gt_bboxes', 'gt_bboxes_ignore', 'gt_labels']:
            if key not in results:
                continue
            results[key] = DC(to_tensor(results[key]))           #  Turn into tensor In the DC
        if 'gt_masks' in results:
            results['gt_masks'] = DC(results['gt_masks'], cpu_only=True)
        if 'gt_semantic_seg' in results:
            results['gt_semantic_seg'] = DC(
                to_tensor(results['gt_semantic_seg'][None, ...]), stack=True)
        return results
	
	#  Add some "pad_shape,scale_factor The default fields of avoid none Resize Of pipline Resulting in a lack of key The problem of "
    def _add_default_meta_keys(self, results):
        """Add default meta keys. """
        img = results['img']
        results.setdefault('pad_shape', img.shape)
        results.setdefault('scale_factor', 1.0)
        num_channels = 1 if len(img.shape) < 3 else img.shape[2]
        results.setdefault(
            'img_norm_cfg',
            dict(
                mean=np.zeros(num_channels, dtype=np.float32),
                std=np.ones(num_channels, dtype=np.float32),
                to_rgb=False))
        return results

  This kind of simple sentence , Will be result Some of them key： such as img,gt_bbox,gt_labels Turn into tensor Put a DataContainer In the container of , Haha, let me introduce you DataContainer.

3.1 DataContainer class

  Realize in mmcv/parallel/data_container.py in , See the notes for this one .

class DataContainer:

	 Original translation ：
		pytorch in collate The packing function is to put a batch Data is stacked on a dimension , This method has the following disadvantages ：
		1、 all tensor Must have the same size 
		2、 The type restriction is relatively dead , have only tensor perhaps numpy.
	 In order to be more flexible in handling packaging tensor, Designed DataContainer, but pytorch Cannot package operation DataContainer object .
	 So we designed MMDataPareel To handle the computation DC Chinese content .

    def __init__(self,
                 data,
                 stack=False,
                 padding_value=0,
                 cpu_only=False,
                 pad_dims=2):
        self._data = data
        self._cpu_only = cpu_only
        self._stack = stack
        self._padding_value = padding_value
        assert pad_dims in [None, 1, 2, 3]
        self._pad_dims = pad_dims

4、Collate

@PIPELINES.register_module()
class Collect(object):

    def __init__(self,
                 keys,
                 meta_keys=('filename', 'ori_filename', 'ori_shape',
                            'img_shape', 'pad_shape', 'scale_factor', 'flip',
                            'flip_direction', 'img_norm_cfg')):
        self.keys = keys
        self.meta_keys = meta_keys

    def __call__(self, results):
        """Call function to collect keys in results. The keys in ``meta_keys`` will be converted to :obj:mmcv.DataContainer. Args: results (dict): Result dict contains the data to collect. Returns: dict: The result dict contains the following keys - keys in``self.keys`` - ``img_metas`` """

        data = {
    }
        img_meta = {
    }
        for key in self.meta_keys:
            img_meta[key] = results[key]
        data['img_metas'] = DC(img_meta, cpu_only=True)  #  Added a img_meta Of key.
        for key in self.keys:
            data[key] = results[key]
        return data

  Is to add a new key: ‘img_meta’.

summary

  Finally, review this picture .