Maxiouassigner of mmdet line by line interpretation

Preface

  This is MMdet Read Chapter 2 line by line , Code address ：mmdet/bbox/assigners/max_iou_assigner.py. Because of the source code assigner There are many input parameters and they are not easy to understand , Therefore, this paper analyzes the function of each parameter step by step from simple to difficult . The historical article is as follows ：
 AnchorGenerator Reading

1、 close match_low_quality Parameters

  We first constructed a match_low_quality Of assigner, The meaning of this parameter will be explained later . This section mainly analyzes pos_iou_thr and neg_iou_thr Two parameters . in addition , I also artificially constructed four prediction boxes and four GT.

import torch
from mmdet.core.bbox import build_assigner
    # maxIOU Module debugging 
    config = dict(
        #  Maximum  IoU  Principle allocator 
        type='MaxIoUAssigner',
        #  Positive sample threshold 
        pos_iou_thr=0.5,
        #  Negative sample threshold 
        neg_iou_thr=0.4,
        #  Lower limit of positive sample threshold 
        min_pos_iou=0.,     #  This threshold only works when matching low quality is turned on 
        match_low_quality = False,  # For the first FALSE
        #  Ignore  bboes  The threshold of ,-1 Don't ignore 
        ignore_iof_thr=-1)
        
    assigner = build_assigner(config) #  Build an allocator 
    bboxes = torch.Tensor([[0, 0, 10, 10], [10, 10, 20, 20],
							    [3, 3, 6, 6],[2, 2, 3, 3]]) 
    gt_bboxes = torch.Tensor([[0, 0, 10, 9], [10,10,19,19], 
    							[10,10,15,15],[3,3,4,4]])
    res = assigner.assign(bboxes, gt_bboxes)
    print(res.gt_inds)  # tensor([1, 2, 0, 0])

  First of all, from the running results ,bboxes and gt_bboxes The matching result of is [1,2,0,0]. It means ： first bbox Match the first gt, the second bbox Match the second gt, Third and fourth bbox Match background （0 Background representation ）. Next , We analyze the source code ：
 

	overlaps = self.iou_calculator(gt_bboxes, bboxes)
    #  Create a -1 Of tensor, Used to save matching results .-1 At present bbox To ignore samples 
    assigned_gt_inds = overlaps.new_full((num_bboxes, ),
                                         -1,
                                         dtype=torch.long)  
    #  From the column direction, we get each bbox With which gtbox Of iou Maximum 
    max_overlaps, argmax_overlaps = overlaps.max(dim=0) 
    #  take iou Below threshold [0,0.4] Of is set to 0, That is, at the threshold of the range bbox In the background .
	if isinstance(self.neg_iou_thr, float):
		assigned_gt_inds[(max_overlaps >= 0)
	                  & (max_overlaps < self.neg_iou_thr)] = 0 
    # if iou be in [0.5,1] Scope , take bbox Set as ind+1, Express bbox And the gt matching 
   pos_inds = max_overlaps >= self.pos_iou_thr                      
   assigned_gt_inds[pos_inds] = argmax_overlaps[pos_inds] + 1         
                                         

  In fact, only the above lines of code are actually executed , Note I have almost already noted , Let me draw a picture here to illustrate , Readers can combine notes and figures to understand by themselves . First calculate each gt And all bbox Of iou, Get one 4*4 Of iou matrix overlaps.

  After that dim=0 On the implementation max operation , Got max_overlaps=[0.9,0.81,0.11,0.01],argmax_overlaps=[0,1,3,0]. That is, the colored font in the figure . after , Green font due to <0.4 So it is divided into background ; The red two bbox>0.5 Therefore, it is divided into positive samples , But the final result vector assigned_gt_inds Need to store each bbox And the number gt matching , With bbox1 For example , Is and index=0 Of gt matching , but 0 Has been occupied by the background , So we need +1. namely bbox1 And the first gt1 matching . So the final result vector is the result of code execution [1,2,0,0].

2、 Turn on match_low_quality Parameters

  In the previous section, we closed this parameter , As a result , There are four in all gt, But in the end there were only two gt The match is successful . There are two gt No match . From the perspective of ensuring recall rate , This is certainly unreasonable . therefore ,assigner Set up again match_low_quality Parameters , That is, try to make all gt There is one. anchor. This time we turn on this parameter to learn its function , Need extra attention , This parameter requires and min_pos_iou Parameters are used together . Empathy , Let's look at the implementation effect first ：

import torch
from mmdet.core.bbox import build_assigner
    # maxIOU Module debugging 
    config = dict(
        #  Maximum  IoU  Principle allocator 
        type='MaxIoUAssigner',
        #  Positive sample threshold 
        pos_iou_thr=0.5,
        #  Negative sample threshold 
        neg_iou_thr=0.4,
        #  Lower limit of positive sample threshold 
        min_pos_iou=0.,     #  This threshold only works when matching low quality is turned on 
        match_low_quality = True,  #  Turn on this parameter 
        #  Ignore  bboes  The threshold of ,-1 Don't ignore 
        ignore_iof_thr=-1)
        
    assigner = build_assigner(config) #  Build an allocator 
    bboxes = torch.Tensor([[0, 0, 10, 10], [10, 10, 20, 20],
							    [3, 3, 6, 6],[2, 2, 3, 3]]) 
    gt_bboxes = torch.Tensor([[0, 0, 10, 9], [10,10,19,19], 
    							[10,10,15,15],[3,3,4,4]])
    res = assigner.assign(bboxes, gt_bboxes)
    print(res.gt_inds)  # tensor([1, 3,4,0])

  After opening, the execution result becomes [1,3,4,0], That is, the second bbox And the third gt matching , Third bbox And the fourth one , It's confusing . Let's analyze the code ：

	''' overlaps = self.iou_calculator(gt_bboxes, bboxes) #  Create a -1 Of tensor, Used to save matching results .-1 At present bbox To ignore samples  assigned_gt_inds = overlaps.new_full((num_bboxes, ), -1, dtype=torch.long) #  From the column direction, we get each bbox With which gtbox Of iou Maximum  max_overlaps, argmax_overlaps = overlaps.max(dim=0) #  take iou Below threshold [0,0.4] Of is set to 0, That is, at the threshold of the range bbox In the background . if isinstance(self.neg_iou_thr, float): assigned_gt_inds[(max_overlaps >= 0) & (max_overlaps < self.neg_iou_thr)] = 0 # if iou be in [0.5,1] Scope , take bbox Set as ind+1, Express bbox And the gt matching  pos_inds = max_overlaps >= self.pos_iou_thr assigned_gt_inds[pos_inds] = argmax_overlaps[pos_inds] + 1 '''     
   # For each gt With which bbox Of iou Maximum .
   gt_max_overlaps, gt_argmax_overlaps = overlaps.max(dim=1)   
   if self.match_low_quality:
   	  #  Through each gt
      for i in range(num_gts):
          #  If the threshold exceeds 0.
          if gt_max_overlaps[i] >= self.min_pos_iou:
              #  Will all gt Are matched 
              if self.gt_max_assign_all:
                  #  Find and gt Maximum iou equal overlaps The location of 
                  max_iou_inds = overlaps[i, :]==gt_max_overlaps[i]
                  #  Empathy +1
                  assigned_gt_inds[max_iou_inds] = i + 1                                

  The green code above is the effect of not opening the parameter , That is, after executing the annotation code, the matching result is [1,2,0,0]. The uncommented part is analyzed below ：

  The first red font is gt_max_overlaps. each gt With which bbox Of iou Maximum . And then I'm going to go through each one gt, If the maximum iou>min_pos_iou, Then the bbox and gt Match . so bbox1 Obviously, it is also related to gt1 matching ,bbox2 And also gt2 matching , But traverse to the third gt3 When , Because it is still and bbox2 Of iou The highest , Therefore, the bbox2 The match has been changed gt3, hold gt2 It's covered . That at this time assigned_gt_inds from [1,2,0,0] Turned into [1,3,0,0]. namely Bbox2 Matching gives a low quality gt. after , Traverse gt4 When , Match it to bbox3, The final matching result is obtained assigned_gt_inds = [1,3,4,0]. In terms of the final effect , It does guarantee that all gt On the match , But it is possible to match the back to the front iou High matching is covered , Result in low quality matching .

summary

  In one stage algorithm or RPN Stage , Low quality matching can be enabled , Ensure maximum recall rate . But in the second stage RoI Head Ensure accuracy in , You won't start low-quality matching .