[Intensive reading of the paper] R-CNN's Bounding box regression problem is detailed

本文就R-CNNIntensive reading of the thesis The prediction box regression of （Bounding box regression）issues are discussed in detail.

---

R-CNNThe feature vector extracted from the candidate frame,进行分类+偏移预测的并行处理.
Offset prediction is prediction box regression（Bounding box regression）问题,We need to position the generated candidate boxes、Size fine-tuning.

（图摘自b站up“同济子豪兄”的R-CNN论文解读）

We need to think about such a question：Why join this oneRegressionIt can correct the problem of inaccurate positioning？

The prediction problem of the candidate box offset（Bounding box Regression）

因为selective searchThe generated candidate boxes inevitably generate localization errors,So we can perform offset correction on the generated candidate boxes.
Bounding box regression是延续了DPM算法（同一个作者）的设计,It does this by training a linear regression model,Given a set of characteristics（CNN提取的特征）,to predict a new detection box,The offset for this new box is thisRegression预测的目标.
This offset is relative to the correct position（如Ground TruthFrame the complete frame of a target）的偏移量,The offset is calculated from a set of offset coefficients,The offset coefficient is learned.
The following is a detailed description of the paper and my thinkingBounding box Regressiondid something.

①为什么一个RegressionCan solve the problem of inaccurate positioning？

Personally, I think it is a very good idea to predict the offset of the candidate frame through modeling.When reading the paper for the first time,I've been thinking about this：Why join this oneRegressionIt can solve the problem of inaccurate positioning？
Just imagine in training,We set the positive sample to be Ground Truth框,It describes a completecat的特征;one of our candidate boxes and GTBoxes partially overlap,It describes an incomplete onecat的特征.So incompletecatBox is relative to fullcatThe frame needs an offset correction.比如下图,The red frame needs to be corrected to the blue frame position.

We assume that the feature vector of a candidate box can infer the relative position information for a certain target,举个例子：上图中的红框,The cat's tail and limbs are not completely framed,Then from the feature vector of the red box, it should be inferred that the features of the cat's tail or limbs are relatively incomplete,Then you need to expand the box to the position where the feature is located,for complete features.In this example, the red box needs to be extended to the upper right to pursue the complete tail of the box,At the same time, it also needs to expand down to pursue the complete limbs in the frame.

注意,The extension is only based on the features of the current red box1次Offset prediction,When offset to the blue frame,In this algorithm, the blue box will not be judged again whether an offset is needed.The author also pointed out1A second offset adjustment is sufficient,Multiple tweaks do not improve performance.

②Model the offset

基于上面的思考,We can convert the feature vector extracted from a candidate box into an offset,To guide the current candidate box to adjust the size and position.
The positioning of a candidate box is generally required4个参数,Commonly used are the coordinates of the center of the frame$（x,y）$and the height and width of the box$（w,h）$.那么偏移量为$\Delta x,\Delta y,\Delta w,\Delta h$.
即：$feature（4096维向量）\to \Delta x,\Delta y,\Delta w,\Delta h$
The author used relative offsets in the text,具体为：

设Ground Truth框$G$的位置信息、候选框$P$的位置信息为：
$$\begin{array}{c}G=\left(G_x,G_y,G_w,G_h\right)\\ P=\left(P_x,P_y,P_w,P_h\right)\end{array}$$
Our goal is to build$G$和$P$之间的关系.
established in the text4offset factor$t_x,t_y,t_w,t_h$,as a relative offset：
$$\begin{array}{rl}{G}_x& =P_w{t}_x+P_x\\ G_y& =P_h{t}_y+P_y\\ G_w& =P_w\exp \left(t_w\right)\\ G_h& =P_h\exp \left(t_h\right).\end{array}$$
And then at the time of forecasting,这个系数$t_x,t_y,t_w,t_h$should consist of candidate boxes$P$推断出来,Then ours should be built from the candidate box$P$extracted features$feature(P)$和$(t_x,t_y,t_w,t_h)$的关系.

③建立回归模型

我们假设$feature(P)$和$(t_x,t_y,t_w,t_h)$存在线性关系,You can set a weight$w$,This creates a regression equation：
$$({\hat{t}}_x,{\hat{t}}_y,{\hat{t}}_w,{\hat{t}}_h)=w\ast feature(P)$$
about this regression equation：

1. 向量$({\hat{t}}_x,{\hat{t}}_y,{\hat{t}}_w,{\hat{t}}_h)$is what we want to predict4offset factor,权重$w$is the parameter we want to learn.
2. 通过②Modeling of medium offsets,A pair is knownGTThe location information of boxes and candidate boxes$(G,P)$Solvable equations are calculated accordingly$(t_x,t_y,t_w,t_h)$,文中设为$t_{\star }$ .
   This way we get a lot$t_{\star }$和$feature(P)$的训练数据,weights can be fitted$w$来.
3. 论文中$feature(P)$是用CNNThe features extracted by the last pooling layer of the model are used as feature vectors,论文中用${\varphi }_5(P)$表示.
4. 论文中,用$d_x(P),d_y(P),d_w(P),d_h(P)$表示${\hat{t}}_x,{\hat{t}}_y,{\hat{t}}_w,{\hat{t}}_h$,represents the regression results availablePThe eigenvector linear representation of .

④回归模型的训练

We know for linear regression models,The loss function is the sum varianceSSE,And seek the best$w$There is an analytical solution.
对于本例,损失函数可表示为：
$${\mathbf{w}}_{\star }=\underset{{\hat{\mathbf{w}}}_{\star }}{\mathrm{argmin}}\sum _i^N{\left(t_{\star }^i-{\hat{\mathbf{w}}}_{\star }^{\mathrm{T}}{\mathbf{\varphi }}_5\left(P^i\right)\right)}^2$$
其中：

1. $i$represents each training sample,$t_{\star }$the corresponding categoryGTboxes and candidate boxes$(G,P)$The vector of obtained offset coefficients.
2. ${\varphi }_5(P)$As mentioned above, it is a candidate box$P$经CNNFeatures extracted by the last pooling layer（即4096维向量）.
3. A regression model is trained for each class,一共20个.

这里,The author added for the loss functionL2The regularization term improves model performance,防止过拟合.作者在文中指出,This regularization term is very important,他将$\lambda$设为了1000.
（在统计学中,This is also a ridge regression problem）
The final function to be optimized is ：
$${\mathbf{w}}_{\star }=\underset{{\hat{\mathbf{w}}}_{\star }}{\mathrm{argmin}}\sum _i^N{\left(t_{\star }^i-{\hat{\mathbf{w}}}_{\star }^{\mathrm{T}}{\mathbf{\varphi }}_5\left(P^i\right)\right)}^2+\lambda {\Vert {\hat{\mathbf{w}}}_{\star }\Vert }^2$$

We can use the least squares method to find the regression coefficients$w$的解析解.
训练出$w$The offset regression prediction can be performed

⑤Bounding box RegressionApplicable conditions and precautions

The regression model we built above is based on the assumption that there is a linear relationship between the eigenvectors and the offset coefficients.The linear problem is a simple one,has its own scope of application.
并且,Our regression problem solves the candidate box orientationGT的微调.If too many features are lost in the candidate box,It is also difficult to adjust.
This determines in actual training,The candidate box cannot be separatedGTbox too far.
作者指出了IoU必须大于0.6The candidate boxes can be used to train the regressor.

此外,在①中已提到,Only for candidate boxes1次回归,当调整完毕后,No more regressions will be performed on the adjusted box.Because the authors found that multiple fine-tuning did not improve the model accuracy.

最后如下图所示,在推理阶段,This returns the head with the previous oneSVMHeads are parallel,同时计算,The classification sum of the final detection frame is obtained(微调后的)定位.
（图摘自b站up“同济子豪兄”的R-CNN论文解读）