Auto SEG loss: automatic loss function design

Click on the above “ Xiaobai studies vision ”, Optional plus " Star standard " or “ Roof placement ”

 Heavy dry goods , First time delivery 

The author 丨 [email protected] You know

Source https://zhuanlan.zhihu.com/p/266102401

Edit the market platform

Reading guide

 

What is proposed in this paper Auto Seg-Loss Is designed to reduce the cost for a given index （ For example, the edge part IoU perhaps F-score） Trial and error costs in designing and adjusting loss functions , And further to the design of automatic loss function .

Some time ago , One piece of news is quite popular ：

The National Swimming Championship has caused controversy , Fuyuanhui and other five top players in the preliminary round missed the final due to their low scores in the physical fitness test

Can the physical level reflect the competitive level ？ For the average person , There is a positive correlation between physical fitness level and individual competition ability . However , What high-level athletes need is specific training for specific events , A higher level of physical fitness does not mean a better performance , For example, for some projects （ Such as long-distance running ） Come on , The upper limbs are too strong but a burden . therefore , Many high-level athletes even break the Asian record in special events , In the face of physical fitness test, I also failed . Think in reverse , If an athlete only takes the physical fitness test as his training goal everyday , The final long run 、 sprint 、 Pull in 、 Squatting and other projects are perfect , Then it is likely that he can achieve far more than ordinary people in some special projects by virtue of his physical quality , But not enough to be a top athlete .

Why say this news ？ actually , If we think of our neural network model as an athlete , A lot of time , The evaluation index of the special competition faced by this athlete （ For example, in semantic segmentation mIoU） And its training goals （ For example, common Cross Entropy Loss） It's not exactly the same . Even though CE Loss You can train a good model most of the time , But this is by virtue of being strong enough “ Physical fitness ” The results obtained , Lack of targeted optimization for special projects .

that , Whether special evaluation indicators can be used , such as mIoU To guide the training ？ Unfortunately , Most of the evaluation indicators are non differentiable , It is not possible to train directly through back propagation . Of course , This does not stop researchers . Many studies try to approximate the evaluation index by means of differentiability , Get a proxy loss function to guide the training （ such as The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks and Learning Surrogates via Deep Embedding）. however , These differentiable approximations do not necessarily make the model achieve good training results —— Although the training objectives are consistent , But the coach's training program （ gradient ） Quality also determines the athletes （ Model ） Training level . therefore , The design of proxy loss function needs expertise And higher trial and error costs ; even so , Many of the proxy loss functions designed are not enough to be independent , Need and CE Loss Joint training can achieve good results .

Auto Seg-Loss hope （ On the task of semantic segmentation ） Automate this process . Simply speaking , We found that the mainstream semantic segmentation indicators （ Basically by TP/TN/FP/FN form ） Can be written as a differentiable operation （ For example, add 、 ride ）、 quantitative （one-hot） and logical operation （ And AND、 or OR） In the form of . because logical The operation is actually defined only in    On , We use a parameterized surface logical Operation for interpolation , Make it in    There is a differentiable definition on , And use softmax replace one-hot quantitative , Thus, a differentiable version of the proxy loss function is obtained . Next , We use reinforcement learning algorithms （PPO2） Search the shape of this surface , So that the agent loss function can guide the training well . chart 1 Is the overall framework of our algorithm .

chart 1 The overall framework

At the time of implementation , We tried piecewise Bezier curve and Piecewise linear curve Two ways of parameterization . We proposed Truth table constraints and Monotonicity constraint As a priori , To limit the shape of the parametric surface . Experiments show that these two parameterization methods can get good proxy loss function , At the same time, these two constraints effectively improve the efficiency and effect of search .

We are PASCAL VOC and Cityscapes We did experiments on . Relative to manually designed proxy loss functions or Cross Entropy Improvement , The searched agent loss function can reach the main semantic segmentation index on par Or higher , In particular, the improvement of edge related indicators is relatively large . There are two points worth mentioning ：

1. May benefit from the design of the search space , Our search efficiency is relatively high , stay VOC Upper use DeepLab V3+, about mIoU Your search only needs 8 In hours or so （8 card V100, In fact, half of the search results have basically converged , The time is roughly equivalent to the same data set two Normal training ）;

2. The agent loss function we searched can be well migrated to other Model architecture and Data sets , So you can use it many times with just one search （ More Than This , We found that mIoU The searched parameters also apply to FWIoU and Boundary IoU And other similar indicators ）. The following two tables are the experimental results of two parameterized forms we tried .

surface 1 piecewise bezier Parameterized experimental results

surface 2 piecewise linear Parameterized experimental results

For edge related indicators , We found that , Using edge index alone to guide the training will make the model only focus on the segmentation results of edges . chart 2 It shows this phenomenon . By comparing the edge index with the overall index （ Such as mIoU） Train in groups , The model can not only ensure the reasonable overall performance, but also improve the edge segmentation effect . Another interesting finding is , Used to evaluate edge accuracy Boundary F1 score When the allowable error is not 0 when , Used to guide training may cause jagged edges , This is actually a reflection of this indicator hack. We discussed this problem in the appendix .

chart 2 Using edge indicators alone and combining overall indicators

We hope Auto Seg-Loss It can reduce the risk of , For a given metric （ For example, the edge part IoU perhaps F-score） Trial and error costs in designing and adjusting loss functions , Take a step forward to the design of automatic loss function . Our article is already in arxiv Hang it up , The code will also be open source and integrated into some open source Segmentation frame , I look forward to your trial and discussion ！

Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation
https://arxiv.org/abs/2010.07930

The good news ！

Xiaobai learns visual knowledge about the planet

Open to the outside world

 download 1：OpenCV-Contrib Chinese version of extension module 

 stay 「 Xiaobai studies vision 」 Official account back office reply ： Extension module Chinese course , You can download the first copy of the whole network OpenCV Extension module tutorial Chinese version , Cover expansion module installation 、SFM Algorithm 、 Stereo vision 、 Target tracking 、 Biological vision 、 Super resolution processing and other more than 20 chapters .


 download 2：Python Visual combat project 52 speak 
 stay 「 Xiaobai studies vision 」 Official account back office reply ：Python Visual combat project , You can download, including image segmentation 、 Mask detection 、 Lane line detection 、 Vehicle count 、 Add Eyeliner 、 License plate recognition 、 Character recognition 、 Emotional tests 、 Text content extraction 、 Face recognition, etc 31 A visual combat project , Help fast school computer vision .


 download 3：OpenCV Actual project 20 speak 
 stay 「 Xiaobai studies vision 」 Official account back office reply ：OpenCV Actual project 20 speak , You can download the 20 Based on OpenCV Realization 20 A real project , Realization OpenCV Learn advanced .


 Communication group 

 Welcome to join the official account reader group to communicate with your colleagues , There are SLAM、 3 d visual 、 sensor 、 Autopilot 、 Computational photography 、 testing 、 Division 、 distinguish 、 Medical imaging 、GAN、 Wechat groups such as algorithm competition （ It will be subdivided gradually in the future ）, Please scan the following micro signal clustering , remarks ：” nickname + School / company + Research direction “, for example ：” Zhang San  +  Shanghai Jiaotong University  +  Vision SLAM“. Please note... According to the format , Otherwise, it will not pass . After successful addition, they will be invited to relevant wechat groups according to the research direction . Please do not send ads in the group , Or you'll be invited out , Thanks for your understanding ~