【MobileNet V3】《Searching for MobileNetV3》

ICCV-2019

---

---

1 Background and Motivation

【MobileNet】《MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications》（CVPR-2017）

【MobileNet V2】《MobileNetV2：Inverted Residuals and Linear Bottlenecks》（CVPR-2018）

deliver the next generation of high accuracy efficient neural network models to power on-device computer vision

2 Related Work

• Manually design the network
  reducing the number of parameters -> reducing the number of operations (MAdds) -> reducing the actual measured latency

• NAS
  cell level -> block level

• Quantization

• knowledge distillation

3 Advantages / Contributions

NAS + Manually design and assemble mobilenet v3 backbone, Put forward hard swish Activation function （swish Improved version ）, Put forward Lite R-ASPP Split head （R-ASPP Improved version ）, In the classification 、 object detection 、 The speed and accuracy of segmented data sets are improved

4 Method

1）Network Search

1. Platform-Aware NAS for Blockwise Search（ come from MnastNet, A little modification reward design The weight of ）

2. NetAdapt for Layerwise Search
   search per layer for the number of filters
   maximizes $\frac{△Acc}{△latency}$

2）Network Improvements

1. Redesigning Expensive Layers

   search The back of the network is heavy , optimized

   Head

   channels 32 + ReLU or swish Reduced to channels 16 + hard swish

   The tail
   

2. Nonlinearities
   

   $$swish(x)=x\cdot \sigma (x)$$

   swish activation function Although it improves the accuracy of the network , But it is not friendly to hardware deployment , Increased computing time , The author adopts the following improvements （piece-wise linear）

   $$h-swish(x)=x\frac{ReLU6(x+3)}{6}$$

   Than relu Slow

   only use h-swish at the second half of the model（we find that most of the benefits swish are realized by using them only in the deeper layers）

3. Large squeeze-and-excite
   

   v3 Compared with v2, Adopted SE modular ,SE Inside sigmoid It's also used hard form , That is to say $\frac{ReLU6(x+3)}{6}$

   

   The author put SE Module squeeze fc Fixed to block in expand Of channels 1/4（ chart 4 red √ It's about ）

   no discernible latency cost

4. MobileNetV3 Definitions

The picture is from Lightweight skeleton preferred ：MobileNetV3 Complete resolution

5 Experiments

use single-threaded large core in all our measurements

5.1 Datasets

• ImageNet
• COCO
• Cityscapes

5.2 Classification

The upper left corner is best

1）Impact of non-linearities

there 112 What you read is not particularly understood ,N The bigger it is, the more reasonable it is h-swish The more , It's slower , How fast

2）Impact of other components

5.3 Detection

mAP Out of line , ha-ha

5.4 Segmentation

R-ASPP Improve on the basis of

6 Conclusion（own） / Future work

• Pareto-optimal, Pareto is the best （ From Baidu Encyclopedia ）

  Pareto is the best （Pareto Optimality）, Also known as Pareto efficiency （Pareto efficiency）, It refers to an ideal state of resource allocation , Suppose there is an inherent group of people and distributable resources , The change from one allocation state to another , Without making anyone worse , Make at least one person better , This is Pareto improvement or Pareto optimization .
  Pareto optimal state It is impossible to have more room for Pareto improvement ; let me put it another way , Pareto improvement is the path and method to achieve Pareto optimality . Pareto optimality is fair and efficient “ Ideal kingdom ”. It was proposed by Pareto .

• MobileNet V3 = MobileNet v2 + SE + hard-swish activation + half initial layers channel & last block do global average pooling first（ come from Gai Rou is in a panic ）