Semantic segmentation ｜ Learning record （1） Semantic segmentation Preface

Tips ： come from up Lord thunderbolt Wz, I'm just taking study notes , Original video

---

Preface

The preface of semantic segmentation mainly introduces the content involved in this paper ：

• What is semantic segmentation
• Tentative learning objectives
• Common dataset formats for semantic segmentation tasks
• The specific form of the result obtained by semantic segmentation
• Common evaluation indicators for semantic segmentation
• Semantic segmentation annotation tool

One 、 What is semantic segmentation ？

Semantic segmentation is one of the common segmentation tasks , Common segmentation tasks have the following three aspects ：

• Semantic segmentation (semantic segmentation）FCN
• Instance segmentation （Instance segmentation）Mask R-CNN
• Panoramic segmentation （Panoramic segmentation） Panoptic FPN

Panoramic segmentation is not only to distinguish the background and foreground , Moreover, the background should be classified and segmented in some columns .
The difficulty of the above three segmentation tasks increases in turn .

Two 、 Learning Planning

Several semantic segmentation algorithm source code introduction

Two 、 Common dataset formats for semantic segmentation tasks

1.PASCAL VOC

PASCAL VOC What is provided in semantic segmentation is actually a PNG picture , In this PNG The file records the category of each pixel , there PNG Pictures are stored in palette format （ The original picture is a 1 Grayscale image of the channel ）, The corresponding pixel value is mapped to the corresponding color value . such as ：

• Pixels 0 The corresponding is （0,0,0） black
• Pixels 1 The corresponding is （127,0,0） Deep red
• Pixels 255 The corresponding is （224,224,129）
  This 255 It's necessary to explain ： When we calculate the loss, we will ignore that the pixel value is 255 These pixels , Because it's hard to say which category the edge of the target strictly belongs to , Including some goals that are not easy to divide , We also have 255 Fill in . such as , The figure above has a quadrilateral , It's actually the tail of an airplane , This segmentation is very difficult , We just ignore it .
  

2.MS COCO

The feature is that each target is given a polygon , And record the coordinates of each corner of the polygon .
MS COCO Data set introduction and pycocotools Easy to use

3、 ... and 、 The specific form of the result obtained by semantic segmentation

Why not directly display grayscale images , But to turn it into color ？
for instance The plane is a pixel value corresponding to 1, Person correspondence is 15, The difference between them is very big , If in the form of gray , It's hard for us to see the difference .
So we map the pixel value to the color format , also Each pixel value corresponds to the category index .

Four 、 Semantic segmentation evaluation index

Pixel Accuracy(Global Acc):$\frac{premeasuringjustindeedOfimageplainindividualCount}{totalOfimageplainindividualCount}$
$$\frac{{\Sigma }_i{n}_{ii}}{{\Sigma }_i{t}_i}$$
mean Accuracy: Average the accuracy of each category of pixels
$$\frac{1}{n_{cls}}{\Sigma }_i\frac{n_{ii}}{t_i}$$
mean IoU: Yes IoU averaging
$$\frac{1}{n_{cls}}{\Sigma }_i\frac{n_{ii}}{t_i+{\Sigma }_j{n}_{ji}-n_{ii}}$$
$\frac{YancolorheavyStackDistrictDomainOfNoodlesproduct}{totalNoodlesproduct}$

among :

• $n_{ij}$： Category i Predicted into categories j The number of pixels
• $n_{cls}$: Number of target categories （ Include background ）
• $t_i={\Sigma }_j{n}_{ij}:$ Target categories i Total number of pixels （ Real label ）

Have a deep understanding of this evaluation index

mean acc$=\frac{1}{5}\sum (clas{s}_{i}acc)$

mean IoU$=\frac{1}{5}\sum (cl{s}_{i}iou)$

5、 ... and 、 Semantic segmentation annotation tool

Traditional annotation tools , such as ：Labelme

Labelme

A semiautomatic annotation tool ： Baidu EISeg
EISeg

6、 ... and 、 Reference material

PASCAL VOC2012 Data set introduction
EISeg Segmentation and annotation software use