[paper reading] tganet: text guided attention for improved polyp segmentation

[ Address of thesis ][ Code ][MICCAI 22]

Abstract

Colonoscopy is a gold standard , But highly dependent on the operator . Automatic polyp segmentation can minimize the missed diagnosis rate , And in the early stage of timely treatment of colon cancer . Even if there are deep learning methods developed for this task , But the change of polyp size will affect the training of the model , Thus, it is limited to the size attribute of most samples in the training data set , May provide suboptimal results for polyps of different sizes . In this work , In the training process, we use the features related to the size and the number of polyps in the form of text attention . We introduce an auxiliary classification task , Weighting text-based embedding , Enable the network to learn additional feature representations , It can obviously adapt to polyps of different sizes , And can adapt to the situation with multiple polyps . Our experimental results show that , Compared with the most advanced segmentation methods , These additional text embeddings improve the overall performance of the model . We explored four different datasets , And the improvement suggestions for specific dimensions are provided . We propose a text guided attention network (TGANet) Polyps of different sizes in different data sets can be well summarized .

I. Network Architecture

The polyp segmentation network in this paper is a complex class . say concretely , The following modules are introduced ：

Next, we analyze them separately .

II. Encoder

Use pre trained resnet50 As backbone( Not common in polyp segmentation tasks res2net). It should be noted that , This article seems to remove resnet the last one encoder block; Besides , The encoder Two additional tasks were undertaken . One is a dichotomy , Determine the number of polyps in the image ( Single / Multiple ); The other is three categories , Determine the size of polyps in the image ( Small , in , Big ).

however , This article does not clarify where the labels needed to supervise the classification task are obtained , The author speculates that manual annotation may be required , Interested readers can study their open source code for analysis .

III. Feature Enhancement Module

Yes encoder The obtained features are further enhanced . This module is very common , Basically, every partition should be designed with a similar attention Things that deal with features ：

IV. Label Attention

This module is the core of this paper . say concretely , Is this part of the text ：

In fact, the structure is quite complicated , So we only introduce ideas here . First, we introduce a semantic category (Semantic Class) The concept of , namely , If you perform a classification task on polyps , Then the images can be roughly divided into five categories ： Single polyp , Polyp , Small polyp , Middle polyp , Large polyps . This category can be obtained from the above classification . Now? , In fact, there is an implicit category imbalance problem . If a model performs well on large polyps , But it doesn't perform well on small polyps , There may be many reasons , For example, the characteristics of small polyps are difficult to learn , Or the sample size of small polyps is very small … So at this point , We can use it attention thought , Right " Lack of study at present " Focus on the characteristics of , So as to improve the overall performance . The semantic category statistics of image samples obtained before can provide guidance for this process , So called "Text-Guided Attention".

V. Multi-Scale Feature Aggregation

It is also a classic multi-level feature fusion in segmentation tasks , I won't elaborate here ：

VI. Experiment

It should be noted that , The data set and " Mainstream polyp segmentation papers " Is not the same , by Kvasir-SEG,CVCClinicDB,BKAI,Kvasir-Sessile. Be careful Kvasir-Sessile It's actually Kvasir-SEG A subset of , So actually only three data sets are used . Besides , The comparison method is also the older method , As an article 2022 Year paper , No comparison 21 Year of sota Polyp segmentation method .