Detr introduction

DETR yes facebook Published in ECCV2020 Use Transformers A framework for end-to-end target detection .

DETR Just use CNN Extraction of image features , And then use it alone Transformer You can predict the target bounding box and classification . It does not require non maximum suppression , Don't need to, Anchor Mechanism .

Above, DETR The network architecture of ,DETR Use CNN Extraction of image features , And then use it alone Transformer Get the predicted target bounding box , Bounding box and ground truth As a geometric prediction problem . It's a binary match (bipartite matching), There is no matching object homing no object This kind of .

The above figure is a more detailed description DETR Network structure , The image passes by CNN Get the feature , Plus the location code (poositioonal encoding), Then flatten and feed into transformer encoder,encoder The output of is sent to transformer decoder, stay decoder There is also object queries The input of ,decoder The output of is sent to the prediction head (prediction heads), There is a feedforward neural network in the prediction head FFN Predict object categories and bounding boxes .

Above, DETR in Transformer Specific architecture , It has Encoder and Decoder Two parts ,Encoder The input is CNN Extracted image features plus position coding , Send it to the multi head self attention module , Then it is sent to the feedforward neural network module . In this way Encoder There can be multiple layers , Then send it to Decoder,Decoder Yes Object queries, Is a learnable location embedded as input , After the multi head self attention module , after Encoder and Decoder Multi head mutual attention module , Then it is sent to the feedforward neural network for processing .Decoder Layers can also stack multiple , Finally, it is sent to the feedforward neural network FFN Carry out object category prediction and boundary box prediction .