Understanding deep learning attention

So-called attention Mechanism , Whether in the field of computer vision or natural language processing , In fact, they are all the same , In fact, they are doing weighting , The so-called weighted , Its meaning is to pay more attention to certain aspects , Ignore sth .

For example ：

such as self-attention in , It's just a formula ,.

among It means to find the weight ,Q,K,V By input X After linear transformation, we get , It is K Dimension root of , Used to prevent the value from being too large .

And the so-called long attention mechanism , It's more than one attention Put together , Forming a so-called multiple attention mechanism .