AMQP protocol details

List of articles

• AMQP Protocol is introduced
• AMQP And JMS
• AMQP The core of
• • Producer( producer )
  • ConnectionFactory( Connection factory )
  • Connection( Connect )
  • Channel( channel )
  • Broker( middleware )
  • VirtualHost( Virtual host )
  • Exchange( Switch )
  • • Default switch
    • Direct connection switch
    • Fan switch
    • Theme switch
    • Head switch
    • Switch summary
  • Binding( binding )
  • Routing Key( Routing key )
  • Queue( queue )
  • Consumer( consumer )
  • Message( news )
• AMQP working process
• AMQP Message mechanism
• • Message confirmation
  • Reject the message

---

AMQP Protocol is introduced

AMQP（Advanced Message Queuing Protocol） Advanced message queue protocol , An application layer standard protocol that provides unified messaging services , Is an open standard for application layer protocols , Designed for message-oriented middleware .AMQP It's a network protocol that delivers asynchronous messages between processes .

The client and message middleware based on this protocol can deliver messages , Not by the client / Different middleware products , Restrictions on different development languages and other conditions .

AMQP The main feature is message oriented 、 queue 、 route （ Including point-to-point and release / subscribe ）、 reliability 、 Security .AMQP The behavior of the message provider and the client is mandatory , This enables different vendors to truly realize interoperability .

AMQP And JMS

JMS It is an attempt to standardize the early message oriented middleware , It's just API Level has been standardized , It's far from creating interoperability .

And JMS Different ,AMQP It's a Wire Level agreement , It describes the format of data transmitted on the network , Stream in bytes . So any tool that adheres to this data format , It creates and interprets messages , Can interoperate with other compatible tools .

AMQP The core of

Producer( producer )

Production news .

ConnectionFactory( Connection factory )

production Connection Our factory .

Connection( Connect )

Connect , Application and Broker The Internet connection TCP/IP/ Three handshakes and four waves .

AMQP A connection is usually a long connection .AMQP It's a use TCP Provide reliable delivery of application layer protocol .AMQP Use the authentication mechanism and provide TLS（SSL） Protect . When an application no longer needs to be connected to AMQP When acting , It needs to be released gracefully AMQP Connect , Not directly TCP Connection is closed .

Channel( channel )

Network channel , It's based on Connection A lightweight connection above the connection . Almost all operations are in Channel In the middle of ,Channel It's a channel for reading and writing messages , The client can set up a system for each Channel, Every Channel Represents a conversational task .

If you put Connection Compared to an optical fiber cable , that Channel The channel is compared to one of the bundles of optical fibers in an optical fiber cable . One Connection You can create any number of Channel.

Most of our business operations are in Channel Completed in this interface , Include :

• Declaration of the queue queueDeclare
• Declaration of the switch exchangeDeclare
• Binding of queues queueBind
• Release the news basicPublish
• News consumption basicConsume etc. .

Broker( middleware )

Accept client connections , Realization AMQP Physical services , Such as rabbitmq.

VirtualHost( Virtual host )

Virtual host , For logical isolation , A virtual host can have several reasons Exchange and Queue, You can't have the same name in the same virtual host Exchange.

To implement multiple isolated environments on a single agent （ user 、 User group 、 Switch 、 queue etc. ）,AMQP Provides a virtual host （virtual hosts - vhosts） The concept of . This one Web servers The concept of virtual host is very similar , This is AMQP Entities provide a completely isolated environment . When the connection is established ,AMQP Client to specify which virtual host to use .

Exchange( Switch )

Switch , Receive a message , Send messages to the bound queue according to the routing key （ No message storage capability ）.

Switches are used to send messages AMQP Entity . The switch takes a message and routes it to one or zero queues . Which routing algorithm it uses is determined by switch type and binding （Bindings） It's decided by the rules .

Switch type ：

• Direct exchange（ Direct connection switch ）
• Fanout exchange（ Sector switches ）
• Topic exchange（ Theme switch ）
• Headers exchange（ Head switch ）

Switch properties ：

• Name： Switch name
• Durability： Persistent flag , Indicates whether the switch is persistent
• Auto-delete： Delete logo , Indicates when all queues are finished using this exchange when , Whether or not to delete
• Arguments： Rely on the agent itself

Switch Status ：

• persistent （durable）
• The staging （transient）

Persistent exchange opportunities in the message broker （broker） It still exists after restart , A temporary switch doesn't （ They need to be re declared when the agent goes online again ）.

Default switch

Default switch （default exchange） In fact, it is a pre declared by the message broker , Isn't there a name （ The name is an empty string ） Of Direct connection switch （direct exchange）.

It can be considered that the default switch is a special direct connected switch .
Default switch name ： An empty string （AMQP default）
Default switch type ： Direct connection switch

Creating a Queue when , As long as the switch to be bound is not specified , Will automatically bind to the default switch , Bound route key （routing key） The name is the same as the queue name .

Direct connection switch

The direct connected switch is based on the routing key carried by the message （routing key） Post the message to the queue corresponding to the binding key . A unicast route used by a direct connect switch to process messages .

Creating a Queue when , If the bound is a direct connected switch , You don't have to specify routing key Name , Because he will have a default routing key name , Same name Queue name .

The queue of a direct connected switch usually circularly distributes tasks to multiple consumers （ We call it polling ）.

Workflow ：

1. When a queue is bound to a switch , Give the binding a binding key （Binding Key）, Assuming that R;
2. When one carries a routing key （Routing Key） by R When a message is sent to a direct connect switch , The switch routes it to the binding key R Queues .

Fan switch

A fan switch routes messages to all queues bound to it , And ignore the bound route key .

If N Queues are bound to a fan switch , When a message is sent to this sector switch , The exchange sends copies of the messages to all of these N A queue . Fan switches are generally used to handle broadcast routing of messages （broadcast routing）.

Application scenarios ：

• Broadcast news ;
• Group chat function .

Theme switch

The theme switch is based on routing key and Exchange The type of will message Send to one or more Queue in , We often use him to realize all kinds of publish/subscribe, That is, publish and subscribe .

The routing rules of direct connected switches are strictly matched , In other words Routing Key Must be with Binding Key Send the message to the Queue.
The routing rule of subject switch is a kind of fuzzy matching , Some rules can be satisfied by wildcards and then transferred .

It is agreed that ：

• binding key There can be two special characters in * And #, For fuzzy matching . among * Used to match a word ,# Used to match multiple words （ It can be zero ）
• routing key A string separated by a dot （ We call each independent string separated by a dot a word ）

• When the producer sends a message Routing Key=A.A.A When , At this time, it's only satisfying A.*.*, Will only be routed to queue1;
• When the producer sends a message Routing Key=A.B.A When , It's time to meet A.*.* and *.B.* Will be routed to queue1、queue2;
• When the producer sends a message Routing Key=A.B.C When , It's time to meet A.*.* and *.B.* and *.*.C Will be routed to queue1、queue2、queue3.

Application scenarios ：

• News updates related to categories or tags ;
• By multiple workers （workers） Completed background tasks , Each worker is responsible for certain tasks .

Head switch

The head switch does not depend on routing key And binding key To route messages , It depends on the headers Property to match .

Head switch can be regarded as another form of direct connection switch . But the routing key of the direct switch must be a string , The header property value does not have this constraint , They can even be integers or hashes （ Dictionaries ） etc. . More flexibility （ But in fact, we rarely use head switches ）.

Workflow ：

1. When binding a queue to the head switch , Multiple headers for matching are bound at the same time （header）;
2. The message will carry header, And there will be a “x-match” Parameters . When “x-match” Set to “any” when , If any value of the header is matched, the condition can be satisfied , And when “x-match” Set to “all” When , You need all the values of the header to match .

Switch summary

Switch type

Switch name

Binding Key

Routing Key

Routing rules

Default switch

Empty string , Do not modify the

The default is Queue name , Cannot be changed

Same as Binding Key name

Routing Key==Binding Key, Strictly match

Direct connection switch

Customize

The default is Queue name , Modifiable

Same as Binding Key name

Routing Key==Binding Key, Strictly match

Sector switches

Customize

nothing Binding Key

nothing Routing Key

nothing Binding Key, Automatically route to all bound switches Queue in

Theme switch

Customize

Customize

Customize

Routing Key==Binding Key, Fuzzy matching

Head switch

Customize

Customize

Customize

According to... In the sent message content headers Property to match

Binding( binding )

Exchange and Queue Virtual connection between .

BindingKey yes Exchange and Queue Binding rule description .Binding Key Specify the current Exchange Next , Like what? Routing Key Will be assigned to the currently bound Queue in .

Routing Key( Routing key )

Routing rules , Virtual machines can use it to determine how to route a particular message .

Binding Key And Routing Key The relationship between

• Binding Key Is the binding between the queue and the switch key;
• Routing Key It is a message sent by the manufacturer to the switch ;
• When Binding Key and Routing Key When it can correspond to , Put the message in the corresponding queue .

Binding Key yes Exchange and Queue Binding rule description , This description is used to analyze when Exchange When a message is received ,Exchange The received message will be with Routing Key This field ,Exchange According to this Routing Key And the current Exchange All bound Binding Key Do the matching , If the requirements are met , Just go Binding Key Bound to Queue Send a message .

In various switches Binding Key And Routing Key

• Default switch ：Binding Key by Queue name , You can't customize ;Routing Key Also for the Queue Name to successfully route to the queue
• Direct connection switch ：Binding Key by Queue name , You can customize ;Routing Key On Binding Key The same can be successfully routed to the queue
• Sector switches ： nothing Binding Key; Of course not Routing Key. Automatically route to all bound switches Queue in
• Theme switch ： Customize Binding Key; Customize Routing Key.Routing Key==Binding Key, Only when fuzzy matching is successful can it be successfully routed to the queue
• Head switch ： nothing Binding Key; Of course not Routing Key. According to... In the sent message content headers Property to match

Queue( queue )

Store the messages that will be consumed by the application .

Queue properties ：

• Name： Queue name
• Durable： After the message broker restarts , The queue still exists
• Exclusive： Only connected by one （connection） Use , And when the connection is closed, the queue is deleted
• Auto-delete： When the last consumer unsubscribes, it is deleted
• Arguments： Some message agents use it to do something similar to TTL Some of the extra features of

Queue creation ：
The queue cannot be used until it is declared . If a queue doesn't exist yet , Declaring a queue creates it . If the declared queue already exists , And the attributes are exactly the same , So this statement will not have any impact on the original queue . If the properties in the declaration are different from those of the existing queue , So an error code is 406 Channel level exceptions will be thrown .

Queue persistence ：
The persistent queue will be stored on disk , Be a message broker （broker） When restarting , It still exists . Queues that are not persisted are called staging queues （Transient queues）. Not all scenarios and cases need to persist the queue .

Persistent queues don't make messages that are routed to them persistent . If the message agent fails , Restart , Then the persistent queue will be re declared during restart , No matter what , Only persistent messages can be restored .

Consumer( consumer )

Consumer consumption news . stay AMQP in , There are two ways for consumers to obtain information to be consumed ：

• Message middleware delivers messages to consumers （push API）
• Consumers take the initiative to get information (pull API)

We need to pay attention to ： When multiple consumers listen to the same queue , Messages in the queue will only be consumed by one of the consumers （ Not every consumer will spend once ）

Message( news )

news , Data transferred between services and Applications , from Properties（ attribute ） and body（ The main body ） form .

Attribute is to decorate the message , For example, the priority of a message , Advanced features like delay , The body is the content of the message body .

Message properties ：

• Content type（ Content type ）
• Content encoding（ Content encoding ）
• Routing key（ Routing key ）
• Delivery mode (persistent or not)
• Delivery mode （ Persistence or non-persistent ）
• Message priority（ Message priority ）
• Message publishing timestamp（ The timestamp of the message ）
• Expiration period（ The message is valid for ）
• Publisher application id（ Release the app's ID）

Message body ：
AMQP In addition to properties , It also contains a payload Payload（ The data that the message actually carries ）, It has been AMQP Proxies are treated as opaque arrays of bytes .

The message broker does not check or modify the payload . Messages can contain only properties and not carry payloads . It usually uses something like JSON This serialized format data , In order to save , Protocol buffers and MessagePack Serialize structured data , To be published as a payload of messages .AMQP And its followers usually use “content-type” and “content-encoding” These two fields are used to communicate with messages to identify payloads , But it's just a matter of engagement .

Message persistence ：
Messages can be published in a persistent way ,AMQP The agent stores this message on disk . If the server restarts , The system will confirm that the persistent message received is not lost .

Simply send a message to a persistent switch or route it to a persistent queue , Does not make this message persistent ： The persistence of a message depends entirely on the persistence pattern of the message itself （persistence mode）.

When a message is persistently published , Will have a certain impact on performance .

AMQP working process

• Publisher （Publisher） Release the news （Message）, Through the switch （Exchange）.

• The switch distributes the received messages to the queue bound to the switch according to the routing rules （Queue）.

• Last AMQP The agent delivers the message to the consumer who has subscribed to the queue , Or consumers can get it by themselves .

AMQP Message mechanism

Message confirmation

Consumers occasionally fail to process messages or sometimes collapse directly . And network reasons may also cause all kinds of problems .
This gives us a problem ,AMQP When is the right time for the agent to delete the message .

AMQP Two message confirmation modes ：

• Automatic confirmation mode ： When the message middleware sends the message to the consumer, it will delete it immediately .（ Use AMQP Method ：basic.deliver or basic.get-ok)
• Explicit confirmation mode ： Wait for the consumer to send a confirmation receipt （acknowledgement） Then delete the message .（ Use AMQP Method ：basic.ack）

If a consumer hangs up without sending a confirmation receipt , that AMQP The agent will resend the message to another consumer . If there were no consumers available , The message broker will wait for the next consumer registered in this queue , Then try again to deliver .

Reject the message

When a consumer receives a message , The process is likely to succeed , It's possible to fail . Consumers can send messages to the message broker （ Message middleware ） indicate , This message is due to “ Reject the message ” The reason processing failed （ Or fail to complete at this time ）.
When a message is rejected , The consumer can tell the message broker how to handle the message —— Destroy it or put it back in the queue .

When there is only one consumer in this queue , Please make sure that messages are not allowed to loop indefinitely in the same consumer due to message rejection and re queuing .

stay AMQP in ,basic.reject Method is used to perform the operation of rejecting a message . but basic.reject There's a limitation ： You can't use it to deny multiple acknowledgements （acknowledgements） The news of . But if you use RabbitMQ, So you can use what's called negative acknowledgements（ Also called nacks） Of AMQP 0-9-1 Expand to solve this problem .