当前位置：网站首页>What are the common message queues?

⒈ Multi language and protocol writing client . Language : Java,C,C++,C#,Ruby,Perl,Python,PHP. Application Protocol ： OpenWire,Stomp REST,WS Notification,XMPP,AMQP

⒉ Fully support JMS1.1 and J2EE 1.4 standard （ Persistence ,XA news , Business )

⒊ Yes Spring Support for ,ActiveMQ Can be easily embedded to use Spring In the system of , And it also supports Spring2.0 Characteristics of

⒋ Through the common J2EE The server （ Such as Geronimo,JBoss 4,GlassFish,WebLogic) Test of , Among them through JCA 1.5 resource adaptors Configuration of , It can make ActiveMQ Can be automatically deployed to any compatible J2EE 1.4 On the business server

⒌ Support for multiple delivery protocols ：in-VM,TCP,SSL,NIO,UDP,JGroups,JXTA

⒍ Supported by JDBC and journal Provides high-speed message persistence

⒎ High performance cluster is guaranteed by design , client - The server , Point to point

⒏ Support Ajax

⒐ Support and Axis Integration of

⒑ It's easy to call inline JMS provider, To test

2、RabbitMQ

RabbitMQ It's a popular open source message queuing system , use erlang Language development .RabbitMQ yes AMQP（ Advanced message queue protocol ） Standard implementation of . Support multiple clients such as ：Python、Ruby、.NET、Java、JMS、C、PHP、ActionScript、XMPP、STOMP etc. , Support AJAX, Persistence . Used to store forward messages in a distributed system , In the ease of use 、 Extensibility 、 Things like high availability are doing well

Several important concepts ：

Broker： Simply put, it's a message queuing server entity .
Exchange： Message exchange , It specifies what rules the message follows , Which queue to route to .
Queue： Message queue carrier , Each message is put into one or more queues .
Binding： binding , What it does is it takes exchange and queue Bind to the routing rules .
Routing Key： Routing keyword ,exchange Message delivery based on this keyword .
vhost： Virtual host , One broker Can open more than one vhost, Separate permissions for different users .
producer： Message producer , It's the program for delivering messages .
consumer： Message consumer , It's the program that receives the message .
channel： News channel , In every connection on the client side , Multiple can be created channel, Every channel Represents a conversational task .

Usage of message queues , as follows ：

（1） The client connects to the message queue server , Open one channel.

（2） The client declares one exchange, And set the relevant properties .

（3） The client declares one queue, And set the relevant properties .

（4） Client side usage routing key, stay exchange and queue Between the establishment of a good binding relationship .

（5） The client delivers the message to exchange.

exchange After receiving the message , According to the news key And what has been set up binding, Routing messages , Post messages to one or more queues .

3、ZeroMQ

Known as the fastest message queue in history , It's actually similar to Socket A series of interfaces , He followed Socket Is the difference between the ： ordinary socket It's end-to-end （1:1 The relationship between ）, and ZMQ But you can N：M The relationship between , People are right. BSD Socket is known more about point-to-point connection , Point to point connections need to be explicitly established 、 Destroy connection 、 Selection protocol （TCP/UDP） And handling errors , and ZMQ Shielding these details , Make your network programming easier .ZMQ be used for node And node Communication between ,node It can be a host or a process .

To quote the official saying ： “ZMQ( following ZeroMQ abbreviation ZMQ) It's a simple and easy to use transport layer , A frame like one socket library, He made Socket Programming is easier 、 Simplicity and higher performance . It's a message processing queue library , Can be in multiple threads 、 Elastic scaling between the kernel and the mainframe .ZMQ Our clear goal is “ Become part of the standard network protocol stack , After entering Linux kernel ”. We haven't seen their success yet . however , It is undoubtedly very promising 、 And what people need more “ Tradition ”BSD One above the socket Layer encapsulation .ZMQ Make writing high-performance web applications extremely simple and fun .”

Characteristic is ：

High performance , non-persistent ;
Cross platform ： Support Linux、Windows、OS X etc. .
Multilingual support ; C、C++、Java、.NET、Python etc. 30 Multiple development languages .
It can be deployed separately or integrated into the application ;
Can be used as Socket The communication library uses .

And RabbitMQ comparison ,ZMQ It's not like a traditional message queuing server , in fact , It's not a server at all , More like an underlying network communication library , stay Socket API There is a layer of encapsulation on it , Will network communication 、 Process communication and thread communication are abstracted as unified API Interface . Support “Request-Reply “,”Publisher-Subscriber“,”Parallel Pipeline” Three basic models and extended models .

ZeroMQ High performance design points ：

1、 Lockless queue model

For cross thread interactions （ The client and session） Data exchange channel between pipe, The queue algorithm without lock is adopted CAS; stay pipe Asynchronous events are registered at both ends , Reading or writing messages to pipe When the , Will automatically trigger read and write events .

2、 Algorithms for batch processing

For traditional message processing , Every message is sent and received , All need system calls , So for a lot of news , The overhead of the system is relatively large ,zeroMQ For bulk messages , We optimized the adaptability , Can receive and send messages in batches .

3、 Thread binding under multi-core , need not CPU Switch

Different from the traditional multithreading mode , Semaphore or critical area , zeroMQ Make full use of the advantages of multi-core , Each core binding runs a worker thread , Avoid multithreading between CPU Switching overhead .

4、Kafka

Kafka Is a high throughput distributed publish-and-subscribe messaging system , It can handle all action flow data in a consumer scale website . This movement （ Web browsing , Search and other user actions ） It's a key factor in many social functions on the modern network . These data are usually solved by processing log and log aggregation due to throughput requirements . For things like Hadoop The same log data and offline analysis system , But it also requires the limitation of real-time processing , This is a viable solution .Kafka The aim is to pass through Hadoop Parallel loading mechanism to unify online and offline message processing , It is also to provide real-time consumption through cluster machines .

Kafka Is a high throughput distributed publish-and-subscribe messaging system , Has the following characteristics ：

adopt O(1) The disk data structure provides persistence of the message , This structure is for even Numbers TB The message store can also maintain stable performance for a long time .（ Write data by appending files , Overdue data is deleted regularly ）
High throughput ： Even very ordinary hardware Kafka It can also support millions of messages per second .
Supported by Kafka Servers and consumers cluster to partition messages .
Support Hadoop Parallel data loading .

Kafka Relevant concepts ：

Broker：Kafka A cluster contains one or more servers , This kind of server is called broker

Topic： Publish each to Kafka There is a category for messages in the cluster , This category is called Topic.（ Physically different Topic The messages are stored separately , Logically, a Topic Although the message is stored in one or more broker But the user only needs to specify the Topic You can produce or consume data without having to care where it's stored ）

Partition：Parition It's a physical concept , Every Topic Contains one or more Partition.

Producer： Be responsible for releasing information to Kafka broker

Consumer： Message consumer , towards Kafka broker Clients reading messages .

Consumer Group： Every Consumer Belong to a particular Consumer Group（ For each Consumer Appoint group name, If not specified group name It belongs to the default group）.

Generally used in big data log processing or on real-time （ Small delay ）, reliability （ A small amount of lost data ） Use in a slightly lower level scenario .

Two 、 How to select a message queue

As a programmer , I'm sure you have “ There is no silver bullet ” The idea that , The silver bullet here means that it can easily kill a werewolf 、 Bullets made of silver , What does that mean ？ My understanding of this sentence is that , In software engineering , There is no such thing as “ Silver bullet ” This can solve all the problems of design 、 Architecture or software , Every software system , It's all unique , You can't solve all the problems in one way .

On the problem of technology selection of message queue , The same is true . There is no saying that , Which message queue is “ first-class ”. Common message queues , Every product has its own advantages and disadvantages , You need to build on the current system , Choose the product that suits you best .

1、 Basic criteria for selecting message queuing products

Although these message queuing products have their own advantages and disadvantages in terms of functions and features , But we have to have a minimum standard when we choose , Ensure that the selected products are at least qualified .

First , It has to be an open source product , This is very important . Open source means , If one day the message queue you are using encounters a Bug, You have at least a chance to fix or circumvent this quickly by modifying the source code Bug, Solve the burning problem of your system , Instead of waiting for the next version that developers may not release at any time .

secondly , This product must be popular in recent years and have a certain degree of community activity . The advantage of fashion is , As long as your use scenario is not too popular , You meet Bug The probability is very low , Because most of the Bug, Others have long met and fixed . Some of the problems you encounter in using , It is also easier to search for similar problems on the Internet , And then quickly find a solution .

Another advantage is , Popular products and surrounding ecosystems will have a better integration and compatibility , such as ,Kafka and Flink It has better compatibility ,Flink Built in Kafka Of Data Source, Use Kafka It's easy to be Flink Data source development flow computing applications , If you use a relatively niche message queuing product , In the flow calculation , You have to develop one yourself Flink Of Data Source.

Last , As a passing message queuing product , Several features that must be possessed include ：

Reliable delivery of news ： Make sure you don't lose the message ;
Cluster： Support clusters , Ensure that services are not unavailable due to a node outage , Of course, we can't lose the news ;
performance ： Good enough performance , It can meet the performance requirements of most scenes .

2、 Alternative message queuing products

1）RabbitMQ

First , Let's talk about the old news queue RabbitMQ, It's commonly known as rabbit MQ.RabbitMQ It's using a relatively niche programming language ：Erlang language-written , It was first designed for reliable communication between systems in the telecommunications industry , There are also a few who support AMQP One of the message queues of the protocol .

RabbitMQ Just like the rabbit in its name ： Lightweight 、 Fast , its Slogan, That's the slogan , It is also clear that RabbitMQ Characteristics ：Messaging that just works,“ Out of the box message queuing ”. in other words ,RabbitMQ It's a fairly lightweight message queue , Very easy to deploy and use .

in addition RabbitMQ Also known as the world's most widely used open source message queue , Is it true that the utilization rate is the highest in the world , We have no way to count , But at least “ Among the most popular news ”, There is no problem with that .

RabbitMQ A more distinctive feature is to support very flexible routing configuration , Unlike other message queues , It's in the producer （Producer） And queue （Queue） There's an extra Exchange modular , You can understand it as a switch .

This Exchange The function of the module is very similar to that of the switch , According to the configured routing rules, the messages sent by the producers are distributed to different queues . Routing rules are also very flexible , You can even implement routing rules yourself . Based on this Exchange, There are many ways to play , If you just need this feature ,RabbitMQ It's a good choice .

RabbitMQ The programming language supported by our client is probably the most in all message queues , If your system is developed in some cool language , Then you can probably find the corresponding RabbitMQ client .

shortcoming ：

The first question is ,RabbitMQ Support for message heaps is not good , In its design philosophy , Message queuing is a pipeline , A large backlog of messages is an abnormal situation , Try to avoid . When there's a huge backlog of information , It can lead to RabbitMQ Performance of .

The second question is ,RabbitMQ The performance of is the worst of the message queues we've described , According to the official test data, combined with our daily experience , Depending on the hardware configuration , It can process tens of thousands to hundreds of thousands of messages per second . Actually , This performance is enough to support most application scenarios , however , If your application requires very high performance of message queuing , Then don't choose RabbitMQ.

The last question is RabbitMQ The programming language used Erlang, This programming language is not just a very niche language , More troubling , The learning curve of this language is very steep . Most popular programming languages , such as Java、C/C++、Python and JavaScript, Although grammar 、 There are many different features , But their basic architecture is the same , You only know one language , It's also easy to learn other languages , In a short time, even if you can't master it , But at least “ Will use ” The level of .

2） RocketMQ

RocketMQ It's Alibaba 2012 Open source message queuing products in , Later donated to Apache Software foundation ,2017 Graduated from a formal , Become Apache Top projects . Ali also uses it internally RocketMQ As the message queue supporting its business , I've been through it many times “ A double tenth ” test , Its performance 、 Stability and reliability are both trustworthy . As an excellent domestic message queue , In recent years, it is more and more used by many large factories in China .

I'm summing up RocketMQ The characteristics of , It's hard to find out RocketMQ There's something particularly impressive about it , And it's hard to find out what's wrong with it .

RocketMQ Just like a good student with excellent character and learning , It has good performance , Stability and reliability , With a modern message queue should have almost all the functions and features , And it's still growing .

RocketMQ There is a very active Chinese community , You can find answers to most of the questions in Chinese , It may be a reason why you choose it . in addition ,RocketMQ Use Java Language development , Most of its contributors are Chinese , The source code is relatively easy to read , It's easy for you to RocketMQ Expand or redevelop .

RocketMQ The response delay of online service has been optimized a lot , In most cases, millisecond response can be achieved , If your application is concerned about response delay , You should choose to use RocketMQ.

RocketMQ Performance ratio of RabbitMQ One order of magnitude higher , It can handle hundreds of thousands of messages per second .

shortcoming ：

RocketMQ One of my weaknesses is , As a domestic message queue , Compared with the more popular similar products abroad , It's not that popular internationally , The degree of integration and compatibility with the surrounding ecosystem is slightly inferior .

3）Kafka

Kafka The first is the LinkedIn Development , So far Apache Top projects .Kafka It was originally designed to handle massive logs .

In earlier versions , For ultimate performance , A lot of sacrifice in design , For example, the reliability of messages is not guaranteed , There may be a loss of information , It doesn't support clustering either , The function is also relatively simple , These sacrifices are acceptable for the specific scenario of processing massive logs . In this period Kafka It can't even be called a qualified message queue .

however , Please note that , The point is usually at the back . The following years Kafka We have gradually supplemented these shortcomings , Many of the comparative articles you found on the Internet still say Kafka unreliable , In fact, this kind of saying has passed for a long time . Present Kafka It has developed into a very mature message queuing product , No matter in data reliability 、 Stability and functional features can meet the needs of most scenarios .

Kafka Compatibility with surrounding ecosystems is one of the best , Especially in big data and stream computing , Almost all related open source software systems will give priority to support Kafka.

Kafka Use Scala and Java Language development , A lot of ideas of batch and asynchrony are used in the design , This design makes Kafka Can achieve ultra-high performance .Kafka Performance of , Especially the performance of asynchronous transceiver , Is the best of the three , But with RocketMQ There is no difference in magnitude , It can process hundreds of thousands of messages per second .

I used to use a better configured server to Kafka Carry out over voltage measurement , There are enough clients to send asynchronously , And turn on compression ,Kafka The ultimate processing power of can exceed per second 2000 Ten thousand messages .

shortcoming :

however Kafka The problem with this asynchronous batch design is , Its response time delay of synchronous sending and receiving messages is relatively high , Because when the client sends a message ,Kafka It will not be sent out immediately , But to wait for a while to save a batch and then send , In its Broker in , It's used in many places “ Save a wave and deal with it together ” The design of the . When your business scenario , When there are not so many messages per second ,Kafka The delay will be higher . therefore ,Kafka Not suitable for online business scenarios .

3、 Second tier message queues

In addition to the three message queues described above , There are also a few second tier products , My personal view is , The reason these products are not so popular , There are more or less obvious shortcomings , It is not recommended to use . Here it is , Let me give you a brief introduction , Pure when you enrich the breadth of your knowledge .

First say ActiveMQ,ActiveMQ It's the oldest open source message queue , It was the only open source message queue available a decade ago , We are now in old age , The community is not active . Whether it's function or performance ,ActiveMQ There's a big gap between them and the modern message queue , It exists only to be compatible with the systems of the grandparents who are still in use .

Now let's talk about ZeroMQ, Strictly speaking ZeroMQ It's not a message queue , It's a multithreaded network library based on message queuing , If your requirement is to integrate the function of message queuing into your system process , Consider using ZeroMQ.

One last word Pulsar, Many people may not have heard of this product ,Pulsar Is an emerging open source message queuing product , The first is the Yahoo Development , It's growing up right now , The popularity and maturity are not so high . The biggest difference from other message queues is ,Pulsar Adopt the design of separating storage and calculation , I personally like this design very much , It may lead the development of message queuing in the future , I suggest you keep an eye on this project .

4、 summary

After understanding the characteristics and advantages and disadvantages of the above open source message queues , I believe you have a clear idea of the choice of message queue . I also sum up a few suggestions for your reference .

1） if , Message queuing is not one of the protagonists in the system you're going to build , You don't have high requirements for the function and performance of message queuing , Just a product that's out of the box and easy to maintain , I suggest you use RabbitMQ.

2） If your system uses message queuing, the main scenario is dealing with online business , For example, in the transaction system, the order is delivered by message queue , that RocketMQ Low latency and financial grade stability are what you need .

3） If you need to deal with huge amounts of information , Like collecting logs 、 Monitoring information or front-end buried point data , Or your application scenario uses a lot of big data 、 Open source products related to stream computing , that Kafka It's the most suitable message queue for you .

If these scenes I'm talking about don't match yours , After you look at the features of these message queues that I described earlier , Still don't know how to choose , Then choose the one you are most familiar with , After all, these products can meet most application scenarios , Using familiar products can also quickly get started, not ？

原网站

版权声明
本文为[Qin Tian]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/188/202207062341085076.html