Why do I need message idempotence?

background

Why is this problem ？ Still have to rmq Message delivery and consumption , Here is an example

Production and consumption process

1： The generator delivers the message to broker

2：broker Receive the message and ack

3： Consumers from broker in push/pull news

4：3 Reply after the steps broker ——》 ACK

Cause

For the above completion process , Just destroy one of them ack The process will cause messages to be sent repeatedly or consumed again , The following sections are listed

Destructive conditions	influence	Case study
damage broker For producers ack	The producer may repeatedly send	broker High load , Big delay , Suddenly not online
Destroy the consumer to broker Of ack	Consumers may repeat their consumption	stay ack When replying ,broker I feel like I'm ack Timeout and other reasons
…	…	…

analysis

We just need to control every message and consume it once ok 了 , Then the idempotent processing of the message comes , According to the consumer deployment topology, it can be divided into the following situations

Deployment Topology	terms of settlement	remarks
Single point	have access to synchronized + Business check
colony	Consider using high-performance Distributed lock + Business check

Single point model

The process is as follows

The message enters the consumer , It must be entered in sequence , Just make sure that this method of processing messages Process business logic It's sequential , It is mainly about data processing , When the kitchen industry has specific business logic , Judge whether this message has been consumed , If you haven't consumed , Just consume directly , If we consume , Direct discarding , Reply to one OK

Cluster model

The process is as follows

Messages enter the consumers respectively 1 And consumers 2 in , unfortunately , At this time two jvm Will go to the database and so on check state , The discovery is legal , Will continue the following operation , This leads to multiple consumption of messages . Instead, there is the following process

Improve the process

Expand

Distributed lock ： There are several common distributed solutions on the market

programme	remarks
mysql
Zookeeper
Redis
…

Reference resources

Common solutions for distributed locks

https://blog.csdn.net/weixin_45191798/article/details/103115152?spm=1001.2014.3001.5501

summary

1、 This article begins with the introduction of why there is repeated consumption of messages , reason , And basic Elimination principle

2、 Expanded Distributed lock Several common solutions