当前位置:网站首页>Redis based distributed locks and ultra detailed improvement ideas
Redis based distributed locks and ultra detailed improvement ideas
2022-07-06 12:13:00 【A pole】
List of articles
be based on Redis Distributed locks for
What is distributed lock
Meet the visible and mutually exclusive locks of multiple processes in distributed system or cluster mode
Two basic methods to be implemented
Get the lock
Mutually exclusive : Ensure that only one thread can acquire the lock , You can use
setnx
Mutual exclusion ofNon blocking : Try it once , Successfully returns true, Failure to return false
Release the lock
Hand release ,
DEL key
Time out release : Add a timeout when acquiring a lock , Avoid deadlocks caused by service downtime ,
EXPIRE lock 10
Combine
SET lock thread1 NX EX 10 # NX It's mutual exclusion 、EX Is to set the timeout
Realize the idea
The first version of the code
package cn.sticki.common.redis.utils;
import org.springframework.data.redis.core.StringRedisTemplate;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
/** * @author A pole * @version 1.0 * @date 2022/6/21 21:57 */
public class RedisSimpleLock implements ILock {
private final StringRedisTemplate stringRedisTemplate;
private final String name;
private final static String KEY_PREFIX = "lock:";
public RedisSimpleLock(StringRedisTemplate stringRedisTemplate, String name) {
this.stringRedisTemplate = stringRedisTemplate;
this.name = name;
}
@Override
public boolean tryLock(long timeout) {
// 1. Get thread id
long threadId = Thread.currentThread().getId();
// 2. Try to write redis
Boolean success = stringRedisTemplate.opsForValue().setIfAbsent(KEY_PREFIX + name, threadId, timeout, TimeUnit.SECONDS);
// 3. Return the write condition , Success means getting the lock
return Boolean.TRUE.equals(success);
}
@Override
public void unlock() {
// 1. Release the lock
stringRedisTemplate.delete(KEY_PREFIX + name);
}
}
Improve your thinking
The problem is
There is a situation , Execute in the following order :
- Threads 1 Get lock , Start the business , Blocking occurred during execution , In the process of blocking , Lock timeout released
- Threads 2 Get the lock , Because of the thread 1 The lock of has been released overtime , So threads 2 The lock can be successfully obtained
- Threads 1 The business is finished , Release the lock , But threads 1 The lock of has been released for a long time , So the thread is released 2 Lock of
- Threads 3 Get the lock , Threads 3 And thread 2 Concurrent execution , At this time, the lock does not play its due role ...
Sketch Map :
Solution
This problem , In fact, it releases the lock that is not generated by itself , So we can use specific signs , Before releasing the lock, judge whether the lock is generated by itself , And only release the lock generated by yourself .
Threads can be id Deposit in value, Judge the lock before releasing value Is it equal to your own thread id, If equal to, it indicates that the lock is generated by the current thread , You can release .
The new problem
If I have multiple servers , Forming a cluster , Then different servers may have threads id The same thing , It will lead to value identical , Thus, the lock of others is released by mistake .
Solution
Let each started service have a different identity , Then splice threads id, And then we can solve this problem .
Final plan
Store the thread ID when acquiring the lock ( It can be used UUID + Threads id Express )
Get the thread ID in the lock first when releasing the lock , Determine whether it is consistent with the current thread ID
If consistent, release the lock
If not, do not release the lock
The second version of the code
package cn.sticki.common.redis.utils;
import org.springframework.data.redis.core.StringRedisTemplate;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
/** * @author A pole * @version 1.0 * @date 2022/6/21 21:57 */
public class RedisSimpleLock implements ILock {
private final StringRedisTemplate stringRedisTemplate;
private final String name;
private final static String KEY_PREFIX = "lock:";
private final static String KEY_UUID = UUID.randomUUID() + ":";
public RedisSimpleLock(StringRedisTemplate stringRedisTemplate, String name) {
this.stringRedisTemplate = stringRedisTemplate;
this.name = name;
}
@Override
public boolean tryLock(long timeout) {
// 1. Generate key, By splicing prefix and business name
String key = KEY_PREFIX + name;
// 2. Generate value, Used to determine whether the lock is generated by the current thread . Use random UUID+ Current thread id, Prevent clustering value Collision .
String value = KEY_UUID + Thread.currentThread().getId();
// 3. Try to write redis
Boolean success = stringRedisTemplate.opsForValue().setIfAbsent(key, value, timeout, TimeUnit.SECONDS);
// 4. Return the write condition , Success means getting the lock
return Boolean.TRUE.equals(success);
}
@Override
public void unlock() {
String key = KEY_PREFIX + name;
String value = KEY_UUID + Thread.currentThread().getId();
// 1. Get the value of the lock
String lockValue = stringRedisTemplate.opsForValue().get(key);
if (value.equals(lockValue)) {
// 2. If the values are the same , Then the current lock is created by the current thread , You can delete
stringRedisTemplate.delete(key);
}
}
}
The new problem
If , I mean, if , Above unlock() In the code , After obtaining the value of the lock , Delete key Before , There's a blockage (GC Blocking ), When the blocking is complete , The lock created by the current thread has been released , Then something similar to the above happened , It will also lead to lock failure .
Maybe the description is not very clear , Look at the diagram :
Solutions
The main reason for this problem is that judging the lock ID and releasing the lock are two operations performed separately , Solve this problem , Can pass Lua The script binds the two operations together .
Redis Of Lua Script
Redis Provides Lua Script function , Write multiple in one script Redis command , Ensure atomicity when multiple commands are executed .Lua Is a programming language , Its basic syntax can be referred to the website :https://www.runoob.com/lua/lua-tutorial.html
Here we focus on Redis Provided calling function , The grammar is as follows :
# perform redis command
redis.call(' Command name ', 'key', ' Other parameters ', ...)
for example , We have to carry out set name jack, Then the script is like this :
# perform set name jack
redis.call('set', 'name', 'jack')
for example , We need to execute first set name Rose, Re execution get name, The script is as follows :
# Execute first set name jack
redis.call('set', 'name', 'jack')
# Re execution get name
local name = redis.call('get', 'name')
# return
return name
Third edition code
use Lua Write release lock
The business process of releasing locks is like this :
- Get the thread ID in the lock
- Judge whether it is consistent with the specified mark ( The current thread is marked ) Agreement
- If consistent, release the lock ( Delete )
- If not, do nothing
unlock.lua( This document is in mian/resource below )
-- there KEYS[1] It's locked key, there ARGV[1] Is the current thread identifier
-- Get the mark in the lock , Determine whether it is consistent with the current thread mark
if (redis.call('GET', KEYS[1]) == ARGV[1]) then
-- Agreement , Then delete the lock
return redis.call('DEL', KEYS[1])
end
-- atypism , Then return directly
return 0
Java The code is as follows
package cn.sticki.common.redis.utils;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import java.util.Collections;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
/** * @author A pole * @version 1.0 * @date 2022/6/21 21:57 */
public class RedisSimpleLock implements ILock {
private final static String KEY_PREFIX = "lock:";
private final static String KEY_UUID = UUID.randomUUID() + ":";
private static final DefaultRedisScript<Long> UNLOCK_SCRIPT;
static {
// load lua file
UNLOCK_SCRIPT = new DefaultRedisScript<>();
UNLOCK_SCRIPT.setLocation(new ClassPathResource("unlock.lua"));
UNLOCK_SCRIPT.setResultType(Long.class);
}
private final StringRedisTemplate stringRedisTemplate;
private final String name;
public RedisSimpleLock(StringRedisTemplate stringRedisTemplate, String name) {
this.stringRedisTemplate = stringRedisTemplate;
this.name = name;
}
@Override
public boolean tryLock(long timeout) {
// 1. Generate key, By splicing prefix and business name
String key = KEY_PREFIX + name;
// 2. Generate value, Used to determine whether the lock is generated by the current thread . Use random UUID+ Current thread id, Prevent clustering value Collision .
String value = KEY_UUID + Thread.currentThread().getId();
// 3. Try to write redis
Boolean success = stringRedisTemplate.opsForValue().setIfAbsent(key, value, timeout, TimeUnit.SECONDS);
// 4. Return the write condition , Success means getting the lock
return Boolean.TRUE.equals(success);
}
@Override
public void unlock() {
String key = KEY_PREFIX + name;
String value = KEY_UUID + Thread.currentThread().getId();
// call lua Script
stringRedisTemplate.execute(
UNLOCK_SCRIPT,
Collections.singletonList(key),
value);
}
}
summary
Realize the idea :
- utilize set nx ex Get the lock , And set expiration time , Save thread id
- When releasing the lock, first judge whether the thread mark is consistent with itself , If it is consistent, delete the lock
- utilize set nx Satisfy the mutual exclusion
- utilize set ex Ensure that the lock can still be released in case of failure , Avoid deadlock , Improve safety
- utilize Redis The cluster guarantees high availability and high concurrency
Points for improvement :
- Do not reenter : The same thread cannot acquire the same lock more than once
- Can't try again : Only one attempt to acquire a lock returns false, No retry mechanism
- Time out release : Lock timeout release can avoid deadlock , But if the business takes a long time to execute , It will also cause the lock to release , There are safety risks
- Master slave consistency : If Redis Provides a master-slave cluster , There is a delay in master-slave synchronization , When the main goes down , If you slave and synchronize the lock data in the master , The lock will fail
introduce Redisson
Redisson It's a Redis On the basis of implementation Java In memory data grid (In-Memory Data Grid). It not only provides a series of distributed Java Common objects , There are also many distributed services , It includes the implementation of various distributed locks .
Official website address : https://redisson.org
GitHub Address : https://github.com/redisson/redisson
Postscript
This article is a summary and notes I made when learning from the video course of dark horse programmer , Interested students can watch the video by themselves :https://www.bilibili.com/video/BV1cr4y1671t?p=56
边栏推荐
- ESP8266使用arduino连接阿里云物联网
- Pytorch four commonly used optimizer tests
- E-commerce data analysis -- User Behavior Analysis
- VIM command line notes
- History object
- ARM PC=PC+8 最便于理解的阐述
- OSPF message details - LSA overview
- Inline detailed explanation [C language]
- Embedded startup process
- Kaggle competition two Sigma connect: rental listing inquiries
猜你喜欢
Kconfig Kbuild
open-mmlab labelImg mmdetection
js 变量作用域和函数的学习笔记
JS變量類型以及常用類型轉換
数据分析之缺失值填充(重点讲解多重插值法Miceforest)
JS regular expression basic knowledge learning
E-commerce data analysis -- User Behavior Analysis
Stm32f1+bc20+mqtt+freertos system is connected to Alibaba cloud to transmit temperature and humidity and control LED lights
Basic operations of databases and tables ----- classification of data
[template] KMP string matching
随机推荐
OSPF message details - LSA overview
Missing value filling in data analysis (focus on multiple interpolation method, miseforest)
uCOS-III 的特点、任务状态、启动
Detailed explanation of 5g working principle (explanation & illustration)
Priority inversion and deadlock
Fashion Gen: the general fashion dataset and challenge paper interpretation & dataset introduction
Kconfig Kbuild
Implementation scheme of distributed transaction
Analysis of charging architecture of glory magic 3pro
Rough analysis of map file
ES6 grammar summary -- Part 2 (advanced part es6~es11)
js题目:输入数组,最大的与第一个元素交换,最小的与最后一个元素交换,输出数组。
. elf . map . list . Hex file
Arm pc=pc+8 is the most understandable explanation
电商数据分析--用户行为分析
List and set
JS变量类型以及常用类型转换
Understanding of AMBA, AHB, APB and Axi
C语言函数之可变参数原理:va_start、va_arg及va_end
PyTorch四种常用优化器测试