当前位置:网站首页>Redis can only cache? Too out!
Redis can only cache? Too out!
2022-07-07 13:45:00 【Heapdump performance community】
Most databases , Because I often deal with disks , In high concurrency scenarios , The response will be very slow . In order to solve this speed difference , Most systems habitually add a cache layer , To speed up data reading .redis Because of its excellent processing power and rich data structure , It has become a de facto distributed caching standard .
however , If you think redis If you can only cache , Then it's too small .
redis Rich data structure , This makes its business usage scenarios very extensive , add rdb The persistence feature of , It can even be used as a landing database . under these circumstances ,redis Can support most Internet companies , Especially social 、 game 、 Half of the live broadcasting companies .
1、Redis Capable of storage
redis It provides a very rich cluster mode : Master-slave 、 sentry 、cluster, Meet the demand for high availability of services . meanwhile ,redis There are two ways to persist :aof and rdb, What is commonly used is rdb.
adopt bgsave Instructions , The main process will fork A new process , Write back to disk .bgsave It's equivalent to taking a snapshot , Because it does not WAL Journal and checkpoint Mechanism , You can't do real-time backup . If the machine suddenly loses power , Then it's easy to lose data .
Fortunately, ,redis It's a memory database , The speed of main plex synchronization is very fast . If your cluster is well maintained , Reasonable memory allocation , Then unless the machine room is powered off , otherwise redis Of SLA, Will remain at a very high level .
It doesn't sound absolutely reliable , There is a possibility of losing data ! This is in general CRUD The business of , It's intolerable . What is it for? redis It can meet the needs of most Internet companies ? This is also determined by business attributes .
In deciding to embrace as much as possible redis Before , You need to confirm whether your business has the following characteristics :
In addition to the core business , Whether most businesses have low requirements for data reliability , Losing one or two pieces of data is tolerable ?
Faced with C End user , According to the user ID Quickly locate a class of data , Data sets are generally small ? There is no need for a large number of range queries ?
Whether it can bear the cost demand of memory data ?
Whether the business requires almost no transaction operations ?
Fortunately , There are many special needs for this kind of business . For example, common social , game 、 live broadcast 、 Operational business , Can be completely dependent on Redis Of .
2. Reids Application scenarios
Redis With loose document structure , Rich data types , Able to adapt to ever-changing scheme Change requirements , I'm going to talk about that Redis A large number of application scenarios other than caching .
2.1 Basic user data storage
In traditional database design , User tables are very difficult to design , When changing, it will hurt muscles and bones . Use Redis Of hash structure , Loose data model design can be realized . Some are not fixed , Functional properties of verification type , We can use JSON The interface is stored directly in hash Of value in . Use hash structure , May adopt HGET and HMGET Such as instruction , Only get the data you need , It is also very convenient to use .
>HSET user:199929 sex m>HSET user:199929 age 22>HGETALL user:1999291) "sex"2) "m"3) "age"4) "22"
This non statistical 、 Read more and write less , It is very suitable for KV Structure for storage .Redis Of hash Structure provides a very rich set of instructions , You can also use HINCRBY Increase and decrease , Very convenient .
2.2 Implementation counter
It mentioned a little HINCRBY Instructions , And for Redis Of Key itself , Also have INCRBY Instructions , Realize the increment and decrement of a value .
For example, the following scenarios : Count the likes of a post ; Store the number of concerns about a topic ; Number of fans storing a tag ; Store a general number of comments ; The popularity of a post ; The number of red dot messages ; give the thumbs-up 、 like 、 Collection number, etc .
> INCRBY feed:e3kk38j4kl:like 1> INCRBY feed:e3kk38j4kl:like 1> GET feed:e3kk38j4kl:like"2"
A hot business like microblog , Traditional database , It must not hold , With the help of an in memory database . because Redis Very fast , You don't have to use tradition DB Very slow count operation , All such incremental operations are on the millisecond level , And the effect is real-time .
2.3 Ranking List
Leaderboards can improve the enthusiasm of participants , So this business is very common , It's essentially a topn The problem of .
Redis One of them is called zset Data structure of , An ordered list implemented using a jump table , You can easily implement problems such as leaderboards . When the zset Data in , Reach the level of tens of millions or even billions , Can still maintain very high concurrent read and write , And has a great average response time (5ms within ).
Use zadd You can add new records , We will use ranking related scores , As a record score value , And then use zrevrange Command to obtain real-time leaderboard data , and zrevrank You can easily get the real-time ranking of users .
>ZADD sorted:xjjdog:2021-07 55 dog0>ZADD sorted:xjjdog:2021-07 89 dog1>ZADD sorted:xjjdog:2021-07 32 dog2>ZCARD sorted:xjjdog:2021-07>3> ZREVRANGE sorted:xjjdog:2021-07 0 -10 WITHSCORES # top10 Ranking List 1) "dog1"2) "89"3) "dog0"4) "55"5) "dog2"6) "32"
2.4 Friend relationship
set structure , Is a collection without duplicate data , You can put a user's attention list 、 Fans list 、 Two way follow list 、 The blacklist 、 Like list, etc , Use independent zset For storage .
Use ZADD、ZRANK etc. , Use the user's blacklist ZADD add to ,ZRANK Use the returned sorce Value to determine whether it is in the blacklist . Use sinter Instructions , Can get A and B Our mutual friends .
In addition to friends , With a clear blacklist 、 White list business scenario data , You can use set Structure for storage . There are many other business scenarios , For example, the address book uploaded by a user , Calculate the friend relationship of the address book, etc .
In practical use , Use zset Store more of these relationships .zset Same as set equally , Duplicate values are not allowed , but zset One more. score Field , We can store a timestamp , Used to indicate when the relationship was established , Have a clearer business meaning .
2.5 Count the number of active users
Similar statistics of daily active users 、 User check-in 、 User online status , This scattered demand , There are too many . If you store one for each user bool Variable , That takes up too much space . In this case , We can use bitmap structure , To save a lot of storage space .
>SETBIT online:2021-07-23 3876520333 1>SETBIT online:2021-07-24 3876520333 1>GETBIT online:2021-07-23 38765203331>BITOP AND active online:2021-07-23 online:2021-07-24>GETBIT active 38765203331>DEBUG OBJECT online:2021-07-23Value at:0x7fdfde438bf0 refcount:1 encoding:raw serializedlength:5506446 lru:16410558 lru_seconds_idle:5(0.96s)
Be careful , If your id It's big , You need to do a preprocessing first , Otherwise it will take up a lot of memory .
bitmap Contains a continuous string of 2 Hexadecimal Numbers , Use 1bit To express the question of true and false . stay bitmap On , have access to and、or、xor Equipotential operation (bitop).
2.6 Distributed lock
Redis Distributed locks for , Is a lightweight solution . Although it's not as reliable as Zookeeper Systems like that , but Redis Distributed locks have very high throughput .
One of the most rudimentary locking actions , have access to redis belt nx and px Parametric set Command to complete . Here is a simple distributed sample code .
public String lock(String key, int timeOutSecond) { for (; ; ) { String stamp = String.valueOf(System.nanoTime()); boolean exist = redisTemplate.opsForValue().setIfAbsent(key, stamp, timeOutSecond, TimeUnit.SECONDS); if (exist) { return stamp; } }}public void unlock(String key, String stamp) { redisTemplate.execute(script, Arrays.asList(key), stamp);}
Delete the lua by .
local stamp = ARGV[1]local key = KEYS[1]local current = redis.call("GET",key)if stamp == current then redis.call("DEL",key) return "OK"end
redisson Of RedLock, Is the most widely used distributed lock solution , There is a difference between read and write locks , And handled many redis Exception problem in case of instance .
2.7 Distributed current limitation
Use counters to achieve simple current limiting , stay Redis It is very convenient in , Just use incr coordination expire Command can .
incr key expire key 1
This simple implementation , Usually there's no problem , But when the flow is relatively large , There is a risk of a sudden surge in traffic over a time span . The root cause , It's this way of time segmentation. It's too fixed , There is no smooth transition scheme like sliding window .
The same is redisson Of RRateLimiter, Realized and guava A similar distributed current limiting tool class in , It's very convenient to use . Here's a short example :
RRateLimiter limiter = redisson.getRateLimiter("xjjdogLimiter"); // It only needs to be initialized once // Every time 2 Second 5 Permission limiter.trySetRate(RateType.OVERALL, 5, 2, RateIntervalUnit.SECONDS); // No license available , Will be blocked all the time limiter.acquire(3);
2.8 Message queue
redis Simple queues can be implemented . On the producer side , Use LPUSH Add to a list ; On the consumer side , Continuous use RPOP Instruction to fetch the data , Or use blocked BRPOP Command to get data , Suitable for small-scale rush buying demand .
Redis also PUB/SUB Pattern , however pubsub It is more suitable for business such as message broadcasting .
stay Redis5.0 in , Added stream Type of data structure . It is more similar to Kafka, There is the concept of theme and consumption group , Multicast and persistence can be realized , Has been able to meet most business needs .
2.9 LBS application
Early in Redis3.2 edition , It's launched GEO function . adopt GEOADD Command append lat、lng Latitude and longitude data , The distance between coordinates can be calculated 、 Include relationship calculations 、 Functions such as people nearby .
About GEO function , The most powerful open source solution is based on PostgreSQL Of PostGIS, But for a general scale GEO service ,redis It's enough .
2.10 More extended application scenarios
Want to see redis Can do , I have to mention the following java Client class library redisson.redisson Contains rich distributed data structures , All based on redis Designed .
redisson For example Set、 SetMultimap、 ScoredSortedSet、 SortedSet, Map、 ConcurrentMap、 List、 ListMultimap、 Queue、BlockingQueue And so on , Make based on redis Programming is more convenient . stay github On , You can see that there are hundreds of such data structures :https://github.com/redisson/redisson/tree/master/redisson/src/main/java/org/redisson/api.
For a language , Basic array 、 Linked list 、 Collection etc. api, Together, it can complete the development of most businesses .Redis No exception , It has these basic api Operational capability , Can also combine the composition of cloth 、 Thread safe high concurrency applications .
because Redis It's memory based , So it's very fast , We will also use it as a storage place for intermediate data . For example, some common configurations , Put it in redis Share in , It acts as a configuration center ; For example JWT Store your token in Redis in , You can break through JWT Some of the limitations of , Make sure you log out safely .
3. "One-stop" work style Redis challenges
redis Rich data structure , It generally does not cause functional problems . But as the number of requests increases ,SLA An increase in demand , We are bound to Redis Do some transformation and customization development .
3.1 High availability challenges
redis Provides master-slave 、 sentry 、cluster And so on , among cluster The model is the way most companies use now .
however ,redis Of cluster Pattern , There are a lot of hard injuries .redis cluster Adopt the concept of virtual slot , Put all the key Mapping to 0~16383 In an integer slot , It belongs to a decentralized architecture . But its maintenance cost is high ,slave Nor can it participate in read operations .
Its main problem , The limitation lies in some batch operations . because key By hash To multiple machines , therefore mget、hmset、sunion Such operations are very unfriendly , Performance problems often occur .
redis The master-slave mode is the simplest mode , But you can't do it automatically failover, Usually after master-slave switching , You also need to modify the business code , It's intolerable . Even with haproxy Such load balancing components , Complexity is also very high .
Sentinel mode when there are a large number of masters and slaves , Can significantly reflect its value . A cluster of sentinels , Able to monitor hundreds of clusters , However, it is difficult to maintain the sentinel cluster itself . Fortunately, ,redis The text protocol is very simple , stay netty in , Even directly provide redis Of codec. Self developed a sentinel system , Strengthen its function , It is feasible. .
3.2 Separation of hot and cold data
redis Is characterized by , Whatever the data , They all go into memory to do calculations , This is for the concept of time series , Business with hot and cold data , Caused a very large cost test . Why do most developers like to store data in MySQL in , instead of Redis in ? In addition to transactional requirements , The big reason is the problem of historical data .
Usually , This switching of hot and cold data , It is completed by middleware . We also talked about ,Redis It's a text protocol , It's simple . Make a middleware , Or make a protocol compatible Redis Analog storage , It's easier .
For example, we Redis in , Keep only active users in the last year . An inactive user for several years , Suddenly accessed the system , When we get the data , You need middleware to transform , From larger capacity , Find in slower storage .
This is the time ,Redis The role of , It's more like a heat reservoir , More like a tradition cache What the layer does , When the business is already on scale . But notice , By this time , Our business layer code , It's always operational redis Of api. They use these numerous function instructions , I don't care whether the data is really stored in redis in , Still ssdb in .
3.3 Functional requirements
redis Can also play a lot of tricks . for instance , Full text search . Many people will choose es, but redis Ecology provides a module :RediSearch, You can query , You can do it filter.
But we usually have more needs , Such as statistics 、 Search class 、 Operation effect analysis, etc . Such demands are related to big data , Even traditional DB Not competent . Now , Of course we have to redis Data in , Import to other platforms for calculation .
If you choose redis database , that dba Dealing with , Namely rdb, instead of binlog. There are a lot of rdb Parsing tool ( such as redis-rdb-tools), Be able to regularly rdb Parse into records , Import to hadoop And other platforms .
here ,rdb Become the hub of all teams , Become the basic data exchange format . Import to other db Business after , How to play, how to play , Not because the business system chooses redis It won't work .
4. summary
Most business systems , Run in the redis On , This is a lot that has been used MySQL Students who do business systems can't imagine . After reading the above introduction , I believe you can be right redis Have a general understanding of the storage functions that can be realized . Open your social app、 game app、 video app, Take a look at their functions , How much can it cover ?
What I want to emphasize here is , Some data , You don't have to land on RDBMS It's safe , They are not a strong demand .
Since the redis So powerful , Why do we have to mysql、tidb Such storage ? The key lies in business attributes .
If a business system , Data for each interaction , Is a very large result set , And involves very complex Statistics 、 Filtering work , that RDBMS Is a must ; But if a system , Be able to pass through an identification , Quickly locate a class of data , This kind of data in the foreseeable future , It is limited. , That's perfect for Redis Storage .
An e-commerce system , choose redis Storage is death , But a social system is much happier . Choose the right tools in the right scene , That's what we should do .
But a system , Whether it can be in the product validation period , Can quickly respond to changes , Fast development online , Is the key to success . This is also used redis Do the database , The biggest benefit . Don't let the data loss scenario with very low probability , Scared . Compared to product success , Your system is as strong as steel , It's worthless .
Taste of little sister (xjjdog), An official account that does not allow programmers to take turns. . Focus on infrastructure and Linux. Ten year structure , 10 billion traffic per day , Discuss the world of high concurrency with you , Give you a different taste . My personal wechat xjjdog0, Welcome to add friends , Further communication .
边栏推荐
- 为租客提供帮助
- Co create a collaborative ecosystem of software and hardware: the "Joint submission" of graphcore IPU and Baidu PaddlePaddle appeared in mlperf
- MongoDB优化的几点原则
- Getting started with MySQL
- Japanese government and enterprise employees got drunk and lost 460000 information USB flash drives. They publicly apologized and disclosed password rules
- Build a secure and trusted computing platform based on Kunpeng's native security
- 2022-7-6 Leetcode 977.有序数组的平方
- Ikvm of toolbox Net project new progress
- PAcP learning note 1: programming with pcap
- SSRF漏洞file伪协议之[网鼎杯 2018]Fakebook1
猜你喜欢
1、深拷贝 2、call apply bind 3、for of for in 区别
10 pictures open the door of CPU cache consistency
566. Reshaping the matrix
"Song of ice and fire" in the eleventh issue of "open source Roundtable" -- how to balance the natural contradiction between open source and security?
Introduction and basic use of stored procedures
交付效率提升52倍,运营效率提升10倍,看《金融云原生技术实践案例汇编》(附下载)
118. Yanghui triangle
LIS longest ascending subsequence problem (dynamic programming, greed + dichotomy)
Write it down once Net a new energy system thread surge analysis
单片机学习笔记之点亮led 灯
随机推荐
cmake 学习使用笔记(一)
My "troublesome" subordinates after 00: not bad for money, against leaders, and resist overtime
Split screen bug notes
C语言数组相关问题深度理解
Cinnamon taskbar speed
clion mingw64中文乱码
2022-7-6 Leetcode27. Remove the element - I haven't done the problem for a long time. It's such an embarrassing day for double pointers
Help tenants
2022-7-7 Leetcode 34.在排序数组中查找元素的第一个和最后一个位置
2022-7-7 Leetcode 844.比较含退格的字符串
"Song of ice and fire" in the eleventh issue of "open source Roundtable" -- how to balance the natural contradiction between open source and security?
Shell batch file name (excluding extension) lowercase to uppercase
[1] ROS2基础知识-操作命令总结版
566. Reshaping the matrix
2022-7-6 初学redis(一)在 Linux 下下载安装并运行 redis
Navicat运行sql文件导入数据不全或导入失败
Summary of import, export, backup and recovery of mongodb
2022-7-6 beginner redis (I) download, install and run redis under Linux
【堡垒机】云堡垒机和普通堡垒机的区别是什么?
2022-7-6 Leetcode27.移除元素——太久没有做题了,为双指针如此狼狈的一天