How to traverse massive data in redis

Click on the official account , utilize Fragment time to learn

Catalog

1. Preface

2. The accident happened

3. The analysis reason

4. Solution

5. summary

Preface

Sometimes we need to know the online redis Usage situation , In particular, we need to know some prefixes key value , How can we check it ？ Today, Gu shared a little knowledge

The accident happened

Because our users token Caching is using 【user_token:userid】 Format key, Save the user's token Value . In order to help the development partners check how many login users are on the line, our operation and maintenance department .

Directly with the keys user_token* Way to query , The accident happened . Lead to redis Unavailable , Feign death .

The analysis reason

We have millions of online login users , There's a lot of data ;keys The algorithm is a traversal algorithm , Complexity is O(n), That is, the more data , The more time complexity .

The amount of data is millions ,keys This command will lead to Redis Service carton , because Redis It's a single threaded program , Execute all instructions in sequence , Other instructions must wait until the current keys After the command is executed, you can continue .

Solution

So how do we traverse large amounts of data ？ This is also often asked in interviews . We can use redis Another order of scan. Let's see scan Characteristics

1、 The complexity is O(n), But it's done step by step with cursors , Will not block threads

2、 Provide count Parameters , Not the number of results , yes redis The number of slots in a single traverse Dictionary ( About equal to )

3、 Same as keys equally , It also provides pattern matching ;

4、 The server does not need to save state for cursors , The only state of a cursor is  scan Cursor integer returned to the client ;

5、 The returned results may be repeated , Need client to repeat , This is very important ;

6、 Just because the result of a single return is empty doesn't mean the end of the traversal , It depends on whether the returned cursor value is zero

One 、scan Command format

SCAN cursor [MATCH pattern] [COUNT count]

Two 、 Command interpretation ：scan The cursor MATCH < Returns the elements that match the given pattern > count The number of elements returned per iteration

SCAN The command is an incremental loop , Each call returns only a small number of elements . So I won't let redis Feign death

SCAN The command returns a cursor , from 0 To traverse the , To 0 End traversal

3、 ... and 、 give an example

redis > scan 0 match user_token* count 5 
 1) "6"
 2) 1) "user_token:1000"
 2) "user_token:1001"
 3) "user_token:1010"
 4) "user_token:2300"
 5) "user_token:1389"

from 0 To traverse the , Cursor returned 6, Back to the data , continue scan Traverse , It's from 6 Start

redis > scan 6 match user_token* count 5 
 1) "10"
 2) 1) "user_token:3100"
 2) "user_token:1201"
 3) "user_token:1410"
 4) "user_token:5300"
 5) "user_token:3389"

summary

This is what interviews often ask , It is also often used by our small partners in the process of work , Small companies in general , There won't be any problem , But when there's a lot of data , Your operation is not right , Your performance will be deducted , ha-ha . thank you ！！！

source ：toutiao.com/i6697540366528152077

recommend ：

Main stream Java Advanced technology （ Learning material sharing ）

PS： Because the official account platform changed the push rules. , If you don't want to miss the content , Remember to click after reading “ Looking at ”, Add one “ Star standard ”, In this way, each new article push will appear in your subscription list for the first time . spot “ Looking at ” Support us ！