Store Limit usage documentation

Store Limit is a feature introduced by PD in version 3.0, which aims to control the scheduling speed in a more fine-grained manner and optimize for different scheduling scenarios.

implementation principle

PD scheduling is performed in operator units.An operator may contain multiple dispatch operations.Examples are as follows;

"replace-down-replica {mv peer: store [2] to [3]} (kind:region,replica, region:10(4,5), createAt:2020-05-18 06:40:25.775636418 +0000 UTC m=+2168762.679540369, startAt:2020-05-18 06:40:25.775684648 +0000 UTC m=+2168762.679588599, currentStep:0, peer learner steps:[add learner 20 on store 3, promote peer learneron store 3 to voter, remove peer on store 2])"

In the above example, the operator replace-down-replica includes the following operations:

1. Add a learner peer on store 3 with ID 20.
2. Promote the learner peer with ID 20 on store 3 to voter.
3. Delete the peer on store 2.

Store Limit implements store-level speed limit by maintaining a map of store IDs to token buckets in memory.Different operations here correspond to different token buckets. Currently, only the speed of adding learners/peers and deleting peers is limited, that is, there are two types of token buckets in each store.

After each operator is generated, it will check whether there are enough tokens in the token bucket corresponding to the included operation.If the token is sufficient, the operator will be added to the scheduled queue, and the corresponding token will be removed from the token bucket, otherwise the operator will be discarded.The token bucket will replenish tokens at a fixed rate, so as to limit the rate.

Store Limit is different from other PD limit-related parameters (such as region-schedule-limit, leader-schedule-limit, etc.)It is the consumption speed of the operator, and other limits mainly limit the production speed of the operator.Before the Store Limit feature was introduced, the scheduling speed limit was mainly global, so even if the global speed was limited, it was possible that scheduling was concentrated on some stores, thus affecting the performance of the cluster.The Store Limit can better control the scheduling behavior by further refining the granularity of the speed limit.

How to use

Store Limit related parameters can be set through pd-ctl.

View current store-limit settings

The example of viewing the limit of the current store is as follows:

store limit // Displays the speed limit for adding and removing peers from all stores.store limit add-peer // Display the speed limit for adding peers to all stores.store limit remove-peer // Display the speed limit for removing peers from all stores.

set all-store limit

The example of setting the limit of all stores is as follows:

store limit all 5 // Set the speed limit for adding and deleting peers in all stores to 5 per minute.store limit all 5 add-peer // Set the speed limit for adding peers to all stores to 5 per minute.store limit all 5 remove-peer // Set the speed limit for removing peers from all stores to 5 per minute.

Set a single-store limit

An example of setting the limit of a single store is as follows:

store limit 1 5 // Set the speed limit for adding and deleting peers in store 1 to 5 per minute.store limit 1 5 add-peer // Set the speed limit for adding peers in store 1 to 5 per minute.store limit 1 5 remove-peer // Set the speed limit for removing peers from store 1 to 5 per minute.