Tikv key performance parameters and optimization

TiKV The overall architecture

 

Focus on three modules :Transaction、Raft、 RocksDB

Focus on the thread pool of these three modules

• scheduler Pool： Coordinate concurrent write conflicts of things , And write down the received modification operation ;
• raftstore thread: After receiving the write request , Convert a write request to raft journal ;raft The log will be stored in rocksdb raft in , Parallel transmission
• apply thread:  from rocksdb raft Read the log in , Applied to the rocksdb kv, So as to complete the data persistence

 

TiKV Read and write flow

  Write request from TiDB The incoming to scheduler pool,scheduler pool Responsible for coordinating concurrent write conflicts ; If there are multiple write requests to write the same KEY Or when you encounter a lock ,scheduler pool adopt latch To line up , To succeed in latch You can go on and pass it on to raftstore pool, Other write requests continue to wait ;

 raftstore pool Will convert the write request to write log raft log, raft log One side will fall to raft Master rocksdb raft log, In addition, it will be sent to follower node ;

apply pool Will raft log Apply to local rocksdb kv, Complete data persistence ;

  Follow the flow chart above to see which part of the monitoring system is elevated , If which position is raised , You can adjust the corresponding parameters according to the following figure （ Turn it up properly ）;

 store-pool-size: Default processing raft Number of thread pools , Default 2;

store-max-bach-size: By default, each batch of requests rows Number , Default 256

raft-max-inflight-msgs: If exceeded raft log The number of write waits exceeds raft-max-inflight-msgs, Will slow down the writing , Carry out current limiting ;

apply-pool-size: The number of threads processing data dropping ;

apply-max-batch-size: Number of requests processed in batch ;

TiKV Read process

  Check the process ：

  Non click Check process

Read bottleneck analysis  

 

 

 

 readpool.unified.max-thread-count: Read thread pool

storage.block-cache.capacity: Block Cache The amount of , If you find that Block Cache Low hit rate , It can be increased properly , This value usually takes up... Of the machine memory 45-60%

split.qps-threshold, Default 3000

split.byte-threshold, Default 30MB/s, When this value is reached, it will default to region Break up , Thus dispersing hot spots ;

common problem