Tidb cross data center deployment topology

This paper takes the typical two places and three centers as an example , Introduce the topology and key parameters of cross data center deployment . The city involved in this example is Shanghai （ namely  sha） And Beijing （ namely  bja  and  bjb）.

Topology information

example	Number	Physical machine configuration	BJ IP	SH IP	To configure
TiDB	5	16 VCore 32GB * 1	10.0.1.1 10.0.1.2 10.0.1.3 10.0.1.4	10.0.1.5	Default port Global directory configuration
PD	5	4 VCore 8GB * 1	10.0.1.6 10.0.1.7 10.0.1.8 10.0.1.9	10.0.1.10	Default port Global directory configuration
TiKV	5	16 VCore 32GB 2TB (nvme ssd) * 1	10.0.1.11 10.0.1.12 10.0.1.13 10.0.1.14	10.0.1.15	Default port Global directory configuration
Monitoring & Grafana	1	4 VCore 8GB * 1 500GB (ssd)	10.0.1.16		Default port Global directory configuration

Topology template

Cross machine room configuration template

above TiDB In the cluster topology file , See adopt TiUP Deploy TiDB Cluster topology file configuration .

Key parameter configuration

This section describes cross data center deployment TiDB The key parameter configuration of the cluster .

TiKV Parameters

• Set up gRPC Compression format for , The default is  none. In order to improve the target nodes of the cross machine room deployment scenario gRPC Packet transmission speed , Recommended setting is gzip Format .

  server.grpc-compression-type: gzip

• label To configure

  Due to the cross machine room deployment TiKV, To avoid physical downtime leading to Raft Group default 5 Lost in copy 3 copy , The problem of making the cluster unavailable , Can pass label To achieve PD Intelligent scheduling , Ensure the same center 、 Same cabinet 、 With the machine TiKV Instances will not appear Raft Group Yes 3 The situation of the copy .

• TiKV To configure

  The same physical machine has the same configuration host Level label Information ：

  config: server.labels: zone: bj dc: bja rack: rack1 host: host2

• Prevent offsite TiKV Nodes initiate unnecessary Raft The election , You need to relocate TiKV The node initiates the election after the least tick The number and the maximum number of tick Increase the number , The default settings of these two parameters are  0.

  raftstore.raft-min-election-timeout-ticks: 1000 raftstore.raft-max-election-timeout-ticks: 1020

PD Parameters

• PD Metadata information record TiKV Topology information of the cluster , Schedule according to four dimensions Raft Group copy .

  replication.location-labels: ["zone","dc","rack","host"]

• adjustment Raft Group The amount of replica data is 5 , Ensure high availability of cluster .

  replication.max-replicas: 5

• Reject remote machine rooms TiKV Of Raft The duplicate election is Leader.

  label-property: reject-leader: - key: "dc" value: "sha"

  Be careful

  TiDB 5.2 And above versions do not support  label-property  To configure . To set a replica policy , Please use  Placement Rules.

of Label The use of and Raft Group Copy number , See Through topology label Scheduling replicas .

Be careful

• There is no need to manually create  tidb  user ,TiUP cluster The component will automatically create the user on the target host . You can customize the user , It can also be consistent with the users of the central control computer .
• If the deployment directory is configured as a relative path , It will be deployed in the user's home directory .