[etcd] etcd usage and cluster construction

etcd It's using Go An open source for language development 、 Highly available distributed key-value The storage system , It can be used for ：

• Configure sharing
• Service registration and discovery
• Distributed lock

There are similar projects or middleware zookeeper and consul, among zookeeper, stay java Is the most used technology stack , And in the go Language is more about using etcd perhaps consul, These two contrasts ,etcd The document of consul More complete .

etcd adopt raft The algorithm achieves strong consistency 、 Highly available service storage directory , Each node in the cluster can use a complete archive , Cluster deployment also achieves high availability .

1. Application scenarios

1.1 Service registration / Service discovery

Service discovery is also one of the most common problems in distributed systems , That is, processes or services in the same distributed cluster , How to find each other and establish a connection . In essence , Service discovery is to find out whether there are processes listening in the cluster udp or tcp port , And through the name you can find and connect . It's like looking it up in a dictionary .etcd Can act as a service Dictionary , The service goes online etcd To register ,etcd Maintain a heartbeat with the service , Ensure that the service is available .

1.2 Configuration center / Configure sharing

Put all kinds of configuration information in the distributed system into etcd On , centralized management . Deployment can be reduced 、 O & M costs .

This kind of application scenario is usually used in ： Write the shared configuration information through the program , When other distributed applications are started, they actively start from etcd Get the configuration information once , meanwhile , The application is etcd Register one on the node Watcher And wait for , Equivalent to a subscriber , as long as etcd Configuration has been updated , These subscribers will be notified in real time , To get the latest configuration information .

1.3 Distributed lock

because etcd Use Raft The algorithm keeps the consistency of data , Therefore, the values stored in the cluster after the operation must be globally consistent , So it's easy to implement distributed locks . There are two ways to implement distributed lock services ：

• Keep exclusive ： Only one can get the lock ,etcd Provides a set of implementation of distributed atomic lock operations CAS(CompareAndSwap) Of API. By setting prevExist value , It can ensure that multiple nodes can create a directory at the same time , There is only one success . The one that is created successfully can be regarded as having obtained the lock .
• Control timing ： Sort all the locks you want to acquire , This order is globally unique ,etcd A set of API, Used to automatically create ordered keys .

To make a long story short ： Maintain access between different processes in different languages etcd The temporal .

2. contrast

etcd Functions that can be realized ,zookeeper Can do it . Why did you choose etcd？

ok , In fact, bloggers are right zookeeper Don't know much . The following is hearsay , Don't spray bloggers if you are wrong ：

• zookeepr Deployment and maintenance are complex ： Administrators need to master a series of knowledge and skills ;java To write , Will introduce a large number of dependencies ; The operation and maintenance personnel generally hope to maintain strong consistency 、 Highly available clusters are as simple as possible , Maintenance is simple and error free .
• Algorithm is complicated ：zookeeper Adopted Paxos Strong consistency algorithm is famous for its complexity ;
• Slow development ：Apache Unique to foundation projects Apache Wag Controversial in the open source community , Due to the huge structure and loose management of the foundation, the development of the project is slow .

etcd Rising star , Technically ’ Rising star ‘, Can often avoid ’ predecessors ‘ The problem of , Improve or even abandon ：

• Simple deployment ：Go Language writing , Simple deployment is a feature ;
• Easy to use ： Use HTTP As an interface ;
• Method is simple ：Raft The algorithm ensures strong consistency , Easy to understand ;
• Data persistence ：etcd The default data is persisted as soon as it is updated ;
• Security ：etcd Support SSL Client security authentication .

3.etcd framework

etcd Divided into four parts ：

• HTTP Server： Used to process the API Requests and others etcd Node synchronization and heartbeat request ;
• Store： Used for processing etcd Transactions of various functions supported , Include Data index 、 Node state change 、 Monitoring and feedback 、 Event handling and execution wait , yes etcd For most of the users API The concrete realization of the function ;
• Raft： Of course it means Raft The realization of strong consistency algorithm , This is a etcd At the heart of ;
• WAL：Write Ahead Log- Pre written logs , yes etcd Data storage mode of . The state of storing all data except in memory 、 Beyond the index of the node ,etcd Will pass WAL Conduct Data persistence Storage .WAL in , All data will be logged prior to submission .
  • Snapshot： State snapshots to prevent excessive data ;
  • Entry： Represents the specific log content stored .

4. Download and install

file ：https://etcd.io/docs/

4.1 download

Go to the official github Download the latest version of etcd, Find the latest release

https://github.com/etcd-io/etcd/releases

I can't tell adm And arm For children's shoes, please refer to the following link ：

Extended reading -x86 x86_64/x64 amd64 arm64/aarch64

4.2 Copy to the server

Bloggers are first in windows Download good , The use of winscp Copy to the server

4.3 decompression

cd /home/randyfield/etcd-release/
tar -zxvf etcd-v3.2.32-linux-amd64.tar.gz

Copy

View the extracted directory

5. function

5.1 Direct operation

cd /home/randyfield/etcd-release/etcd-v3.2.32-linux-amd64
./etcd

Copy

Port is 2379

5.2 nohup

Run command without hanging up , Because in the mainstream Linux It's installed by default in every distribution nohup Command tool :

# start-up 
nohup ./etcd &
# Check the process - Back to the process  id
ps -ef | grep etcd

Copy

6. Use

6.1 Command line

So you need to set environment variables ：

export ETCDCTL_API=3
./etcdctl put name "garfield"
./etcdctl get name 

Copy

If you want to specify an endpoint , Please take it with you --endponits Parameters

 ./etcdctl --endpoints=[127.0.01:2379] put name "garfield"

Copy

7. Cluster building

etcd As a highly available key value storage system , From the moment of birth, it is designed for clustering , So in the actual production environment , Cluster deployment should be used . in addition ,Raft The algorithm needs most nodes to vote when making decisions , therefore etcd An odd number of nodes are recommended for cluster deployment , recommend 3、5、7 Form a cluster . Physical machines can be used 、 virtual machine 、 Container building cluster .

Example ： build 3 Of nodes etcd colony

export ETCDCTL_API=3
TOKEN=token-01
CLUSTER_STATE=new
NAME_1=machine-1
NAME_2=machine-2
NAME_3=machine-3
HOST_1=192.168.31.204
HOST_2=192.168.31.205
HOST_3=192.168.31.206
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380

Copy

• TOKEN: identification etcd colony , Distinguish between different clusters
• CLUSTER_STATE： State of the cluster , Identify that this cluster is new
• CLUSTER： Specify the cluster specific nodes

stay n1 Execute the command in the node to start etcd

etcd --data-dir=data.etcd --name n1 \
 --initial-advertise-peer-urls http://192.168.31.204:2380 --listen-peer-urls http://192.168.31.204:2380 \
 --advertise-client-urls http://192.168.31.204:2379 --listen-client-urls http://192.168.31.204:2379 \
 --initial-cluster ${CLUSTER} \
 --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

Copy

• etcd: start-up etcd command
• --data-dir: Data storage location
• --name : The name of the node
• --initial-advertise-peer-urls: Used to connect other nodes in the same cluster ,2380 port
  • peer： People of equal status , contemporary , So it means between nodes
• --advertise-client-urls 2379 Client connection
• --initial-cluster: Initialize a cluster

stay n2 The node executes a command to start etcd

etcd --data-dir=data.etcd --name n2 \
 --initial-advertise-peer-urls http://192.168.31.205:2380 --listen-peer-urls http://192.168.31.204:2380 \
 --advertise-client-urls http://192.168.31.204:2379 --listen-client-urls http://192.168.31.204:2379 \
 --initial-cluster ${CLUSTER} \
 --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

Copy

stay n3 The node executes a command to start etcd

etcd --data-dir=data.etcd --name n3 \
 --initial-advertise-peer-urls http://192.168.31.206:2380 --listen-peer-urls http://192.168.31.206:2380 \
 --advertise-client-urls http://192.168.31.206:2379 --listen-client-urls http://192.168.31.206:2379 \
 --initial-cluster ${CLUSTER} \
 --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

Copy

Using clusters

Even if the cluster is set up , Nodes in the cluster pass through 2380 Ports communicating with each other , adopt 2379 Communicating with clients

export ETCDCTL_API=3
HOST_1=192.168.31.204
HOST_2=192.168.31.205
HOST_3=192.168.31.206
ENDPONITS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379

etcdctl --endpoints=$ENDPOINTS member list

Copy

• Connect 3 Cluster of nodes , Will be in --endpoints It is specified in 3 Of nodes .