Introduction and use of etcd

Catalog

1.etcd brief introduction

2. Raft Election algorithm  

3. etcd install

4. etcd Basic use

---

1.etcd brief introduction

etcd Is a highly available distributed key value pair storage system , Commonly used for configuration sharing and service discovery , from CoreOS An open source project initiated by the company , suffer ZooKeeper And doozer Inspired projects , name ”etcd” From two ideas , namely Unix Of ”/etc” The folder and ”d” Distributed systems .”/etc” A folder is a place where configuration data for a single system is stored , and etcd Used to store large-scale distributed configuration information ,etcd It has the following characteristics ：

• Simple ： be based on HTTP+JSON Of API, use curl It's easy to use .
• trusted ： Use Raft The algorithm fully realizes the distributed .
• Security ： Optional SSL Customer authentication mechanism .
• Fast ： Each node can support tens of thousands of QPS Reading and writing .

etcd Yes V2 and V3 Two versions ,V3 Version offers more features and improved performance , The application uses the new grpc API visit mvcc Storage ,mvcc Storage and old storage v2 Is separate and isolated , Write to storage v2 Does not affect the mvcc Storage , write in mvcc Storage does not affect storage v2.

API v2 and API v3 There are some significant differences between ：

• Business ： stay v3 in ,etcd Provides multi key conditional transactions . Applications should use transactions instead of Compare-And-Swap operation .
• Planar bond space ：API v3 There is no directory in , Just the keys . for example ,”/a/b/c/“ It's a key . Range queries support getting all keys that match a given prefix .
• Compact response ：delete The operation no longer returns the previous value . To get the deleted value , You can use transactions to obtain keys atomically , Then delete its value .
• lease ： replace v2 TTL;TTL Bind to lease , The key is attached to the lease .TTL After expired , The lease will be revoked , All additional keys will also be deleted .

2. Raft Election algorithm  

A cluster is usually composed of two or more servers , Each server is a node . Database cluster 、 Management cluster ... The database cluster provides read and write functions , The management cluster provides management 、 Fault recovery and other functions .
For a cluster , The coordination and management of multiple nodes is very important . The master node realizes collaboration and management , The existence of the master node , It can ensure the orderly operation of other nodes , And the write data in the database cluster on each node The consistency of .
Consistency here means , The data is the same in each cluster node , There is no difference . The function of distributed election is to select a master node , It coordinates and manages other nodes , To ensure the orderly operation of the cluster and the consistency of data between nodes .
use Raft Algorithmic election , The roles of cluster nodes are 3 Kind of ：

• Leader: The primary node , There is only one... At the same time Leader, Responsible for coordinating and managing other nodes ;
• Candidate: The candidate , Every node can be Candidate,
• Nodes can only be selected as new under this role Leader; Follower: Leader Follower , You can't have an election .

The election features are as follows ： 

• When the cluster is initialized , Every node is Follower role .
• There are at most two problems in the cluster 1 A valid master node , Synchronize data with other nodes through heartbeat ;
• When Follower No heartbeat is received from the master node within a certain time , Will change their role to Candidate, And launch an election vote .
• After receiving the approval of more than half of the nodes including yourself , The election was a success .
• When less than half of the votes are received, the election fails , Or the election timeout .
• If no master node is selected in this round , There will be the next round of elections ( This happens , Because multiple nodes elect at the same time , All nodes get more than half of the votes ).
• Candidate After the node receives the information from the master node , Will immediately terminate the election process , Get into Follower role .

To avoid a cycle of electoral failure , The time when each node does not receive the heartbeat to initiate the election is a random value in a certain range , This can avoid 2 Nodes initiate elections at the same time .

3. etcd install

Download address ：Releases · etcd-io/etcd · GitHub

  Switch to etcd root directory , take etcd and etcdctl Binary files are copied to /usr/local/bin Directory so that the system can directly call etcd/etcdctl These two programs

cp etcd etcdctl /usr/local/bin

Enter the command etcd, You can start a single node etcd service ,ctrl+c You can stop the service

xxxx-xx-xx xx:xx:xx.xxxxxx I | embed: name = default
xxxx-xx-xx xx:xx:xx.xxxxxx I | embed: data dir = default.etcd 
xxxx-xx-xx xx:xx:xx.xxxxxx I | embed: member dir = default.etcd/member 
xxxx-xx-xx xx:xx:xx.xxxxxx I | embed: heartbeat = 100ms 
xxxx-xx-xx xx:xx:xx.xxxxxx I | embed: election = 1000ms 
xxxx-xx-xx xx:xx:xx.xxxxxx I | embed: snapshot count = 100000 
xxxx-xx-xx xx:xx:xx.xxxxxx I | embed: advertise client URLs = http://localhost:2379 
xxxx-xx-xx xx:xx:xx.xxxxxx I | etcdserver: starting member 8e9e05c52164694d in cluster cdf818194e3a8c32

  establish etcd Related contents ( That is, the storage location of data files and configuration files )

mkdir -p /var/lib/etcd/ && mkdir -p /etc/etcd/

establish etcd The configuration file

vim /etc/etcd/etcd.conf

#  The name of the node 
ETCD_NAME="etcd0"
#  Specify the data file storage location 
ETCD_DATA_DIR="/var/lib/etcd/"

establish systemd The configuration file

vim /etc/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
User=root
Type=notify
WorkingDirectory=/var/lib/etcd/
##  Modify... According to the actual situation EnvironmentFile and ExecStart These two parameter values 
## 1.EnvironmentFile That is, the location of the configuration file , Be careful “-” It can't be less 
EnvironmentFile=-/etc/etcd/etcd.conf
## 2.ExecStart namely etcd Start program location 
ExecStart=/usr/local/bin/etcd
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

4. etcd Basic use

Add and modify , If it exists, replace

etcdctl put < Key name > < Key value > 

Inquire about  

etcdctl get < Key name > 

Delete  

etcdctl del < Key name >

View the cluster status

etcdctl endpoint status --write-out=table

 watch command

watch Is the flow of events that listen for key or prefix changes .

#  To someone key Monitoring operations , When /key1 When there is a change , Will return the latest value  
etcdctl watch /key1 
#  monitor key Prefix  
etcdctl watch /key --prefix 
#  Perform relevant operations after listening to the changes  
etcdctl watch /key1 -- etcdctl member list