当前位置：网站首页>Consul learning

Consul learning

2022-06-09 16:09:00 【Wind god Shura envoy】

Consul

Consul It's a set Service Mesh Solutions for , It provides the functions of the complete service grid control panel , Including service discovery , To configure , Functional segmentation . These features can be used separately as needed , It can also be used to form a complete service grid .Consul Built in agent , So it can be used out of the box , It also supports the integration of three-party agents , Such as Envoy.

Let's talk about Consul Main features supported ：

Service discovery ：Consul The client can register the service , such as api or mysql, And others Consul Client discovery service provider . Use DNS or HTTP, Applications can easily find the upstream services they depend on .
health examination ：Consul Provide services closely related to any health examination （ Whether the service returns 200 state ）, Or through local node Association （ Is the memory utilization lower than 90%）. The operator uses this information to monitor the health status of the cluster , The service discovery component also uses it to divert unhealthy requests .
KV Storage ： The application can use Consul Level key / Value store , For example, in dynamic configuration , Feature identification , Collaboration , Leader election, etc . These are simple HTTP API Make it easier to use .
Security service communication ： By setting up distribution for different services TLS Credential connection
Multi-data center ：Consul Provide out of the box multi data centers , This eliminates the need to add an additional layer of abstraction to expand multiple areas .

Consul Basic framework

Consul Is distributed 、 Highly available system .Consul Each node that provides services runs in a Consul agent . It is not necessary to run the agent when the service finds or assigns values to key values . The main responsibility of this agent is health examination , Whether the node is normal .

This agent can work with one or more Consul Service dialogue . This Consul Services are where data is stored and assigned . Suggest Consul Service running on 3 To 5 platform , This can effectively avoid the scenario of data loss due to failure . Suggestions for each Consul The service cluster is configured with a data center .

The service maintains a catalog, It consists of aggregated information submitted by the agent . This catelog Maintain the cluster advanced view , Include which services are available , Which nodes run on the service , Health information and so on .

Infrastructure layer components that need to discover other services or nodes can discover other services Consul Service or any Consul agent . This agent can automatically forward queries to the server .

Each data center runs one Consul Server cluster . When a cross data center service discovery or configuration request is generated , Local Consul The server will forward the request to the remote data center and return the result .

Consul practice

In the production environment , To run each agent in server or client mode . Every Consul The data center must have at least one server , Responsible for maintaining Consul The state of .

Non server agents running in client mode . The client is a lightweight process , For registering services 、 Run health checks and forward queries to the server . The client must be running the service Consul Run on every node in the data center , Because the client is the source of the truth about the health of the service .

Download and install Consul

yum -y install consul
consul --version

Open development mode

consul agent -dev
#  Find the data center 
consul members

Be careful , Development mode can only be enabled in the development environment , This option is not safe , Non scalable .
adopt DNS Interface discovery node ,DNS The interface will send your request to Consul The server , If you don't have cache enabled .DNS The query defaults to port 8600 run a query

dig @127.0.0.1 -p 8600 95527e8116d6.node.consul	# 95527e8116d6  This value is determined by  consul members  From the query node

Stop acting

consul leave
#  Output  Graceful leave complete  That is to exit correctly

Use `Consul` Service discovery registration service

As mentioned above ,Consul Provide a DNS The interface is used by downstream services to query upstream dependencies IP Address .Consul Know the location of these services , Because every service is local Consul Client registration . The operator can also register the service manually , Configuration management tools can also register services when deploying services , Or the container orchestration platform can integrate automatic registration services .

Define service meaning

establish consul The configuration file /etc/consul.d, such consul The configuration file of this directory will be loaded
```
mkdir ./etc/consul.d
```
Write a service definition configuration file , Name it web.json
```
vi ./consul.d/web.json
{
      
  "service": {
      
    "name": "web",
    "tags": [
      "rails"
    ],
    "port": 80
  }
}
```
Suppose there is a service running in 80 port , The above content defines the service name , Port and an option tags, You can use this query service later .
Be careful ： Enabling script checking in some configurations may introduce a remote execution vulnerability locked by malware . In the production environment , We strongly recommend that it be changed to -enable-local-script-checks
Start the broker , Use the command line flag to specify the configuration directory , And enable script checking on the agent .
```
consul agent -dev -enable-script-checks -config-dir=./consul.d
```

In multi-agent Consul In the data center , Each service will be delivered locally Consul Client registration , And the client will forward the registration to Consul The server , These servers maintain the service catalog.

If you want to register multiple services , Then just in Consul Create multiple service definition configuration files in the configuration file directory （ First step ）

Query service

Once the agent adds the service to Consul Service for catalog, You can use it DNS Interface or HTTP API Query service .

DNS Interface

As mentioned above , Can pass DNS Interface query service . Service registered DNS The name is Consul Of NAME.service.consul, Among them NAME Is the name of your registered service （ In this case , Namely web）. By default , be-all DNS Names and namespaces are in consul , Can pass To configure modify .

dig @127.0.0.1 -p 8600 web.service.consul
#  Enter the following 
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-16.P2.el7_8.6 <<>> @127.0.0.1 -p 8600 web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41554
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.            IN      A
;; ANSWER SECTION:
web.service.consul.     0       IN      A       127.0.0.1
;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Mon Oct 26 09:11:18 UTC 2020
;; MSG SIZE  rcvd: 63

You can also use the entire IP/port As a SRV Record query

dig @127.0.0.1 -p 8600 web.service.consul SRV

It can also be done through tags Filter query

dig @127.0.0.1 -p 8600 rails.web.service.consul

HTTP API

It can also be done through HTTP API Inquire about

curl http://localhost:8500/v1/catalog/service/web

The result will return all node information of the given service . It can also be in API Specify the services that are running well in the filter query ,DNS Automate behind the scenes .

curl 'http://localhost:8500/v1/health/service/web?passing'

Update service

When the service configuration is updated , You can send a signal SIGNUP To agent or run consul reload Update without closing . besides , You can also use HTTP API Realize increase , Delete and dynamically change services

edit consul The configuration file
```
echo '{ "service": { "name": "web", "tags": [ "rails" ], "port": 80, "check": { "args": [ "curl", "localhost" ], "interval": "10s" } } }' > ./consul.d/web.json
```
Added check node , It will run health checks based on scripts , This check will pass curl Every time 10 Connect to in seconds web service . This script acts as Consul Run under the same user that started .
If the command to exit the code >= 2 sign out , Then the check will fail ,Consul Will consider the service unhealthy . The exit code is 1 Will be treated as a warning state .

Reload profile

consul reload
#  If the operation is successful, every  10  The following information is output in seconds 
Check is now critical: check=service:web

adopt DNS Query services in health status

dig @127.0.0.1 -p 8600 web.service.consul
#  If a service is in an unhealthy state , It doesn't show up , Similarly passed  HTTP  So it is with （ As mentioned above ）

adopt `Consul Service Mesh` Connection service

consul Can pass sidecar Agents connect services to each other ,sidecar Agents are deployed locally with each service instance . This kind of deployment , Local sidecar Agents can control network traffic between service instances , This is a service mesh（ Service grid sidecar Pattern ）. About sidecar Mode see https://docs.microsoft.com/en-us/azure/architecture/patterns/sidecar
consul The service grid allows you to secure and observe the communication between your services , Without modifying the code . contrary ,consul take sidecar Proxy configuration establishes mutual... Between your services TLS, And allow or deny communication between them according to their registered names . because sidecar Agents control the traffic between all services , They can collect （gather） Relevant indicators , And spread it all over the third-party aggregators , such as Prometheus.
You can also use native integration to integrate applications with Consul Connect , For better performance and security .
Registering a service using the service grid is similar to registering a general service ：

Open a service
Register it normally , But add one connect node
Register a second proxy to communicate with the service
Start a sidecar agent
By creating a intention To practice blocking connections to services

Be careful , This step only applies to the development environment , For security reasons, this deployment method does not recommend deployment to the production method , The specific production deployment is detailed in Production environment Consul Connect to the service grid .

Open an unknown Consul service

You can use socat To start a basic echo service , It is used as an upstream service in this example . In the production environment, it is the database 、 Any service that a back-end service or other service depends on .
Socat It has a history of several decades Unix Utilities , It lacks encryption or TLS The concept of agreement . You can use it as Consul An example of what the service grid solves . If your machine does not have socat, You can download and install directly in package management .

socat -v tcp-l:8181,fork exec:"/bin/cat"
#  To start a  socat  service , Specify the  TCP  The port is  8181

You can then use the command nc To verify that it works （ Execute on the correct port echo service ）. Once you enter some characters and press enter . Then the text you input will be output back in its original form

nc 127.0.0.1 8181
#  Input 
hello
#  Output 
hello
#  Input 
how are you
#  Output 
how are you

Consul Registration services and agents

This step is to register , adopt consul Write a new service definition , As mentioned earlier Service definition . This registration will include a connect node , Will register a sidecar The agent handles the traffic of the back-end service strength .
Add one socat.json Service definition profile to /etc/consul.d Directory , The order is as follows

echo '{ "service": { "name": "socat", "port": 8181, "connect": { "sidecar_service": {} } } }' > ./consul.d/socat.json

to update consul

consul reload

Look back at the new connect node , This empty configuration notification consul Register a for this process on a dynamically allocated port sidecar agent .consul With... For testing purposes L4 agent , And right Envoy First class support for , You should use it for production deployment and the 7 Layer traffic management
Now start connecting to the service

consul connet proxy -sidecar-for socat

The following information is output as success

==> Consul Connect proxy starting...
    Configuration mode: Agent API
        Sidecar for ID: socat
              Proxy ID: socat-sidecar-proxy
==> Log data will now stream in as it occurs:
    2020-10-26T16:05:12.943Z [INFO]  proxy: Proxy loaded config and ready to serve
    2020-10-26T16:05:12.943Z [INFO]  proxy: Parsed TLS identity: uri=spiffe://b712d23c-34b7-a743-88e0-9e2b9b972ba7.consul/ns/default/dc/dc1/svc/socat roots=[pri-xyucbjtb.consul.ca.b712d23c.consul]
    2020-10-26T16:05:12.943Z [INFO]  proxy: Starting listener: listener="public listener" bind_addr=0.0.0.0:21000

Register a dependent service and proxy

Next, register a downstream service , Name it “web”. It's like socat Defined services ,web The configuration file for also contains a specified sidecar Of connect node , But it is not as empty as above , But in socat I'm going to assign a web Upstream dependence , The agent will listen for this 9191 port .

echo '{ "service": { "name": "web", "connect": { "sidecar_service": { "proxy": { "upstreams": [ { "destination_name": "socat", "local_bind_port": 9191 } ] } } } } }' > ./consul.d/web.json

If we run a real web service , It will communicate with its agent at a loop address . The agent encrypts the communication （traffic） And send it to sidecar On the agency socat service .Socat The service will decrypt the communication and send it to the local port number 8181 Of the loop address socat. Because there are no services running here , You will be through the detailed installation is web Service and the port we specify (9191) Upper web The agent of the service .
Before starting proxy processing , Make sure you can't connect 9191 socat service .
Now by using the self sidecar The registered profile starts web agent .

consul connect proxy -sidecar-for web

When the following message appears, the agent is successfully started

==> Consul Connect proxy starting...
    Configuration mode: Agent API
        Sidecar for ID: web
              Proxy ID: web-sidecar-proxy
==> Log data will now stream in as it occurs:
    2020-10-27T02:07:41.123Z [INFO]  proxy: Starting listener: listener=127.0.0.1:9191->service:default/socat bind_addr=127.0.0.1:9191
    2020-10-27T02:07:41.131Z [INFO]  proxy: Proxy loaded config and ready to serve
    2020-10-27T02:07:41.132Z [INFO]  proxy: Parsed TLS identity: uri=spiffe://f747f347-a603-96a1-6c36-cbf0d65cbe47.consul/ns/default/dc/dc1/svc/web roots=[pri-r5m71lq.consul.ca.f747f347.consul]
    2020-10-27T02:07:41.132Z [INFO]  proxy: Starting listener: listener="public listener" bind_addr=0.0.0.0:21001

When the input request from port 9191 when , Will output in real time requst from port 9191. But you will happen , The port to listen to before 8181, Will be output request from port 9191. This is because from 9191 The request is forwarded to the target upstream service socat.

Consul data storage

Consul There are two ways to store Key/Value data ：Consul CLI as well as UI. First say CLI

Add data

Use naming consul kv put Insert a data into K/V Memory . The first parameter is the name of the entry key to store , The second is the key value

consul kv put redis/config/minconns 1 # key=redis/config/minconns;value=1

The following information is output and several bits are successful

Success! Data written to: redis/config/minconns

Note the input below redis/config/users/admin, Set a logo flags The value is 42. Key support settings 64 Integer value of bit , This is in Consul The interior is not used .

consul kv put -flags=42 redis/config/users/admin abcd1234

Query data

consul kv get redis/config/minconns
#  Output 
1

Consul Some metadata information about key value pairs is also preserved . have access to -detailed Retrieve metadata

consul kv get -detailed redis/config/users/admin
#  Output 
CreateIndex      32
Flags            42
Key              redis/config/users/admin
LockIndex        0
ModifyIndex      32
Session          -
Value            abcd1234

Query all key value information , The results are returned in dictionary order .

consul kv get -recurse
#  Output 
redis/config/minconns:1
redis/config/users/admin:abcd1234

Delete data

Use keywords delete

consul kv delete redis/config/minconns
#  Output 
Success! Deleted key: redis/config/minconns

Be careful , If you delete a value whose key does not exist , A successful result will be returned by default .
Delete all key value pairs

consul kv delete -recurse redis #  Delete all prefixes with  redis  Key value of 
#  Output 
Success! Deleted keys with prefix: redis

Modifying data

Modify existing values

consul kv put foo bar	#  Store the data 
consul kv get foo	    #  Query data 
consul kv put foo zip	#  Modifying data 
consul kv get foo	    #  Query data

About consul The update data store operation performed is CAS（check-and-set） It's atomic manipulation , It also includes some other related complex operations . You can enter consul kv put -h For more .

Data Center

When consul When the agent is turned on , They don't know each other . Especially for data centers with only one member . There are two ways for agents to know . One is to add new agents to existing data centers , You have to provide any other agent in this data center IP Address （ Whether client or server ）. It will cause new agents to join the data center . Once this agent becomes a member of the new data center , It will go through the pipe （gossip） Know other agents .
So how to connect two agents to each other , Creating a data center with two members ？