Etcd tutorial - Chapter 8 compact, watch, and lease APIs for etcd

One 、Compact Compress

Compact The method is compression Etcd The event history in the key value pair store . Key value pair storage should be compressed periodically , Otherwise, the event history will continue to grow indefinitely .

rpc Compact(CompactionRequest) returns (CompactionResponse) {}

The message body of the request is CompactionRequest, CompactionRequest Is a compressed key value pair stored to a given revision . All keys with a revision smaller than the compressed revision will be deleted ：

message CompactionRequest {
  //  Revision of the key value store , For comparison operations 
  int64 revision = 1;

  bool physical = 2;
}

physical Set to true when RPC It will wait until the compression physics is applied to the local database , Items compressed to this extent will be completely removed from the back-end database . The message body of the reply CompactionResponse Defined as ：

message CompactionResponse {
  ResponseHeader header = 1;
}

CompactionResponse There is only one generic response header .

Two 、Watch service

Watch API Provides an event based interface , Used to asynchronously monitor key changes .Etcd3 The monitor passes from a given revision （ Current version or historical version ） Keep an eye on key change , And will key The update flows back to the client .

event

Each key change is represented by an event message . The event message will provide both update data and update type ,mvccpb.Event The message body of is defined as follows ：

message Event {
  enum EventType {
    PUT = 0;
    DELETE = 1;
  }

  EventType type = 1;

  KeyValue kv = 2;

  // prev_kv  Hold the key value pair before the event 
  KeyValue prev_kv = 3;
}

type Is the type of event . If the type is PUT, Indicates that the new data has been stored in key; If the type is DELETE, indicate key It has been deleted .

kv Hold for events KeyValue.PUT The event contains the current kv Key value pair .kv.Version=1 Of PUT Events indicate key The creation of .DELETE/EXPIRE The event contains the deleted key, Its modified revision is set to the deleted revision .

Monitor flow

Watch API Provides an event based interface , Used to asynchronously monitor key changes .etcd The monitor passes from a given revision （ Current version or historical version ） Continuous monitoring to wait for key changes , And stream the key update back to the client .

Monitor continuous operation , And use gRPC To stream event data . The monitoring flow is bidirectional , The client writes the stream to establish a monitoring event , And read to receive monitoring events . A single monitoring flow can reuse many different observations by marking events with each observer identifier . This multiplexing helps to reduce etcd Memory footprint and connection overhead on the cluster .

Watch Events have three characteristics ：

• Orderly , Events are sorted in revision order ; If the event is earlier than the published event , It will never appear on the list .
• reliable , Event sequences never discard any event subsequences ; If the chronological order is a < b < c Three events , So if Watch Event received a and c, You can ensure that you receive b.
• atom , Ensure that the event list contains a complete revision ; Updates made through multiple keys in the same revision will not be split into multiple event lists .

Watch service Definition
stay rpc.proto in Watch service The definition is as follows ：

service Watch {
  rpc Watch(stream WatchRequest) returns (stream WatchResponse) {}
}

Watch Observe what will happen or has happened . Both input and output are streams ; The input stream is used to create and cancel observations , And the output stream sends Events . An observation RPC It can be used in multiple... At one time key Observe... In scope , And for multiple observation fluidization Events . The whole event history can be observed from the last compressed revision .

WatchService only one Watch Method .

The message body of the request WatchRequest The definition is as follows ：

message WatchRequest {
  oneof request_union {
    WatchCreateRequest create_request = 1;
    WatchCancelRequest cancel_request = 2;
  }
}

request_union Or a request to create a new observer , Or cancel the request of an existing observer . Request to create a new observer WatchCreateRequest：

message WatchCreateRequest {
  // key  Registration is to observe  key
  bytes key = 1;

  bytes range_end = 2;

  // start_revision  Is an optional start ( Include ) Observed revisions . Not set up  start_revision  said  " Now? ".
  int64 start_revision = 3;

  bool progress_notify = 4;

  enum FilterType {
  //  To filter out  put  event 
  NOPUT = 0;

  //  To filter out  delete  event 
  NODELETE = 1;
  }

  //  filter , Before the server sends the event back to the observer , Filter out events .
  repeated FilterType filters = 5;

  //  If  prev_kv  Set up , The created observer gets the last... Before the event occurs KV.
  //  If last time KV Has been compressed , Nothing is returned 
  bool prev_kv = 6;
}

• range_end Is the range to be observed [key, range_end) The end . If range_end No settings , Then there are only parameters key Be observed ; If range_end Equate to ‘\0’, Is greater than or equal to the parameter key All of the key Will be observed ; If range_end Ratio given key Big 1, Then all are given key Prefixed key Will be observed .

• progress_notify Set up , So if there are no recent events ,etcd The server will periodically send messages without any events WatchResponse To the new observer . Useful when the client wants to recover a disconnected observer from the most recent known revision .etcd The server will determine how often it sends notifications based on the current load .

Cancel an existing observer WatchCancelRequest：

message WatchCancelRequest {
  int64 watch_id = 1;
}

watch_id Is for the observer to be cancelled id, So no more events will spread .

The message body of the reply WatchResponse：

message WatchResponse {
  ResponseHeader header = 1;
  // watch_id  It is the observer's that is related to the response ID
  int64 watch_id = 2;

  bool created = 3;

  bool canceled = 4;

  int64 compact_revision  = 5;

  // cancel_reason  Point out the reasons for canceling the observer .
  string cancel_reason = 6;

  repeated mvccpb.Event events = 11;
}

• If the response is used to create an observer request , be created Set to true. The client should record watch_id And expect to receive events from the same stream for the created observer . All events sent to the created observer will be accompanied by the same watch_id; If the response is used to cancel the observer request , be canceled Set to true. No more events will be sent to the cancelled observer .
• compact_revision Is set to minimum index, If the observer tries to observe the compressed index. Occurs when an observer is created on a compressed revision or when the observer cannot keep up with the progress of key value pair storage . The client should treat the observer as cancelled , You should not try to create any more with the same start_revision The observer .

3、 ... and 、Lease service

Lease service Provide lease support .Lease It is a mechanism to detect the survival of clients . The cluster grants a lease with a lifetime . If Etcd Cluster in a given TTL Not received in time keepAlive, Then the lease expires .

To bind a lease to a key value , Every key At most one lease can be attached . When the lease expires or is revoked , All... Attached to this lease key Will be deleted . Each expired key will generate a delete event in the event history .

stay rpc.proto in Lease service The defined interfaces are as follows ：

service Lease {

  rpc LeaseGrant(LeaseGrantRequest) returns (LeaseGrantResponse) {}

  rpc LeaseRevoke(LeaseRevokeRequest) returns (LeaseRevokeResponse) {}

  rpc LeaseKeepAlive(stream LeaseKeepAliveRequest) returns (stream LeaseKeepAliveResponse) {}

  rpc LeaseTimeToLive(LeaseTimeToLiveRequest) returns (LeaseTimeToLiveResponse) {}
}

• LeaseGrant： Create a lease
• LeaseRevoke： Cancel the lease
• LeaseKeepAlive： Maintain the lease
• LeaseTimeToLive： Get lease information

3.1 LeaseGrant Method

LeaseGrant Method to create a lease . When the server is in a given time to live I didn't receive keepAlive When the lease expires . If the lease expires, all attached to the lease key Will expire and be deleted . Every expired key Generate a delete event in the event history .

The method is defined as follows ：

rpc LeaseGrant(LeaseGrantRequest) returns (LeaseGrantResponse) {}

The message body of the request is LeaseGrantRequest：

message LeaseGrantRequest {
  int64 TTL = 1;

  int64 ID = 2;
}

TTL It is recommended that..., in seconds time-to-live.ID It's a lease request ID, If ID Set to 0, Then the lessor ( That is to say etcd server) Select a ID.

The message body of the reply LeaseGrantResponse The definition is as follows ：

message LeaseGrantResponse {
  ResponseHeader header = 1;

  int64 ID = 2;

  int64 TTL = 3;
  string error = 4;
}

ID Is a recognized lease ID.TTL Is the lease selected by the server in seconds time-to-live.

3.2 LeaseRevoke Method

LeaseRevoke Cancel a lease , At this point, all key Will expire and be deleted .

rpc LeaseRevoke(LeaseRevokeRequest) returns (LeaseRevokeResponse) {}

The message body of the request LeaseRevokeRequest The definition is as follows ：

message LeaseRevokeRequest {

  int64 ID = 1;
}

ID Is the lease to be cancelled ID. When the lease is cancelled , All related key place it on clipboard .

The message body of the reply LeaseRevokeResponse The definition is as follows ：

message LeaseRevokeResponse {
  ResponseHeader header = 1;
}

LeaseRevokeResponse There is only one common response header field in .

3.3 LeaseKeepAlive Method

LeaseKeepAlive Method to maintain a lease .LeaseKeepAlive Through streaming from client to server keep alive Request and streaming from server to client keep alive Answer to maintain the lease .

rpc LeaseKeepAlive(stream LeaseKeepAliveRequest) returns (stream LeaseKeepAliveResponse) {}

The message body of the request LeaseKeepAliveRequest The definition is as follows ：

message LeaseKeepAliveRequest {
  int64 ID = 1;
}

ID A lease that will survive ID.

The message body of the reply LeaseKeepAliveResponse：

message LeaseKeepAliveResponse {
  ResponseHeader header = 1;
  int64 ID = 2;

  int64 TTL = 3;
}

ID It's a lease from the survival request ID.TTL Is the lease new time-to-live.

3.4 LeaseTimeToLive Method

LeaseTimeToLive Method to get the lease information .

rpc LeaseTimeToLive(LeaseTimeToLiveRequest) returns (LeaseTimeToLiveResponse) {}

The message body of the request LeaseTimeToLiveRequest The definition is as follows ：

message LeaseTimeToLiveRequest {
  // ID  It's a lease  ID
  int64 ID = 1;
  bool keys = 2;
}

keys Set to true You can query all... Attached to this lease key.

The message body of the reply LeaseTimeToLiveResponse The definition is as follows ：

message LeaseTimeToLiveResponse {
  ResponseHeader header = 1;
  // ID  From the request  ID
  int64 ID = 2;
  int64 TTL = 3;
  int64 grantedTTL = 4;
  repeated bytes keys = 5;
}

among ,TTL Is the remainder of the lease TTL, The unit is in seconds ; The lease will be in the following TTL + 1 Seconds later .GrantedTTL Lease creation / The time of initial award at the time of renewal , The unit is in seconds .keys Is attached to this lease key A list of .