Es snapshot based data backup and restore

1、 What is a snapshot

snapshot Is from the running Elasticsearch The backup obtained in the cluster . You can take snapshots of the entire cluster , You can also target the data flow and index of the entire cluster . You can also snapshot only specific data streams or indexes in the cluster .

The only reliable and supported way to back up a cluster is to take snapshots . You cannot back up by copying the data directory of its node Elasticsearch colony . The method of recovering any data from a file system level backup is not supported . If you try to recover a cluster from such a backup , It may fail by reporting corrupt or missing files or other data inconsistencies , Or it may seem that it has succeeded in silently losing some data .

The data directory copy of the cluster node cannot be used as a backup , Because it is not a consistent representation of its content at a single point in time . You cannot solve this problem by shutting down the node while making a replica , Nor can it be solved by taking an atomic file system level snapshot , because Elasticsearch Have consistency requirements across the entire cluster . The built-in snapshot function must be used for cluster backup

2、 Snapshot compatibility

The snapshot contains a copy of the disk data structure , These data structures form a back-up index to an index or data stream . This means that the snapshot can only be recovered to the point where the index can be read Elasticsearch edition .

The following table shows snapshot compatibility between versions . The first column indicates the base version from which you can restore the snapshot .

3、 Snapshot based backup and restore

3.1 Register storage warehouse

path.repo: ["~/es/backup"]

Be careful ：

• Ensure that the currently configured warehouse directory exists , If it doesn't exist , It needs to be created in advance
• If the cluster contains multiple nodes , You need to configure the shared directory , Single node does not need

3.2 Register the snapshot Repository

PUT /_snapshot/my_backup
{
    
  "type": "fs",
  "settings": {
    
    "location": "~/es/backup"
  }
}

3.3 Create a snapshot

PUT /_snapshot/my_backup/snapshot_1?wait_for_completion=true

3.4 Restore snapshot

POST /_snapshot/my_backup/snapshot_1/_restore

Specify the index or data stream to restore

POST /_snapshot/my_backup/snapshot_1/_restore
{
    
  "indices": "data_stream_1,index_1,index_2",
  "ignore_unavailable": true,
  "include_global_state": false,              
  "rename_pattern": "index_(.+)",
  "rename_replacement": "restored_index_$1",
  "include_aliases": false
}