Restore backup data on GCS with tidb lightning

This document describes how to put Kubernetes through TiDB Operator The backup data is restored to TiDB colony .

The recovery method used in this article is based on TiDB Operator v1.1 And above CustomResourceDefinition (CRD) Realization , The bottom layer uses  TiDB Lightning TiDB-backend  To recover data .

TiDB Lightning It is a high-speed import of full data to TiDB Clustering tools , Available from local disk 、Google Cloud Storage (GCS) or Amazon S3 Cloud disk reads data . at present ,TiDB Lightning Support three kinds of backend ：Importer-backend、Local-backend、TiDB-backend. The method introduced in this article uses  TiDB-backend. About the differences and choices of these three backend , see also  TiDB Lightning file . If you want to use  Importer-backend  perhaps  Local-backend  Import data , see also Use TiDB Lightning Import cluster data .

The following examples will be stored in  GCS  The cluster backup data on the specified path on is restored to TiDB colony .

Use scenarios

If you need from GCS Export backup data to TiDB colony , And have the following requirements for data recovery , You can use the recovery scheme introduced in this article ：

• We hope to recover with low resource utilization and low network bandwidth utilization , And it's acceptable 50 GB/ Hours of recovery
• It is required to meet ACID
• During backup TiDB The cluster can still provide services

Preparation before recovery

Before data recovery , You need to prepare to restore the environment , And have the relevant permissions of the database .

Environmental preparation

1. Download the file  backup-rbac.yaml, And execute the following command in  test2  This namespace Create the required for recovery RBAC Related resources ：

   kubectl apply -f backup-rbac.yaml -n test2

2. Remote storage access authorization .

   Reference resources  GCS Account Authorization Authorized access GCS Remote storage .

3. establish  restore-demo2-tidb-secret secret, The secret Store to access TiDB Clustered root Account and key ：

   kubectl create secret generic restore-demo2-tidb-secret --from-literal=user=root --from-literal=password=${password} --namespace=test2

Required database permissions

Use TiDB Lightning take GCS Restore the backup data on to TiDB Before cluster , Make sure you have the following permissions to back up the database ：

jurisdiction	Scope
SELECT	Tables
INSERT	Tables
UPDATE	Tables
DELETE	Tables
CREATE	Databases, tables
DROP	Databases, tables
ALTER	Tables

Restore the specified backup data to TiDB colony

1. establish restore custom resource (CR), Restore the specified backup data to TiDB colony ：

   kubectl apply -f restore.yaml

   restore.yaml  The contents of the document are as follows ：

   --- apiVersion: pingcap.com/v1alpha1 kind: Restore metadata: name: demo2-restore namespace: test2 spec: to: host: ${tidb_host} port: ${tidb_port} user: ${tidb_user} secretName: restore-demo2-tidb-secret gcs: projectId: ${project_id} secretName: gcs-secret path: gcs://${backup_path} # storageClassName: local-storage storageSize: 1Gi

   The above examples will be stored in GCS Specify the path on  spec.gcs.path  Backup data to TiDB colony  spec.to.host. About GCS You can refer to  GCS Field is introduced .

   more  Restore CR For detailed explanation of fields, please refer to  Restore CR Field is introduced .

2. Create good  Restore CR After that, you can view the status of recovery through the following command ：

   kubectl get rt -n test2 -owide

Be careful

TiDB Operator Will create a PVC, For data recovery , The backup data will be downloaded from the remote storage to PV, And then recover . If you want to delete this after the recovery PVC, You can refer to Delete resources First restore Pod Delete , Then take it. PVC Delete .