Use br to back up tidb cluster to GCS

This article describes how to run on Kubernetes Upper TiDB The cluster data is backed up to  Google Cloud Storage (GCS)  On .

The backup method used in this article is based on TiDB Operator Of Custom Resource Definition (CRD) Realization , Bottom use  BR  Get cluster data , Then upload the data to the remote end GCS.BR Its full name is Backup & Restore, yes TiDB Command line tools for distributed backup and recovery , Used to deal with TiDB Cluster for data backup and recovery .

Use scenarios

If you have the following requirements for data backup , Consider using BR take TiDB Cluster data in  Ad-hoc Backup or Scheduled full backup Backup to GCS On ：

• The amount of data that needs to be backed up is large （ Greater than 1 TB）, And it requires faster backup
• Need to backup data directly SST file （ Key value pair ）

If there are other backup requirements , Please refer to Introduction to backup and recovery Choose the right backup method .

Be careful

• BR Only support TiDB v3.1 And above .
• Use BR The data backed up can only be restored to TiDB In the database , Cannot recover to other databases .

Ad-hoc Backup

Ad-hoc Backup supports full backup and incremental backup .

To carry out Ad-hoc Backup , You need to create a custom  Backup custom resource (CR) Object to describe this backup . Create good  Backup  After the object ,TiDB Operator Automatically complete the specific backup process according to this object . If an error occurs during the backup , The program will not automatically retry , At this time, it needs to be handled manually .

This document assumes that the deployment is in Kubernetes test1  This namespace Medium TiDB colony  demo1  Data backup . The following is the specific operation process .

The first 1 Step ： Get ready Ad-hoc Backup environment

1. Download the file  backup-rbac.yaml, And execute the following command in  test1  This namespace To create a backup RBAC Related resources ：

   kubectl apply -f backup-rbac.yaml -n test1

2. Grant remote storage access .

   Reference resources  GCS Account Authorization , Authorized access GCS Remote storage .

3. If you use it TiDB Version below v4.0.8, You also need to complete the following steps . If you use it TiDB by v4.0.8 And above , Please skip these steps .

   1. Make sure you have a backup database  mysql.tidb  Tabular  SELECT  and  UPDATE  jurisdiction , Used to adjust before and after backup GC Time .

   2. establish  backup-demo1-tidb-secret secret For storing access TiDB Clustered root Account and key .

      kubectl create secret generic backup-demo1-tidb-secret --from-literal=password=<password> --namespace=test1

The first 2 Step ： Backup data to GCS

1. establish  Backup CR, Back up the data to GCS：

   kubectl apply -f backup-gcs.yaml

   backup-gcs.yaml  The contents of the document are as follows ：

   --- apiVersion: pingcap.com/v1alpha1 kind: Backup metadata: name: demo1-backup-gcs namespace: test1 spec: # backupType: full # Only needed for TiDB Operator < v1.1.10 or TiDB < v4.0.8 from: host: ${tidb-host} port: ${tidb-port} user: ${tidb-user} secretName: backup-demo1-tidb-secret br: cluster: demo1 clusterNamespace: test1 # logLevel: info # statusAddr: ${status-addr} # concurrency: 4 # rateLimit: 0 # checksum: true # sendCredToTikv: true # options: # - --lastbackupts=420134118382108673 gcs: projectId: ${project_id} secretName: gcs-secret bucket: ${bucket} prefix: ${prefix} # location: us-east1 # storageClass: STANDARD_IA # objectAcl: private

   In the configuration  backup-gcs.yaml  When you file , Please refer to the following information ：

   • since v1.1.6 Since version , If you need incremental backup , Only need  spec.br.options  Specifies the last backup timestamp  --lastbackupts  that will do . Restrictions on incremental backups , May refer to Use BR Back up and restore .
   • .spec.br  Some parameters in are optional , for example  logLevel、statusAddr  etc. . complete  .spec.br  Detailed explanation of fields , Please refer to  BR Field is introduced .
   • spec.gcs  Some parameters in are optional , Such as  location、objectAcl、storageClass.GCS Storage related configuration reference  GCS Storage field introduction .
   • If you use it TiDB by v4.0.8 And above , BR Will automatically adjust  tikv_gc_life_time  Parameters , No configuration required  spec.tikvGCLifeTime  and  spec.from  Field .
   • more  Backup CR Detailed explanation of fields , Please refer to  Backup CR Field is introduced .

2. Create good  Backup CR after ,TiDB Operator Will be based on  Backup CR Automatically start backup . You can check the backup status through the following command ：

   kubectl get bk -n test1 -owide

Backup example

Back up all cluster data

Backing up data from a single database

Back up the data of a single table

Use the table library filtering function to back up the data of multiple tables

Scheduled full backup

Users set backup policies to TiDB The cluster performs scheduled backup , At the same time, set the retention policy of backup to avoid too many backups . Scheduled full backup through customized  BackupSchedule CR Object to describe . A full backup will be triggered every time the backup time point , The bottom layer of scheduled full backup passes Ad-hoc Full backup . The following are the specific steps to create a scheduled full backup ：

The first 1 Step ： Regular full backup environment preparation

Same as  Ad-hoc Full backup environment preparation .

The first 2 Step ： Regularly back up data to GCS

1. establish  BackupSchedule CR, Turn on TiDB Scheduled full backup of the cluster , Back up the data to GCS：

   kubectl apply -f backup-schedule-gcs.yaml

   backup-schedule-gcs.yaml  The contents of the document are as follows ：

   --- apiVersion: pingcap.com/v1alpha1 kind: BackupSchedule metadata: name: demo1-backup-schedule-gcs namespace: test1 spec: #maxBackups: 5 #pause: true maxReservedTime: "3h" schedule: "*/2 * * * *" backupTemplate: # Only needed for TiDB Operator < v1.1.10 or TiDB < v4.0.8 from: host: ${tidb_host} port: ${tidb_port} user: ${tidb_user} secretName: backup-demo1-tidb-secret br: cluster: demo1 clusterNamespace: test1 # logLevel: info # statusAddr: ${status-addr} # concurrency: 4 # rateLimit: 0 # checksum: true # sendCredToTikv: true gcs: secretName: gcs-secret projectId: ${project_id} bucket: ${bucket} prefix: ${prefix} # location: us-east1 # storageClass: STANDARD_IA # objectAcl: private

   From above  backup-schedule-gcs.yaml  The file configuration example shows ,backupSchedule  The configuration of consists of two parts . Part of it is  backupSchedule  Unique configuration , The other part is  backupTemplate.

   • About  backupSchedule  Specific introduction to unique configuration items , Please refer to  BackupSchedule CR Field is introduced .
   • backupTemplate  Used to specify the configuration related to cluster and remote storage , Fields and Backup CR Medium  spec  equally , Please refer to  Backup CR Field is introduced .

2. After the scheduled full backup is created , Check the status of the backup through the following command ：

   kubectl get bks -n test1 -owide

   Check all the backup pieces below the scheduled full backup ：

   kubectl get bk -l tidb.pingcap.com/backup-schedule=demo1-backup-schedule-gcs -n test1