当前位置:网站首页>Use dumping to back up tidb cluster data to GCS
Use dumping to back up tidb cluster data to GCS
2022-07-07 04:02:00 【Tianxiang shop】
This document describes how to put Kubernetes On TiDB The data of the cluster is backed up to Google Cloud Storage (GCS) On . In this document “ Backup ”, All refer to full backup ( namely Ad-hoc Full backup and scheduled full backup ).
The backup method described in this document is based on TiDB Operator(v1.1 And above ) Of CustomResourceDefinition (CRD) Realization , Bottom use Dumpling The tool obtains the logical backup of the cluster , Then upload the backup data to the remote end GCS.
Dumpling Is a data export tool , The tool can store in TiDB/MySQL The data in is exported as SQL perhaps CSV Format , It can be used to complete logical full backup or export .
Use scenarios
If you need to TiDB Cluster data in Ad-hoc Full volume backup or Scheduled full backup Backup to GCS, And there are the following requirements for data backup , You can use the backup scheme introduced in this article :
- export SQL or CSV Formatted data
- For single SQL Limit the memory of the statement
- export TiDB Snapshot of historical data
precondition
Use Dumpling Backup TiDB Cluster data to GCS front , Make sure you have the following permissions to back up the database :
mysql.tidb
TabularSELECT
andUPDATE
jurisdiction : Before and after backup ,Backup CR You need a database account with this permission , Used to adjust GC Time- SELECT
- RELOAD
- LOCK TABLES
- REPLICATION CLIENT
Ad-hoc Full volume backup
Ad-hoc Full backup by creating a custom Backup
custom resource (CR) Object to describe a backup .TiDB Operator According to this Backup
Object to complete the specific backup process . If an error occurs during the backup , The program will not automatically retry , At this time, it needs to be handled manually .
To better describe how backups are used , This document provides the following backup examples . The example assumes that the pair is deployed in Kubernetes test1
This namespace Medium TiDB colony demo1
Data backup , The following is the specific operation process .
The first 1 Step :Ad-hoc Full backup environment preparation
Download the file backup-rbac.yaml, And execute the following command in
test1
This namespace To create a backup RBAC Related resources :kubectl apply -f backup-rbac.yaml -n test1
Remote storage access authorization .
Reference resources GCS Account Authorization Authorized access GCS Remote storage .
establish
backup-demo1-tidb-secret
secret. The secret Store for access TiDB Clustered root Account and key .kubectl create secret generic backup-demo1-tidb-secret --from-literal=password=${password} --namespace=test1
The first 2 Step : Backup data to GCS
establish
Backup
CR, And back up the data to GCS:kubectl apply -f backup-gcs.yaml
backup-gcs.yaml
The contents of the document are as follows :--- apiVersion: pingcap.com/v1alpha1 kind: Backup metadata: name: demo1-backup-gcs namespace: test1 spec: from: host: ${tidb_host} port: ${tidb_port} user: ${tidb_user} secretName: backup-demo1-tidb-secret gcs: secretName: gcs-secret projectId: ${project_id} bucket: ${bucket} # prefix: ${prefix} # location: us-east1 # storageClass: STANDARD_IA # objectAcl: private # bucketAcl: private # dumpling: # options: # - --threads=16 # - --rows=10000 # tableFilter: # - "test.*" storageClassName: local-storage storageSize: 10Gi
The above example will TiDB The data of the cluster is fully exported and backed up to GCS.GCS The configuration of the
location
、objectAcl
、bucketAcl
、storageClass
Items can be omitted .GCS Storage related configuration reference GCS Storage field introduction .In the example above
.spec.dumpling
Express Dumpling Related configuration , Can be inoptions
Field assignment Dumpling Operation parameters of , For details, see Dumpling Using document ; By default, this field can be configured without . When you don't specify Dumpling The configuration of ,options
The default values of the fields are as follows :options: - --threads=16 - --rows=10000
more
Backup
CR For detailed explanation of fields, please refer to Backup CR Field is introduced .Create good
Backup
CR after , You can view the backup status through the following command :kubectl get bk -n test1 -owide
Scheduled full backup
Users set backup policies to TiDB The cluster performs scheduled backup , At the same time, set the retention policy of backup to avoid too many backups . Scheduled full backup through customized BackupSchedule
CR Object to describe . A full backup will be triggered every time the backup time point , The bottom layer of scheduled full backup passes Ad-hoc Full backup . The following are the specific steps to create a scheduled full backup :
The first 1 Step : Regular full backup environment preparation
Same as Ad-hoc Full backup environment preparation .
The first 2 Step : Back up data in full on a regular basis to GCS
establish
BackupSchedule
CR Turn on TiDB Scheduled full backup of the cluster , Back up the data to GCS:kubectl apply -f backup-schedule-gcs.yaml
backup-schedule-gcs.yaml
The contents of the document are as follows :--- apiVersion: pingcap.com/v1alpha1 kind: BackupSchedule metadata: name: demo1-backup-schedule-gcs namespace: test1 spec: #maxBackups: 5 #pause: true maxReservedTime: "3h" schedule: "*/2 * * * *" backupTemplate: from: host: ${tidb_host} port: ${tidb_port} user: ${tidb_user} secretName: backup-demo1-tidb-secret gcs: secretName: gcs-secret projectId: ${project_id} bucket: ${bucket} # prefix: ${prefix} # location: us-east1 # storageClass: STANDARD_IA # objectAcl: private # bucketAcl: private # dumpling: # options: # - --threads=16 # - --rows=10000 # tableFilter: # - "test.*" # storageClassName: local-storage storageSize: 10Gi
After the scheduled full backup is created , You can view the status of scheduled full backup through the following commands :
kubectl get bks -n test1 -owide
Check all the backup pieces below the scheduled full backup :
kubectl get bk -l tidb.pingcap.com/backup-schedule=demo1-backup-schedule-gcs -n test1
From the above example ,backupSchedule
The configuration of consists of two parts . Part of it is backupSchedule
Unique configuration , The other part is backupTemplate
.backupTemplate
Specify the configuration related to cluster and remote storage , Fields and Backup CR Medium spec
equally , Please refer to Backup CR Field is introduced .backupSchedule
For specific introduction of unique configuration items, please refer to BackupSchedule CR Field is introduced .
Be careful
TiDB Operator Will create a PVC, This PVC At the same time Ad-hoc Full backup and scheduled full backup , The backup data will be stored in PV, Then upload to remote storage . If you want to delete this after the backup PVC, You can refer to Delete resources First back up Pod Delete , Then take it. PVC Delete .
If the backup and upload to the remote storage are successful ,TiDB Operator Will automatically delete the local backup file . If the upload fails , Then the local backup file will be preserved .
边栏推荐
猜你喜欢
Some common software related
Tencent cloud native database tdsql-c was selected into the cloud native product catalog of the Academy of communications and communications
浅谈网络安全之文件上传
Implementation of map and set
Codeworks 5 questions per day (1700 average) - day 7
2022年上半年HIT行业TOP50
【OA】Excel 文档生成器: Openpyxl 模块
SQL injection -day15
你心目中的数据分析 Top 1 选 Pandas 还是选 SQL?
A 股指数成分数据 API 数据接口
随机推荐
Redis源码学习(30),字典学习,dict.h
MySQL的索引
[dpdk] dpdk sample source code analysis III: dpdk-l3fwd_ 001
太方便了,钉钉上就可完成代码发布审批啦!
Create commonly used shortcut icons at the top of the ad interface (menu bar)
Ggplot facet detail adjustment summary
再AD 的 界面顶部(菜单栏)创建常用的快捷图标
1.19.11.SQL客户端、启动SQL客户端、执行SQL查询、环境配置文件、重启策略、自定义函数(User-defined Functions)、构造函数参数
史上最全MongoDB之部署篇
QT thread and other 01 concepts
idea gradle lombok 报错集锦
API data interface of A-share index component data
leetcode:面试题 17.24. 子矩阵最大累加和(待研究)
Hongmi K40S root gameplay notes
使用 BR 备份 TiDB 集群到 GCS
机械臂速成小指南(十):可达工作空间
When QT uses qtooltip mouse to display text, the picture of the button will also be displayed and the prompt text style will be modified
Gpt-3 is a peer review online when it has been submitted for its own research
Arduino droplet detection
Collection of idea gradle Lombok errors