当前位置：网站首页>[tools for quickly creating MySQL stand-alone and cluster test environments] dbdeployer

[tools for quickly creating MySQL stand-alone and cluster test environments] dbdeployer

2022-06-13 07:37:00 【Fish is not fish】

1. background

Do you have such trouble , In the online MySQL When the cluster has problems , You need to test some in a test environment
thing , But a single machine cluster is built , Especially the test cluster or MGR In clusters , Setting up the environment requires
It takes a lot of time , And then recycle , Moreover, the number of machines is not enough , All in all, these operations are trivial , so much trouble ,
So here I recommend a very useful tool DBdeployer

2. Tool introduction

DBdeployer Is an easy and rapid deployment MySQL Database server tools , Originally from mysql-sandbox
It was later used go Language rewriting mysql-sandbox, The follow-up has become the present DBdeployer.
Currently through DBdeployer You can quickly create the following databases of so many versions and types ：

Community Edition MySQL
Percona MySQL
MySQL NDB Cluster
MySQL-SHELL
TiDB
Official address ：dbdeployer

3. install

# Download address ( Download the latest version below )
https://github.com/datacharmer/dbdeployer/releases

# Download steps 
#1. Download installation package 
wget https://github.com/datacharmer/dbdeployer/releases/download/v1.64.0/dbdeployer-1.64.0.linux.tar.gz

#2. decompression 
tar -xvf dbdeployer-1.62.0.linux.tar.gz

#3. Move the extracted executable file to /usr/bin/ Next 
mv dbdeployer-1.64.0.linux /usr/bin/dbdeployer

#4. Check whether the installation is successful 
dbdeployer --version

4. To configure

dbdeployer There are two ways to initialize ：
1. The system generates by default （ Two default directories will be created ）
2. Specify your own directory generation
The first one is （ Default generation method ）:

# initialization ( Automatically create the corresponding directory ) The two directories are for data files and mysql The address of the package file 
dbdeployer init
# The previous command generates $HOME/sandboxes and $HOME/opt/mysql Catalog

The second kind （ Specify location generation ）:

#1. initialization 
$ dbdeployer init --sandbox-binary sandboxes/mysql_binary --sandbox-home sandboxes/mysql_home
# You will see the following output , It will replace the default directory with the one specified by itself 
SANDBOX_BINARY /root/sandboxes/mysql_binary
SANDBOX_HOME   /root/sandboxes/mysql_home

--------------------------------------------------------------------------------
Directory /root/sandboxes/mysql_binary ($SANDBOX_BINARY) was created
This directory is the destination for expanded tarballs

--------------------------------------------------------------------------------
Directory /root/sandboxes/mysql_home ($SANDBOX_HOME) was created
This directory is the destination for deployed sandboxes

--------------------------------------------------------------------------------
Updating defaults for directory /root/sandboxes/mysql_binary ($SANDBOX_BINARY)
# dbdeployer defaults update sandbox-binary /root/sandboxes/mysql_binary
# Updated sandbox-binary -> "/root/sandboxes/mysql_binary"

--------------------------------------------------------------------------------
Updating defaults for directory /root/sandboxes/mysql_home ($SANDBOX_HOME)
# dbdeployer defaults update sandbox-home /root/sandboxes/mysql_home
# Updated sandbox-home -> "/root/sandboxes/mysql_home"

--------------------------------------------------------------------------------
# dbdeployer downloads get mysql-8.0.27-linux-glibc2.17-x86_64-minimal.tar.xz
·····
#2. Pay attention to manual stop here , Otherwise, I will download all mysql Installation package , We can customize and select our installation package later

5. upgrade

I use the latest in the article 1.64 Version of , However, if there is a version update, you only need to execute the following command to solve it

dbdeployer update

6. Use

Because we didn't choose to install at the beginning of initialization MySQL All of the tar package , So in operation mysql Before
We need to download mysql Installation package

6.1 Download installation package

# See which versions are supported 
$dbdeployer downloads list

Available tarballs  ()
                              name                                 OS     version     flavor        size   minimal
---------------------------------------------------------------- ------- --------- ------------- -------- ---------
 Percona-Server-8.0.20-11-Linux.x86_64.glibc2.12-minimal.tar.gz   Linux    8.0.20   percona       103 MB   Y
 Percona-Server-8.0.21-12-Linux.x86_64.glibc2.12-minimal.tar.gz   Linux    8.0.21   percona       104 MB   Y
 Percona-Server-8.0.22-13-Linux.x86_64.glibc2.17-minimal.tar.gz   linux    8.0.22   percona       107 MB   Y
 Percona-Server-8.0.23-14-Linux.x86_64.glibc2.17-minimal.tar.gz   Linux    8.0.23   percona       108 MB   Y
 Percona-Server-8.0.26-16-Linux.x86_64.glibc2.12-minimal.tar.gz   linux    8.0.26   percona       104 MB   Y
 mysql-4.1.22.tar.xz                                              Linux    4.1.22   mysql         4.6 MB   Y
 mysql-5.0.96-linux-x86_64-glibc23.tar.gz                         Linux    5.0.96   mysql         127 MB
 mysql-5.0.96.tar.xz                                              Linux    5.0.96   mysql         5.5 MB   Y
 mysql-5.1.72.tar.xz                                              Linux    5.1.72   mysql          10 MB   Y
 mysql-5.1.73-linux-x86_64-glibc23.tar.gz                         Linux    5.1.73   mysql         134 MB
 mysql-5.5.61-linux-glibc2.12-x86_64.tar.gz                       Linux    5.5.61   mysql         199 MB
 mysql-5.5.61.tar.xz                                              Linux    5.5.61   mysql         6.6 MB   Y
 mysql-5.5.62-linux-glibc2.12-x86_64.tar.gz                       Linux    5.5.62   mysql         199 MB
 mysql-5.5.62.tar.xz                                              Linux    5.5.62   mysql         6.6 MB   Y
 mysql-5.6.43-linux-glibc2.12-x86_64.tar.gz                       Linux    5.6.43   mysql         329 MB
 mysql-5.6.43.tar.xz                                              Linux    5.6.43   mysql         9.0 MB   Y
 mysql-5.6.44-linux-glibc2.12-x86_64.tar.gz                       Linux    5.6.44   mysql         329 MB
 mysql-5.6.44.tar.xz                                              Linux    5.6.44   mysql         9.1 MB   Y
 mysql-5.7.25-linux-glibc2.12-x86_64.tar.gz                       Linux    5.7.25   mysql         645 MB
 mysql-5.7.25.tar.xz                                              Linux    5.7.25   mysql          23 MB   Y
 mysql-5.7.26-linux-glibc2.12-x86_64.tar.gz                       Linux    5.7.26   mysql         645 MB
 mysql-5.7.26.tar.xz                                              Linux    5.7.26   mysql          23 MB   Y
 .....
 I only put a part here , You can find more by yourself 
# Download the corresponding installation package , Here I download the latest version of the package 
#get-unpack  Download the package and unzip it 
$dbdeployer downloads get-unpack Percona-Server-8.0.26-16-Linux.x86_64.glibc2.12-minimal.tar.gz
# Output is as follows 
Downloading Percona-Server-8.0.26-16-Linux.x86_64.glibc2.12-minimal.tar.gz
.........  104 MB
File /root/Percona-Server-8.0.26-16-Linux.x86_64.glibc2.12-minimal.tar.gz downloaded
Checksum matches
Unpacking tarball Percona-Server-8.0.26-16-Linux.x86_64.glibc2.12-minimal.tar.gz to $HOME/sandboxes/mysql_binary/8.0.26
.........100.........200.........300......364
Renaming directory /root/sandboxes/mysql_binary/Percona-Server-8.0.26-16-Linux.x86_64.glibc2.12-minimal to /root/sandboxes/mysql_binary/8.0.26

# Go to the corresponding directory to view , Will automatically put the package in mysql_binary Directory and unzip 
$cd /root/sandboxes/mysql_binary/8.0.26
$ ll
 Total usage  664
drwxr-x---  2 root root   4096 2 month   21 15:40 bin
drwxr-x---  2 root root     69 2 month   21 15:40 cmake
-rw-r--r--  1 root root  17987 2 month   21 15:40 COPYING.GPLv2
-rw-rw-r--  1 root root   1703 2 month   21 15:40 COPYING-jemalloc
drwxr-x---  2 root root    115 2 month   21 15:40 docs
-rw-r-----  1 root root      7 2 month   21 15:40 FLAVOR
drwxr-x---  6 root root   4096 2 month   21 15:40 include
drwxr-x---  8 root root   4096 2 month   21 15:40 lib
-rw-r--r--  1 root root 276551 2 month   21 15:40 LICENSE
-rw-r--r--  1 root root  47658 2 month   21 15:40 LICENSE.router
-rw-r--r--  1 root root 276551 2 month   21 15:40 LICENSE-test
drwxr-x---  4 root root     30 2 month   21 15:40 man
-rw-r--r--  1 root root   1623 2 month   21 15:40 mysqlrouter-log-rotate
-rw-r--r--  1 root root   2211 2 month   21 15:40 PATENTS
-rw-r--r--  1 root root    804 2 month   21 15:40 README
-rw-r--r--  1 root root   4346 2 month   21 15:40 README.md
-rw-r--r--  1 root root    679 2 month   21 15:40 README.router
-rw-r--r--  1 root root    804 2 month   21 15:40 README-test
drwxr-x---  2 root root      6 2 month   21 15:40 run
drwxr-x--- 28 root root   4096 2 month   21 15:40 share
drwxr-x---  2 root root     77 2 month   21 15:40 support-files
drwxr-x---  3 root root     17 2 month   21 15:40 var

# installation is complete

6.2 Common deployment commands

Deploy 1 Master and slave MGR

$ dbdeployer deploy --topology=group replication 8.0.26 --single-primary
# Probably 20 Seconds to create , The output information is as follows 
Installing and starting node 1
.. sandbox server started
Installing and starting node 2
.. sandbox server started
Installing and starting node 3
... sandbox server started
$HOME/sandboxes/mysql_home/group_sp_msb_8_0_26/initialize_nodes
# Node 1 # reset master; CHANGE MASTER TO MASTER_USER='rsandbox', MASTER_PASSWORD='rsandbox' FOR CHANNEL 'group_replication_recovery';
# Node 2 # reset master; CHANGE MASTER TO MASTER_USER='rsandbox', MASTER_PASSWORD='rsandbox' FOR CHANNEL 'group_replication_recovery';
# Node 3 # reset master; CHANGE MASTER TO MASTER_USER='rsandbox', MASTER_PASSWORD='rsandbox' FOR CHANNEL 'group_replication_recovery';

# Node 1 # SET GLOBAL group_replication_bootstrap_group=ON;
# Node 1 # START GROUP_REPLICATION;
# Node 2 # START GROUP_REPLICATION;
# Node 3 # START GROUP_REPLICATION;
# Node 1 # SET GLOBAL group_replication_bootstrap_group=OFF;
# Node 1 # select * from performance_schema.replication_group_members
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| group_replication_applier | 00023627-1111-1111-1111-111111111111 | 127.0.0.1   |       23627 | ONLINE       | PRIMARY     | 8.0.26         |
| group_replication_applier | 00023628-2222-2222-2222-222222222222 | 127.0.0.1   |       23628 | ONLINE       | SECONDARY   | 8.0.26         |
| group_replication_applier | 00023629-3333-3333-3333-333333333333 | 127.0.0.1   |       23629 | ONLINE       | SECONDARY   | 8.0.26         |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
# Node 2 # select * from performance_schema.replication_group_members
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| group_replication_applier | 00023627-1111-1111-1111-111111111111 | 127.0.0.1   |       23627 | ONLINE       | PRIMARY     | 8.0.26         |
| group_replication_applier | 00023628-2222-2222-2222-222222222222 | 127.0.0.1   |       23628 | ONLINE       | SECONDARY   | 8.0.26         |
| group_replication_applier | 00023629-3333-3333-3333-333333333333 | 127.0.0.1   |       23629 | ONLINE       | SECONDARY   | 8.0.26         |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
# Node 3 # select * from performance_schema.replication_group_members
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
| group_replication_applier | 00023627-1111-1111-1111-111111111111 | 127.0.0.1   |       23627 | ONLINE       | PRIMARY     | 8.0.26         |
| group_replication_applier | 00023628-2222-2222-2222-222222222222 | 127.0.0.1   |       23628 | ONLINE       | SECONDARY   | 8.0.26         |
| group_replication_applier | 00023629-3333-3333-3333-333333333333 | 127.0.0.1   |       23629 | ONLINE       | SECONDARY   | 8.0.26         |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+
Group Replication directory installed in $HOME/sandboxes/mysql_home/group_sp_msb_8_0_26
run 'dbdeployer usage multiple' for basic instructions'

Deploy primary 4 The master-slave replication cluster of the slave

$ dbdeployer deploy --topology=master-slave replication --nodes 5 8.0.26
# The information is as follows 
Installing and starting master
.. sandbox server started
Installing and starting slave1
.. sandbox server started
Installing and starting slave2
.. sandbox server started
Installing and starting slave3
.. sandbox server started
Installing and starting slave4
.. sandbox server started
$HOME/sandboxes/mysql_home/rsandbox_8_0_26/initialize_slaves
initializing slave 1
initializing slave 2
initializing slave 3
initializing slave 4
Replication directory installed in $HOME/sandboxes/mysql_home/rsandbox_8_0_26
run 'dbdeployer usage multiple' for basic instructions

Something to watch out for

$dbdeployer deploy  replication  -h
# You can see some usage here 
Usage:
  dbdeployer deploy replication MySQL-Version [flags]

Examples:

		$ dbdeployer deploy replication 5.7    # deploys highest revision for 5.7
		$ dbdeployer deploy replication 5.7.21 # deploys a specific revision
		$ dbdeployer deploy replication /path/to/5.7.21 # deploys a specific revision in a given path
		# (implies topology = master-slave)

		$ dbdeployer deploy --topology=master-slave replication 5.7
		# (explicitly setting topology)

		$ dbdeployer deploy --topology=group replication 5.7
		$ dbdeployer deploy --topology=group replication 8.0 --single-primary
		$ dbdeployer deploy --topology=all-masters replication 5.7
		$ dbdeployer deploy --topology=fan-in replication 5.7
		$ dbdeployer deploy --topology=pxc replication pxc5.7.25
		$ dbdeployer deploy --topology=ndb replication ndb8.0.14

# We need to pay attention to --topology Function of parameters 
# This parameter controls the replication mode of the cluster 
group MGR colony 
all-masters  Multi master cluster 
master-slave  Master slave copy 
ndb ndb Cluster pattern 
pxc  etc. 
 You can also set the semi synchronous replication mode 
 Asynchronous replication, etc 
 For details, you can explore