Distributed database uses logical volume to manage storage expansion

Inspur Yunxi database is a domestic distributed database independently developed by Inspur , One feature is the ability to support PB Level data services . At the storage level , Yunxi database can not only use physical disks , You can also use logical volumes to manage storage . Using logical volumes , There are two main scenarios ：

（1） When large capacity disks are used in business , When a single physical disk cannot meet the requirements . for example ： We need to be in /data Mount 30TB The storage , A single disk does not have this large capacity . If you use logical volumes , Combine multiple small capacity disks into a volume group , Aggregate into one large logical disk , That is to meet the demand .

（2） When the amount of data is uncertain , When you need to expand the disk later . When planning disks early in the business , You don't know exactly how much disk space you need to allocate . If using physical volumes , Later, it cannot be expanded . If you use logical volumes , According to the later demand , Expand and shrink manually .

 

-  Basic concepts  -

Logical volumes are managed using logical volume groups (Logic Volume Manager) Created devices , If you want to understand logical volumes , First of all, we need to understand some concepts in logical volume management .

1. Logical volume manager （Logical Volume Manager,LVM）：LVM To logically assemble the partitions of one or more hard disks , It's equivalent to a big hard disk to use , When the hard disk is out of space , You can continue to add other hard disk partitions to it , In this way, the dynamic management of disk space can be realized , Compared with ordinary disk partition, it has great flexibility .LVM The system administrator can allocate storage space for applications and users more conveniently .

2. Physical volume （Physical Volume,PV）： Refers to a hard disk partition or a device that logically has the same function as a disk partition （ Such as RAID）, yes LVM Block of basic storage logic .

3. The volume group （Volume Group,VG）：PV Set . Is a storage pool composed of one or more physical volumes , One or more logical volumes can be created on a volume group .

4. Logic volume （Logic Volume,LV）：VG A logical disk drawn in . It's similar to not LVM Hard disk partition in the system , It is based on volume groups , Is a standard block device , File systems can be built on top of logical volumes .

Convert a physical disk or disk partition to a physical volume , One or more physical volumes are aggregated to form one or more volume groups , A logical volume is a piece of disk space abstracted from a volume group . The specific structure is as follows ：

 

 

- establish LVM -

 

1. First, through fdisk -l or lsblk View the properties of the disk , Find the name of the disk you want to add .

vdc Is a disk manually attached to the virtual machine , Execute the order as follows ：

[email protected]:~# lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTvda 252:0 0 40G 0 disk |-vda1 252:1 0 571M 0 part /boot`-vda2 252:2 0 39.5G 0 part /vdb 252:16 0 64M 0 disk [email protected]:~# lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTvda 252:0 0 40G 0 disk |-vda1 252:1 0 571M 0 part /boot`-vda2 252:2 0 39.5G 0 part /vdb 252:16 0 64M 0 disk vdc 252:32 0 10G 0 disk [email protected]:~# fdisk -lDisk /dev/vda: 40 GiB, 42949672960 bytes, 83886080 sectorsUnits: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisklabel type: dosDisk identifier: 0x9b600047Device Boot Start End Sectors Size Id Type/dev/vda1 * 2048 1171455 1169408 571M 83 Linux/dev/vda2 1171456 83886046 82714591 39.5G 83 LinuxDisk /dev/vdb: 64 MiB, 67108864 bytes, 131072 sectorsUnits: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisklabel type: dosDisk identifier: 0x00000000Disk /dev/vdc: 10 GiB, 10737418240 bytes, 20971520 sectorsUnits: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytes

 

2.  Use fdisk Partition the disk logically

fdisk /dev/vdc

m To see command help ;

n Add a new partition, Partition , And the size of the partition , Default values are used here , Create a partition ;

w Command save and exit .

[email protected]:~# fdisk /dev/vdcWelcome to fdisk (util-linux 2.31.1).Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition table.Created a new DOS disklabel with disk identifier 0xbb7719f5.Command (m for help): nPartition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions)Select (default p): Using default response p.Partition number (1-4, default 1): First sector (2048-20971519, default 2048): Last sector, +sectors or +size{K,M,G,T,P} (2048-20971519, default 20971519): Created a new partition 1 of type 'Linux' and of size 10 GiB.Command (m for help): wThe partition table has been altered.Calling ioctl() to re-read partition table.Syncing disks.[email protected]:~# fdisk -l......Device Boot Start End Sectors Size Id Type/dev/vdc1        2048 20971519 20969472  10G 83 Linux

 

3.  Use lvm To manage this partition

1). Installation is required first lvm

apt-get install lvm2

2). establish Physical Volume（PV）

pvcreate /dev/vdc1

After success, you can pass pvdisplay Check the information

[email protected]:~# pvcreate /dev/vdc1 Physical volume "/dev/vdc1" successfully created.[email protected]:~# pvdisplay "/dev/vdc1" is a new physical volume of "<10.00 GiB" --- NEW Physical volume --- PV Name /dev/vdc1 VG Name  PV Size <10.00 GiB Allocatable NO PE Size 0  Total PE 0 Free PE 0 Allocated PE 0  PV UUID               5FxevL-aneV-Xhe6-pFEv-b3pH-l4v4-Gb6TeA

 

3). establish Volume Group (VG).

vgcreate vg-newsql /dev/vdc1

[email protected]:~# vgcreate vg-newsql /dev/vdc1 Volume group "vg-newsql" successfully created[email protected]:~# vgscan Reading volume groups from cache.  Found volume group "vg-newsql" using metadata type lvm2

 

4).  establish Logical Volumes (LV).

lvcreate -n lv-newsql -l 100%VG vg-newsql( Will all vg-newsql All the memory for this lv-newsql Logic volume )

[email protected]:~# lvcreate -n lv-newsql -l 100%VG vg-newsql Logical volume "lv-newsql" created.[email protected]:~# lvdisplay --- Logical volume --- LV Path /dev/vg-newsql/lv-newsql LV Name lv-newsql VG Name vg-newsql LV UUID bMsyAX-xatH-cdTY-JbzE-4ao5-QR6h-j4Z7ON LV Write Access read/write LV Creation host, time newsqltest, 2021-01-27 16:09:54 +0800 LV Status available # open 0 LV Size <10.00 GiB Current LE 2559 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256  Block device           253:0

 

4.  Format and mount logical volumes to /data

1). The first step is to format the logical volume

mkfs.ext4 /dev/vg-newsql/lv-newsql

2). Mount the logical volume to home Under the folder

mount /dev/vg-newsql/lv-newsql /data

[email protected]:~# mkfs.ext4 /dev/vg-newsql/lv-newsqlmke2fs 1.44.1 (24-Mar-2018)Creating filesystem with 2620416 4k blocks and 655360 inodesFilesystem UUID: a667f950-9b91-45ce-9931-8fb147173054Superblock backups stored on blocks:  32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): doneWriting superblocks and filesystem accounting information: done [email protected]:~# mount /dev/vg-newsql/lv-newsql /datamount: /data: mount point does not exist.[email protected]:~# mkdir /data[email protected]:~# mount /dev/vg-newsql/lv-newsql /data[email protected]:~# df -hFilesystem Size Used Avail Use% Mounted onudev 3.9G 0 3.9G 0% /devtmpfs 798M 3.2M 795M 1% /run/dev/vda2 39G 2.8G 35G 8% /tmpfs 3.9G 0 3.9G 0% /dev/shmtmpfs 5.0M 0 5.0M 0% /run/locktmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup/dev/vda1 547M 77M 430M 16% /boottmpfs 798M 0 798M 0% /run/user/0/dev/mapper/vg--newsql-lv--newsql  9.8G   37M  9.3G   1% /data

thus ,10G The disk of is mounted to by volume group /data Catalog .

- Capacity expansion -

 

1. Add disks ：

[email protected]:~# lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTvda 252:0 0 40G 0 disk |-vda1 252:1 0 571M 0 part /boot`-vda2 252:2 0 39.5G 0 part /vdb 252:16 0 64M 0 disk vdc 252:32 0 10G 0 disk `-vdc1 252:33 0 10G 0 part `-vg--newsql-lv--newsql 253:0 0 10G 0 lvm /datavdd                       252:48   0   10G  0 disk

 

2. Format disk

1). When the disk size is less than 2TB When , And create LVM Of the 2 The steps are the same ：

fdisk /dev/vdd

m To see command help

n Add a new partition, Partition , And the size of the partition , I use the default values here , Create a partition

w Command save and exit

2） When more than 2TB When , Use the following command ：

a.  edit /dev/sda disk

parted /dev/vde

b.  Set the partition type

mklabel gpt

c.  Set the default unit to TB、GB

unit TB

d.  Create partition size

mkpart primary 0 0 or mkpart primary 0.00TB 2.10TB

e. quit

f.  Format disk

mkfs.ext4 /dev/vde

[email protected]:~# parted /dev/vdeGNU Parted 3.2Using /dev/vdeWelcome to GNU Parted! Type 'help' to view a list of commands.(parted) help  align-check TYPE N check partition N for TYPE(min|opt) alignment help [COMMAND] print general help, or help on COMMAND mklabel,mktable LABEL-TYPE create a new disklabel (partition table) mkpart PART-TYPE [FS-TYPE] START END make a partition name NUMBER NAME name partition NUMBER as NAMEprint [devices|free|list,all|NUMBER] display the partition table, available devices, free space, all found partitions, or a particular partition quit exit program rescue START END rescue a lost partition near START and END resizepart NUMBER END resize partition NUMBER rm NUMBER delete partition NUMBER select DEVICE choose the device to edit disk_set FLAG STATE change the FLAG on selected device disk_toggle [FLAG] toggle the state of FLAG on selected deviceset NUMBER FLAG STATE change the FLAG on partition NUMBER toggle [NUMBER [FLAG]] toggle the state of FLAG on partition NUMBER unit UNIT set the default unit to UNIT version display the version number and copyright information of GNU Parted(parted) mklabel gpt (parted) unit TB(parted) mkpart primary 0 0 or mkpart primary 0.00TB 2.10TB  align-check TYPE N check partition N for TYPE(min|opt) alignment help [COMMAND] print general help, or help on COMMAND mklabel,mktable LABEL-TYPE create a new disklabel (partition table) mkpart PART-TYPE [FS-TYPE] START END make a partition name NUMBER NAME name partition NUMBER as NAMEprint [devices|free|list,all|NUMBER] display the partition table, available devices, free space, all found partitions, or a particular partition quit exit program rescue START END rescue a lost partition near START and END resizepart NUMBER END resize partition NUMBER rm NUMBER delete partition NUMBER select DEVICE choose the device to edit disk_set FLAG STATE change the FLAG on selected device disk_toggle [FLAG] toggle the state of FLAG on selected deviceset NUMBER FLAG STATE change the FLAG on partition NUMBER toggle [NUMBER [FLAG]] toggle the state of FLAG on partition NUMBER unit UNIT set the default unit to UNIT version display the version number and copyright information of GNU PartedWarning: You requested a partition from 0.00TB to 2.10TB (sectors 0..4101562500).The closest location we can manage is 0.00TB to 0.00TB (sectors 34..2047).Is this still acceptable to you?Yes/No? yes Warning: The resulting partition is not properly aligned for best performance.Ignore/Cancel? ignore (parted) quit Information: You may need to update /etc/fstab.

[email protected]:~# mkfs.ext4 /dev/vde mke2fs 1.44.1 (24-Mar-2018)Found a gpt partition table in /dev/vdeProceed anyway? (y,N) yCreating filesystem with 550502400 4k blocks and 137625600 inodesFilesystem UUID: a08b03f2-05b7-4029-b317-0e07f93fb6d2Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000, 214990848, 512000000Allocating group tables: done Writing inode tables: done Creating journal (262144 blocks): doneWriting superblocks and filesystem accounting information: done 

 

3.  establish pv volume ,  And use lvm To manage the creation of Physical Volume The steps are the same ：

pvcreate /dev/vdd1

 

4.  Expand the physical hard disk to the same volume group , Each group manages different physical disks ：

vgextend vg-newsql /dev/vdd1

[email protected]:~# vgextend vg-newsql /dev/vdd1  Volume group "vg-newsql" successfully extended

 

5. Allocate space from volume group to logical volume ：

lvextend -l +100%FREE /dev/mapper/vg--newsql-lv--newsql（ Allocate space to the logical volume ）

[email protected]:~# lvextend -l +100%FREE /dev/mapper/vg--newsql-lv--newsql Size of logical volume vg-newsql/lv-newsql changed from <10.00 GiB (2559 extents) to 19.99 GiB (5118 extents).  Logical volume vg-newsql/lv-newsql successfully resized.

 

6. Restart the logical volume for it to take effect ：

resize2fs /dev/mapper/vg--newsql-lv--newsql

[email protected]:~# resize2fs /dev/mapper/vg--newsql-lv--newsqlresize2fs 1.44.1 (24-Mar-2018)Filesystem at /dev/mapper/vg--newsql-lv--newsql is mounted on /data; on-line resizing requiredold_desc_blocks = 2, new_desc_blocks = 3The filesystem on /dev/mapper/vg--newsql-lv--newsql is now 5240832 (4k) blocks long.

 

7. see ：

df -h When the logical volume is found （resize2fs /dev/mapper/vg--newsql-lv--newsql） When the space increases , Indicates successful addition .

[email protected]:~# df -hFilesystem Size Used Avail Use% Mounted onudev 3.9G 0 3.9G 0% /devtmpfs 798M 3.2M 795M 1% /run/dev/vda2 39G 2.8G 35G 8% /tmpfs 3.9G 0 3.9G 0% /dev/shmtmpfs 5.0M 0 5.0M 0% /run/locktmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup/dev/vda1 547M 77M 430M 16% /boot/dev/mapper/vg--newsql-lv--newsql   20G   44M   19G   1% /data