Intensive use of glusterfs 4.1

1 Basic commands

gluster peer probe	 Add a node 
gluster peer detach	 Remove node 
gluster volume create	 Create a volume 
gluster volume start $VOLUME_NAME	 Boot volume 
gluster volume stop $VOLUME_NAME	 Stop the volume  
gluster volume delete $VOlUME_NAME	 Delete volume 
gluster volume quota enable	 Open volume quota 
gluster volume quota disable	 Turn off volume quota 
gluster volume quota limitusage	 Set volume quota 

2 Increase replica and disk balance

2.1 Dynamically add replicas

glusterfs You can dynamically add replicas , But because of the hash Method to dynamically calculate the data storage location , Therefore, the newly added replica will not have data , You need to manually perform disk data balancing

#  Add new copy , If 3 Multiple 
gluster volume add-brick gv1 replica 3 \
  gf-node01:/data/brick3/gv1 \
  gf-node02:/data/brick3/gv1 \
  gf-node03:/data/brick3/gv1 \
  force
  
#  View resource pool information 
[[email protected] ~]# gluster volume info gv1 
Volume Name: gv1
Type: Distributed-Replicate
Volume ID: e1e004fa-5588-4629-b7ff-048c4e17de91
Status: Started
Snapshot Count: 0
Number of Bricks: 3 x 3 = 9
Transport-type: tcp
Bricks:
Brick1: gf-node01:/data/brick1/gv1
Brick2: gf-node01:/data/brick2/gv1
Brick3: gf-node02:/data/brick1/gv1
Brick4: gf-node02:/data/brick2/gv1
Brick5: gf-node03:/data/brick1/gv1
Brick6: gf-node03:/data/brick2/gv1
Brick7: gf-node01:/data/brick3/gv1
Brick8: gf-node02:/data/brick3/gv1
Brick9: gf-node03:/data/brick3/gv1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

#  Look at the data 
[[email protected] ~]# ls /data/brick*/*
/data/brick1/gv1:
test1  test2  test4  test5  test8  test9
/data/brick2/gv1:
test1  test2  test4  test5  test8  test9
/data/brick3/gv1:

2.2 Reason and operation of disk balancing

The layout structure is static , When new bricks Join existing volumes , The newly created files are distributed to the old bricks in , So we need to balance the layout structure , New to bricks take effect . Layout balancing just makes the new layout effective , It doesn't move old data in the new layout , You also need to balance the data in the volume .

2.2.1 Write data and view position before layout balancing

[[email protected] ~]# touch /mnt/aaa{1..9}
[[email protected] ~]# ls /data/brick*/*
/data/brick1/gv1:
aaa2  aaa5  aaa8  test1  test2  test4  test5  test8  test9

/data/brick2/gv1:
aaa2  aaa5  aaa8  test1  test2  test4  test5  test8  test9

/data/brick3/gv1:

2.2.2 Write after the disk layout is balanced

[[email protected] ~]# gluster volume rebalance gv1 fix-layout start
volume rebalance: gv1: success: Rebalance on gv1 has been started successfully. Use rebalance status command to check status of the rebala
nce process.ID: eeec5c76-88c7-4793-8074-25565270c128

[[email protected] ~]# touch /mnt/bbb{1..9}
[[email protected] ~]# ls /data/brick*/*
/data/brick1/gv1:
aaa2  aaa5  aaa8  bbb3  bbb8  bbb9  test1  test2  test4  test5  test8  test9
/data/brick2/gv1:
aaa2  aaa5  aaa8  bbb3  bbb8  bbb9  test1  test2  test4  test5  test8  test9
/data/brick3/gv1:

2.2.3 Check again after disk storage balancing

[[email protected] ~]# gluster volume rebalance gv1 start
volume rebalance: gv1: success: Rebalance on gv1 has been started successfully. Use rebalance status command to check status of the rebala
nce process.ID: 5189b20c-6120-40d7-8e74-0ba114130a24

[[email protected] ~]# ls /data/brick*/*
/data/brick1/gv1:
aaa1  aaa2  aaa3  aaa5  aaa8  bbb3  bbb5  bbb8  bbb9  test2  test3  test4  test8
/data/brick2/gv1:
aaa1  aaa2  aaa3  aaa5  aaa8  bbb3  bbb5  bbb8  bbb9  test2  test3  test4  test8
/data/brick3/gv1:
aaa2  aaa4  aaa5  aaa6  aaa7  aaa8  aaa9  bbb1  bbb2  bbb3  bbb4  bbb6  bbb7  bbb8  bbb9  test2  test4  test6  test7  test8

After each expansion, you need to perform a disk balancing . Balancing a large amount of data wastes performance , Balance time needs to be planned in advance

3 glustefs Distributed storage optimization

You can refer to

 Command format ：
glustervolume set < volume >< Parameters >
 for example ：
# Open read ahead mode to access storage 
[[email protected] ~]# gluster volume set gv2 performance.read-ahead on
# Adjust the size of the read cache 
[[email protected] gv2]# gluster volume set gv2 performance.cache-size 256M

Auth_allow		#IP Access authorization ; The default value （*.allow all）; Legal value ：Ip Address 
Nfs.disabled	# close NFS service ; The default value （Off）; Legal value ：Off|on
Network.frame-timeout	# Request waiting time ; The default value （1800s）; Legal value ：1-1800
Network.ping-timeout	# Client wait time ; The default value （42s）; Legal value ：0-42
Cluster.min-free-disk	# Remaining disk space threshold ; The default value （10%）; Legal value ： percentage 
Performance.cache-size	# Read cache size ; The default value （32MB）; Legal value ： byte 
Performance.quick-read	# Optimize the performance of reading small files 
Performance.read-ahead	# Improve the reading performance by pre reading 
Performance.io-cache	# Cache already read 、
Performance.write-behind	# Write to the cache first , Writing to hard disk , To improve write performance .
Performance.io-thread-count	#IO Number of threads ; The default value （16）; Legal value ：0-65
Performance.cache-refresh-timeout	# Cache verification time ; The default value （1s）; Legal value ：0-61