Discussion on beegfs high availability mode

Recently, we are testing beegfs, It's in hpc A widely used parallel file system , But the only way it can ensure data security is mirror One way , This method cannot reduce the effective utilization of disks . The pattern is shown in the figure below

netapp Under the disk sharing architecture , After a single node is disconnected, the effective access to the service can still be guaranteed , This is because their hard disks are actually shared to nodes through storage array cabinets , Therefore, after one of the nodes is disconnected , The other node can still access the hard disk of the disconnected node through the storage array cabinet . But it's not just the ability to access the hard disk that can be added to beegfs In service , There are still some operations to be done . I had a test , When I add the hard disk of the offline node to the online node , The following documents need to be modified .

First, you need to mount the hard disk of the offline node

mount /dev/lose-disk1 /mnt/data*


After mounting , Need modification /etc/beegfs/beegfs-storage.conf The configuration file , Major changes

storeStorageDirectory and storeFsUUID, The attached directory of the dropped node and UUID add to
storeStorageDirectory = ,/mnt/data1 ,/mnt/data2 ,/mnt/data3 ,/mnt/data4
storeFsUUID = ,1c80fdd1-4e81-4463-80fb-bd948265e98d ,050c19f1-0fd9-4725-b21c-88fe13a37f5c ,1e3392ce-b274-4717-979e-b4ac183c812f ,d4f87f8b-b7a3-4297-9824-4fef30e90920


Then you need to go to the mount directory , modify nodeNumID and originalNodeID

nodeNumID To modify the node that is still alive ID, Such as node01 yes 1,node02 yes 2,node02 drops ,node02 The hard disk on the will be changed to 1;originalNodeID Then you need to change the offline node to a live node , Such as node02 It is amended as follows node01

Finish the above operation , Try

systemctl restart beegfs-storage

, Can be restored .

So we can write a script , Automate the above operations , Can achieve a similar effect