Home NAS server (4) | MergerFS and SnapRaid data backup

家用 NAS 服务器（4）| MergerFS和SnapRaid数据备份

原文地址：CSDN博文

前文提要：家用 NAS 服务器（3）| SSD缓存加速机械硬盘

---

---

1. 前言

上一期中,笔者已经把 ubuntu server 的 xfs 文件系统以及 bcache Solid state cache Settings to complete.

This issue is still a full-on,干货满满.I will set up MergerFS Multiple hardpoints unity,之后再设置 SnapRaid 定时备份,The final end of the file system Settings.

哇,Finally ending the hard disk and file system,如释重负,可喜可贺 (●’◡’●).

---

2. Other relative plan

在进入 mergerfs + SnapRaid The combination of before,To explain what the author understands"数据备份"Other solutions：

• RAID：
  • 传统的 Raid：Raid1、Raid5 和 Raid10 都Can do single disk recovery,But expanding usually limited
  • btrfs 文件系统：Can provide the file system level raid 功能,Have certain can expand sex,And have a snapshot backup function
  • zfs 文件系统：的 raidz 功能,也能Provide good fault tolerance,But expanding less than.
  • LVM：LVM 支持 raid 功能,But also face expanding problem
• rsync：基于 rsync Way to backup tool,例如 Timeshift,Can back up your data on a regular basis.

Due to the hard drive after going to the increase of one by one,So the need to consider expanding,只有 btrfs 系统以及 rsync Way can meet the demand,But the two have their own disadvantages：

• btrfs 提供的 raid Function while it is possible to expand the number of disk,但是在The balance of data migration time need takes long
• btrfs 会Additional performance,See the author of the article：ext4、ntfs、xfs、btrfs、zfs、f2fs和reiserFS文件系统性能对比
• Raid 通常会Lead to more disk read and write at the same time,Power consumption big.
• rsync Not suitable for multiple disk system backup,尽管可以使用 mergerfs Make it up,But not pure

Considering the disadvantages of the above,The author finally decided to use mergerfs Carries on the data plate of uniform,并用 SnapRaid For data backup regularly.

---

3. MergerFS

3.1 基本介绍

mergerfs 是一个“融合型”的文件系统,Its purpose is mainly used for the file management of unity on the different storage device together,类似于 mhddfs、unionfs 和 aufs File system such as fusion.

mergerfs 的特性如下：

• Configurable file distribution behavior
• Follow one's inclinations of the add or remove the file system
• A separate file system collapse can be fault tolerant
• Support expanding features xattrs
• Support file features chattr
• 运行时配置（通过 xattrs）
• Can work on a variety of heterogeneous file system
• If there is insufficient space on the file system when writing,Can automatically transfer files
• When written to ignore the read-only file system
• The read-only file into a symbolic links to actual files
• Hard links when writing copy（CoW）机制
• 支持 POSIX ACLs
• … …

总的来说,mergerfs Can merge multiple paths in logic in the same path,Similar to the collection set with the operation.All file operations under the unified path,Will be scattered to the lower level of the file system automatically performed in the.

一个 mergerfs Working example is as follows：

A         +      B        =       C
/disk1           /disk2           /merged
|                |                |
+-- /dir1        +-- /dir1        +-- /dir1
|   |            |   |            |   |
|   +-- file1    |   +-- file2    |   +-- file1
|                |   +-- file3    |   +-- file2
+-- /dir2        |                |   +-- file3
|   |            +-- /dir3        |
|   +-- file4        |            +-- /dir2
|                     +-- file5   |   |
+-- file6                         |   +-- file4
                                  |
                                  +-- /dir3
                                  |   |
                                  |   +-- file5
                                  |
                                  +-- file6

使用 mergerfs 需要注意：

• mergerfs 不支持 aufs 和 overlayfs 中的写时复制（CoW）Or white.
• mergerfs Can't mount a read-only file system and to write.然而,在创建新文件时,mergerfs Ignores read-only drive,So you can read and write mix and read-only drive.
• mergerfs Don't split between each drive data.它不是RAID0/条带化.It's just other file system of a consortium.

3.2 安装使用

先安装 mergerfs：

sudo apt -y install mergerfs

Then multiple mount point（以 /mnt/d1 和 /mnt/d2 为例）临时Mount to the unified（以 /mnt/data 为例）：

mergerfs -o allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs \
  /mnt/d1:/mnt/d2 /mnt/data

然后,You also need to add the configuration to /etc/fstab 中,In order to realize automatically boot：

... ...
# mergerfs
/mnt/d1:/mnt/d2 /mnt/data fuse.mergerfs defaults,use_ino,cache.files=partial,dropcacheonclose=true,allow_other,category.create=mfs 0 0

大功告成,之后对 /mnt/data Write data,The data will automatically spread the /mnt/d1 和 /mnt/d2 中,The equivalent of two partitions on the logic and as one.

---

4. SnapRaid

4.1 基本介绍

SnapRAID Is a disk array backup programs.It stores the data parity information,And can recover from as many as six disk failure.

SnapRAID It is mainly used for family environment,Especially in have a lot of large documents rarely change under the environment of.

In addition to being able to recover from a disk failure outside,SnapRAIDOther features are：

1. All the data are hash,以确保数据的完整性,To avoid damage of silent.
2. If too many failed disk can't restore,那么你Only lose the data on the disk failure.All the data in the other disk is safe.
3. If you accidentally deleted some of a disk file,你可以恢复它们.
4. You can start already full disk.
5. The size of the disk can have different.
6. You can add disk at any time.
7. It won't lock your data.You can stop at any time using theSnapRAID,而Don't need to reformat or mobile data.
8. To access a file,Only need a disk rotation,节省电力,The noise is small.

SnapRaid 的工作原理十分简单,The reserved a calibration plate,To store all the rest of the disk data generated by the calibration value.SnapRaid Can check plate calibration values are stored,以类似 Raid5 的方式,For all the other data“备份”功能.值得注意的是,SnapRaid 并Not updated in real time calibration plate data,But need to manually set the timing task to update the calibration plate.

From the above list of features to see,SnapRaid 相较于传统的 Raid 阵列,Limits are small,And easy to expand,At the same time save power,Therefore suitable for family use this low burden of environment.

4.2 安装使用

This section part reference since：SnapRAID - ArchWiki.

Ubuntu Official sources have joined SnapRaid,可直接安装：

sudo apt install snapraid

Assuming that the data here plate mount point for /mnt/data1 和 /mnt/data2,And calibration plate mount point for /mnt/snapraid_backup,则在 /etc/snapraid.conf 文件中加入以下内容：

disk d1 /mnt/data1
disk d2 /mnt/data2
parity /mnt/snapraid_backup/SnapRAID.parity
content /mnt/data1/SnapRAID.content
content /mnt/data2/SnapRAID.content
content /var/snapraid/SnapRAID.content
exclude /lost+found/

The above configuration file as follows：

• disk Specifies the directory in which the required backup,此处 d1 和 d2 为任意名称,并指定数据目录 /mnt/data1 和 /mnt/data2
• parity Check specified file location,此处放在 /mnt/snapraid_backup.parity 可以指定多个文件,To do several backup,提高容错性
• content The specified file index such as backup needed meta information,You can specify multiple location to prevent lost

注意：SnapRaid 的 parity Files need to be placed onThe remaining space biggest disk中,Otherwise unable to store all data plate calibration value

And then you can start进行备份：

snapraid sync

Restore files or folders,FILENAME Must dish in the absolute path for the data：

snapraid fix -f FILENAME

4.3 数据还原

如果此时 /mnt/data2 The corresponding data plate hung up the,Then perform data reduction before,修改 etc/snapraid.conf：

disk d2 /mnt/data2

为

disk d2 /mnt/data2_new

其中 /mnt/data2_new For the new added data plate mount point,After reduction：

snapraid -d d2 -l recovery.log fix

To restore data will be lost.

4.4 定时备份

Regular backup is an important means to guarantee data integrity,So this section will in detail SnapRaid Timing of backup Settings.

此外,Regular inspection data can be effective to avoid contamination of the backup data,这个过程被称作 scrub.

4.4.1 Backup and check script

本小节中,The backup and check script in /root 下,So the file is written to the command and other commands are need to sudo 权限执行.

首先,创建备份脚本 /root/snapraid_backup.sh：

#/bin/bash

DATE=$(date +'%Y%m%d')
LOG_DIR="/var/log/snapraid/backup"
LOG_PATH="$LOG_DIR/$DATE.log"

mkdir -p $LOG_DIR

# snapraid sync
snapraid sync >> $LOG_PATH 2>&1

Create check script /etc/snapraid_scrub.sh：

#/bin/bash

DATE=$(date +'%Y%m%d')
LOG_DIR="/var/log/snapraid/scrub"
LOG_PATH="$LOG_DIR/$DATE.log"

mkdir -p $LOG_DIR

# snapraid sync
snapraid scrub >> $LOG_PATH 2>&1

4.4.2 （可选）开启/关闭 SSD 缓存

If you have reference series on a period：家用 NAS 服务器（3）| SSD缓存加速机械硬盘,During the regular backup,最好关闭 SSD 缓存,以减少不必要的 SSD Consumption of life.没有设置 SSD Cache of friends you can ignore this chapter cough up！

首先创建 /root/cache_off.sh：

#/bin/bash

CACHE_LIST=(
    "bcache0"
)
CACHE_MODE="none"

for cache in ${CACHE_LIST[@]}; do
    echo $CACHE_MODE > /sys/block/$cache/bcache/cache_mode
done

以及 /root/cache_on.sh：

#/bin/bash

CACHE_LIST=(
    "bcache0"
)
CACHE_MODE="writearound"

for cache in ${CACHE_LIST[@]}; do
    echo $CACHE_MODE > /sys/block/$cache/bcache/cache_mode
done

上面的 CACHE_LIST 为需要开始/Stop buffer device name,For example, here is bcache0;CACHE_MODE 为缓存模式,建议设置为 writethrough 或者 writearound.

之后,修改 /root/snapraid_backup.sh 为：

#/bin/bash

DATE=$(date +'%Y%m%d')
LOG_DIR="/var/log/snapraid/backup"
LOG_PATH="$LOG_DIR/$DATE.log"

mkdir -p $LOG_DIR

# turn off cache
echo "Turn off cache..." > $LOG_PATH
echo "" >> $LOG_PATH
bash /root/cache_off.sh >> $LOG_PATH 2>&1
# snapraid sync
snapraid sync >> $LOG_PATH 2>&1
# turn on cache
echo "" >> $LOG_PATH
echo "Turn on cache..." >> $LOG_PATH
bash /root/cache_on.sh >> $LOG_PATH 2>&1

以及修改 /root/snapraid_scrub.sh 为：

#/bin/bash

DATE=$(date +'%Y%m%d')
LOG_DIR="/var/log/snapraid/scrub"
LOG_PATH="$LOG_DIR/$DATE.log"

mkdir -p $LOG_DIR

# turn off cache
echo "Turn off cache..." > $LOG_PATH
echo "" >> $LOG_PATH
bash /root/cache_off.sh >> $LOG_PATH 2>&1
# snapraid sync
snapraid scrub >> $LOG_PATH 2>&1
# turn on cache
echo "" >> $LOG_PATH
echo "Turn on cache..." >> $LOG_PATH
bash /root/cache_on.sh >> $LOG_PATH 2>&1

脚本运行后,会在 LOG_DIR 处（此处为 /var/log/snapraid）下生成日志.

4.4.3 cron 定时任务

进入 crontab 设置：

sudo crontab -e

Increase the following timing task：

0 5 * * * bash /root/snapraid_backup.sh
* * */10 * * bash /root/snapraid_scrub.sh

Here I set to：

• 每日 5 点备份数据,To avoid the common time
• 每 10 Day ten days before the check 8% 的校验数据

The above Settings can be according to your requirements to adjust.

---

5. 总结

经过上面的一通操作,Finally some of the most important disk backup set up,Don't have to worry about data loss in！( •̀ ω •́ )y

下一期开始,Will start software level set up.不过话说回来,It is imperative that the first winserver To activate the,然后把 winserver The remote desktop function of open,After can remotely make！

If you think of my content useful,还请给我留个赞和评论,最好点个关注,俺都会很开心的！

愿开源精神长存！