Working principle of fastdfs (technical principle)

Hello everyone , I meet you again , I'm your friend, Quan Jun .

FastDFS framework ：

FastDFS The server has three roles ： Tracking server （tracker server）、 Storage server （storage server） And the client （client）.

**tracker server：** Tracking server , Mainly do scheduling work , Play the role of load balancing . Record the status information of all storage groups and storage servers in the cluster in memory , It is the hub of interaction between client and data server . comparison GFS Medium master More streamlined , Do not record file index information , The amount of memory used is very small .

**storage server：** Storage server （ also called ： Storage nodes or data servers ）, File and file properties （meta data） All saved to the storage server .Storage server Direct use of OS File system call management file .

**client：** client , As the originator of the business request , Through proprietary interfaces , Use TCP/IP The protocol interacts with tracker server or storage node .

FastDFS agreement ：

FastDFS Roles are based on TCP/IP Protocol to communicate , The format of the agreement package is ：header + body. The specific structure is shown in the figure ：

FastDFS All nodes are connected by tcp/ip The way to communicate . The package consists of two parts ：header and body

Upload mechanism ：

Synchronous time management ：

When a file is uploaded successfully , The client immediately initiates a request to download the file （ Or delete request ） when ,tracker How to choose a suitable storage server ？

In fact, each storage server needs to regularly report its own information to tracker, This information includes the local synchronization time （ namely , Timestamp of the latest file synchronized to ）. and tracker According to the reporting situation of each storage server , You can know the file just uploaded , Whether the synchronization has been completed in this storage group . The synchronization information reporting is shown in the figure below ：

Download mechanism ：

Delicate FID： When it comes to downloading, you have to mention the file index （ also called ：FID） The exquisite design of . The file index structure is shown in the figure below , After the client uploads the file, the storage server returns it to the client , Index information used to access the file later . File index information includes ： Group name , Virtual disk path , Two levels of data , file name .

ps：

Group name ： The name of the storage group where the file is uploaded , After the file is uploaded successfully, the storage server returns , It needs to be saved by the client . There can be multiple under a group storage, I feel that the group is for management storage Of

Virtual disk path ： Virtual path to store server configuration , With disk options store_path* Corresponding .

Two levels of data ： Two level directories created by the storage server under each virtual disk path , For storing data files .

file name ： It's different from when the file is uploaded . Is generated by the storage server based on specific information , The filename contains ： Source storage server IP Address 、 File creation timestamp 、 file size 、 Random numbers and file extensions .

Quick location of files ：

know FastDFS FID After the composition of , Let's see FastDFS How to get through this ingenious FID Navigate to the file you want to access .

By group name tracker It can quickly locate the storage server group that the client needs to access , And will choose the appropriate storage server to provide client access ;

Storage server according to “ File storage virtual disk path ” and “ Two level directory of data files ” You can quickly navigate to the directory where the files are located , And according to the file name to find the client needs to access the file .

The main content of this sharing includes ：FastDFS The task division of each role / Collaboration , The principle design of document index and document uploading / Download process . Through this study, we are right FastDFS Have a preliminary understanding of , Such as ：

FastDFS There are only three characters ; And neither the tracking server nor the storage server has a single point .

The tracking server passively receives reports from the storage server , Group management of storage servers ; And select the applicable storage server for the client . The same storage server can report status information to multiple tracking servers at the same time .

All storage servers in the storage server group are peer-to-peer , The stored data is one-to-one and the same ; All storage servers are online at the same time , Greatly improve the utilization rate of the server , Share the pressure of data access .

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/129014.html Link to the original text ：https://javaforall.cn