Zero copy technology

Traditional file transfer process ：

• The first step is to copy ： Copy the data on the disk to the kernel buffer of the operating system , adopt DMA Realization
• Step 2 copy ： Copy the data from the kernel buffer to the user's buffer , At this point, our application can use this data , This copy process is performed by CPU complete
• The third copy ： Copy the data just copied to the user buffer to the kernel socket In the buffer , This process is still controlled by CPU complete
• The fourth copy ： Put the kernel of socket The data in the buffer is copied to the buffer of the network card , The process consists of DMA Realization

Zero copy technology

When a computer performs an operation ,CPU There is no need to copy data from one memory to another , This technology is usually used to save time when transferring files over the network CPU Cycle and memory bandwidth . Popular speaking , Zero copy is a way to avoid CPU The technology of copying data from one piece of storage to another , Zero copy technology can reduce the number of data copies and shared bus operations , Eliminate unnecessary intermediate copies of transferred data between memories , So as to improve the data transmission efficiency

• DMA : Put the disk data into the kernel buffer
• read() : During the system call, the data in the kernel buffer will be transferred to the user's buffer
• write() : The operating system directly transfers the kernel buffer data to socket Buffer zone
• CPU Copy ： Transfer the data in the kernel buffer to socket Buffer procedure
  
  It can be seen that in the process of a file transfer 4 Second copy , And during this period, a total of 4 Time User mode and kernel mode Context switch , Because it happened twice system call （ When an application accesses the functions of the operating system kernel, the process of calling the system kernel functions is called system call ）, Once it was read, Once it was write, Each system call must first switch from user mode to kernel mode , After the kernel completes its task , Then switch from kernel mode to user mode , Context switching costs are not small , A switch takes tens of nanoseconds to a few microseconds , In a high concurrency scenario , This time will be magnified , Affect system performance .

problem

Let's look at the file transfer process , We just moved a copy of the data , But it was carried four times , This traditional file transfer method has redundant context switching and data copying , In a high concurrency scenario , Serious impact on system performance

therefore , We need to reduce the number of context switches and data copies between user mode and kernel mode

Solution ：

MMAP+wirte

mmap() The system call function will directly put the data in the kernel buffer 「 mapping 」 To user space , such , The operating system kernel and user space do not need any more data operations .

 buf = mmap(diskfd, len); 
 write(sockfd, buf, len);

Application calls mmap(), The data on the disk will pass through DMA The copied kernel buffer , Then the operating system will share the kernel buffer with the application , In this way, you don't need to copy the contents of the kernel buffer to user space . The application calls write(), The operating system directly copies the contents of the kernel buffer to socket Buffer zone , All this happens in kernel state , Last ,socket The buffer sends the data to the network card . This process reduces one copy

problem

Use mmap replace read There's a significant reduction in one copy , When there is a large amount of copied data , No doubt it improves efficiency . But use mmap There is a price . When you use mmap when , You may encounter some hidden traps . for example , When your program map A document , But when this file is truncated by another process (truncate) when , write The system call will be SIGBUS Signal termination .SIGBUS The signal defaults to killing your process and creating a coredump, If your server is suspended like this , That would be a loss .

Solution

1. by SIGBUS Signal set up signal processing program
When you meet SIGBUS Signal time , The signal processor simply returns ,write The system call returns the number of bytes written before it is interrupted , also errno It's going to be set to success, But it's a bad way to deal with it , Because you don't solve the core of the problem .

2. Use file rental lock
Usually we use this method , Use lease locks on file descriptors , We request a lease lock from the kernel for the file , When other processes want to truncate this file , The kernel will send us a real-time RT_SIGNAL_LEASE The signal , Tell us that the kernel is breaking the read and write locks you've imposed on files . In this way, the program accesses illegal memory and is SIGBUS Before killing , Yours write The system call will be interrupted .write Returns the number of bytes that have been written , And put errno by success.
We should mmap Lock the file before it , And unlock the file after the operation ：

sendfile

from Linux kernel 2.4 The version starts , For support network card support SG-DMA In the case of Technology ,
First step , adopt DMA Copy the data on the disk to the kernel buffer ;
The second step , The buffer copies the file descriptor and data length to socket buffer ,
This network card of SG-DMA The controller can directly transfer the data in the kernel cache to the buffer of the network card ,
This process does not require data to be flushed from the operating system kernel buffer to socket Buffer zone , In this way, the data is reduced once

Compared with the traditional file transfer mode, the file transfer mode of zero Castanopsis shell technology , Less 2 Times of context switching and data retrieval , It only needs 2 Times of context switching and data retrieval , You can complete the file transfer , and 2 Secondary data processing , You don't have to go through CPU,2 It's all by DMA To carry . therefore , overall , Zero shell technology can improve the performance of file transfer by at least twice

Benefits of zero copy

• Reduce or even completely avoid unnecessary CPU Copy , So that CPU Free yourself to perform other tasks
• Reduce the memory bandwidth
• In general, zero copy technology can also reduce the context switching between user space and operating system kernel space