LAN SDN hard core technology insider 23 looking forward to the future - RDMA (Part 1)

Let's summarize the early harvest ：

Moore's law is limited by the physical limits of silicon chips -> Multi core technology promotes the large-scale application of virtualization -> Data center network large-scale virtual machine network ;

Wi-Fi Technology driven mobile office popularization -> Large scale mobile roaming of users in the park network ;

Virtual machines and campus network roaming terminals are popular -> The network needs to be divided into flexible slices , Virtual machines or park users are automatically divided into their own slices when accessing the network ->VXLAN technology

VXLAN There is no control plane ->EVPN+SDN As the control plane ;

Reasonably design the network , avoid VXLAN Hardware specification limitations ;

so ,SDN The emergence of network , Evolution and maturity , Is a typical IaaS The network revolution triggered by the level demand .

that , Let's look up , From the data center IaaS Move layer to PaaS layer ——

The essence of data center , It is a place to run large-scale distributed computing or storage .

Think about it , Our common computing and storage PaaS What are the frameworks ？

Hadoop, Tensorflow, Ceph, Gluster……

These distributed computing and storage frameworks , The similarity is , Need frequent block access to the memory of the remote computer .

Traditional remote access , Need to pass through Linux or Windows Of Socket Mechanism . This mechanism sends every packet , Both need to transfer data packets from user mode memory space , Copy to the kernel memory space where the network card driver is located , Send again . And accept packets , You need to copy the data package from the memory space where the network card driver is located to the memory space of the user mode application .

obviously , In large-scale distributed computing and storage , It's hard to accept .

therefore , Engineers introduced RDMA(Remote Direct Memory Access) Mechanism , So that the application can directly pass RDMA Command word , Let the remote server network card directly from RAM Get the data of the specified address ;

RDMA Of API, With the traditional Linux/Windows Socket Of API Somewhat different , It's called RDMA Verb. To send packets RDMA Verb For example , Its function prototype is as follows ：

int ibv_post_send(struct ibv_qp *qp, struct ibv_send_wr *wr, struct ibv_send_wr **bad_wr)

Let's look at the structure ibv_send_wr The definition of ：

We see , In the red box is the remote address . in other words ,RDMA In the code API Level , Provides a mechanism for direct access to remote addresses .

RDMA With the traditional Socket Mechanism , The comparison of data flow is shown in the figure below ：

so ,RDMA Bypassing the kernel interrupt mechanism , The delay of remote access is greatly reduced , To reduce the CPU Occupy .

Notice that in the figure RDMA The passage of , It is two different channels from the network . What does that mean ？

Tomorrow, , We explain this detail in detail for you .