Network equipment hard core technology insider router 20 dpdk (V)

Except as mentioned above , For data structures that may be accessed concurrently by multiple cores such as counters , Assign one to each core , New mechanisms need to be introduced —— Large page mechanism .

Speaking of the large page mechanism , From the original 8086 The times say .

1979 year , It was a spring . A young man pressed a switch in a laboratory in California ……

Intel 8086 The age of processors , A program written by a programmer , They all directly access the physical address . What is a physical address ？ Is in the CPU Add a logic analyzer to the front-end address bus , Read out the address . Such as memory 0xDEADBEEF, Corresponding bus A31 To A0 Namely ：

11011110 10101101 10111110 11101111.

however , stay 80386 Time , The protection mode appears .

Protection mode maps physical addresses to logical addresses , If the program accesses an address without permission , The system will fall into “ abnormal ”, The exception handler decides whether to report an error or other processing methods .

The mapping table consists of CPU Technical secondary school uses hardware to realize , This kind of hardware is called TLB.CPU By looking for TLB You can immediately find the physical address corresponding to the virtual address . If in TLB We didn't find , It triggers TLB Miss, About pay 30 individual CPU The cost of instruction cycles .

default TLB Page size is 4KB. such , If the application needs 4MB Memory , Will trigger 1024 Time TLB miss, To load all the required memory mapping relationships into TLB in . therefore ,DPDK Take advantage of Linux Special mechanisms provided by the kernel , bypass x86 The maximum size of each page of the architecture 4MB The limitation of , Provides the largest 1GB Of TLB Table item . such , Just a few times TLB Miss You can achieve all the required memory mapping .

This special mechanism is called hugetlbfs, In essence, it is to exist in the big page Linux Next , Mount as part of the file system .

In a general way , Server , You can do this ：

mkdir /mnt/huge

stay /etc/fstab Inside plus

nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0

such , for DPDK Reserved 1GB Big page .DPDK Large page memory will be loaded automatically when starting .

however , In the actual server , One more thing to consider ——NUMA framework .

NUMA The architecture is “Non Uniform Memory Access Architecture” Abbreviation , It's a classic SMP The streamlining of the architecture .

stay SMP Architecture , Any processor can directly access any memory unit , but , As the name suggests ,NUMA Architecture is not the case .

In two CPU Of NUMA Architecture server ,CPU0 visit CPU1 It's on the bottom RAM, Need to go through CPU Interconnection bus between . Predictably , This will have a great impact on the efficiency of reading and writing .

therefore , stay DPDK in , Allocate memory 、 Assign a send receive queue to the network card , All need to use the thread socket_id, Ensure the optimal access path .

And RAM Similarly , The network card is also affected NUMA The limitation of . stay DPDK In the implementation of , Also try to ensure that the data packets of each network card are sent by the corresponding CPU To deal with .

DPDK Some network optimization functions of the network card are also used to improve performance , The most typical is multi queue .

The picture below is intel Of 82575 Logical block diagram of network card . This network card comes with a simple hardware acceleration unit , It can extract five tuples of messages (SIP,DIP,Proto,SPort,DPort), Calculation hash value , And according to hash Value sent to 4 One of the queues . then , various vCPU（ Hardware threads ） From the queue assigned to yourself , Obtain data packets by polling and conduct business processing .

that , What should we do if the number of hardware threads is much more than the number of network card queues ？

We can deal with it in a flexible way , Let several hardware threads perform flow classification services , Distribute the packets obtained from the queue to other threads . such , Even hundreds of hardware threads 4 The road even 8 Road server , You can also make full use of performance .

Of course ,DPDK In addition to using the multi queue function of the network card , You can also use other hardware acceleration functions of the network card , Such as TCP uninstall ,VXLAN Tunnel unloading, etc , Further accelerate the processing of network packets .

With DPDK technology , You can use it x86 Powerful multi-core concurrent processing capability for high-speed forwarding of routing packets .

because x86 Platform compatibility , You can also use the resources of the open source community , Integrate deep packet detection and other functions into the router . You can see , some time , Metro edge and enterprise export router will have x86 A place for .