Network equipment hard core technology insider router Chapter 16 dpdk and its prequel (I)

In the last topic , We mentioned that ,NFV As SDN One of the genres of , With vBRAS And other forms are promoting the reconstruction of operators' Networks . in fact , Not only in the field of operators , meanwhile , stay IDC in , Based on network element virtualization vSwitch as well as LB/ Firewall and other value-added services , It's also popular . these NFV The platform on which the software runs , Is based on x86 Multi core processor platform .

because x86 The tradition of running on Linux The architecture limitations of ,x86 Multi core has been unable to achieve high-performance data forwarding , therefore , Led by Intel DPDK(Data Plane Development Kit) Has become NFV Standard configuration of , Major Internet enterprises recruit network business development 、 Architecture Engineer , Will also be familiar with DPDK Written into the recruitment requirements .DPDK With the container 、SD-WAN、 Concepts such as edge computing have become the hottest ICT technology .

but , What you don't know is , As early as 2006 year , Similar projects have been carried out on multi-core processors in China DPDK Research and development .

2006 year , Multi core processors are the hottest in the Jianghu Spicy chicken Fried chicken Yes MIPS Multicore processor .RMI( Has been Broadcom Merger and acquisition (m&a) ) and Cavium( Has been Marvell Merger and acquisition (m&a) ) A fierce multi-core processor arms race broke out . And China is at the forefront of science and Technology H3C The company has a keen insight into the trend that multi-core general-purpose and semi general-purpose processors will become the mainstream processors of network devices in the future , Using multi-core processors, we have developed a series of medium and high-end routers and security products .

SR6602 Namely H3C The representative work of router based on multi-core technology . It is based on the most advanced 8 nucleus 32 Hardware thread processor , The internal block diagram of the processor is shown in the figure below ：

It can be seen that , The processor has the highest 24Gbps Network of IO processing capacity , It can be expanded TCAM Interface , There is a fast message ring network inside (Fast Messaging Network) From the network interface 、 The data packet of the security encryption algorithm engine quickly notifies the corresponding CPU/ Threads , And each CPU/ Threads can also pass FMN Communicate with each other .

H3C During the development process, the engineers found , Simply put the soft forwarding module in the network operating system in multiple CPU Up operation , There is no linear improvement in performance . As before 1 Get on threads 200kbps Forwarding capability of ,28 The forwarding performance of threads is only 1500kbps, It is much lower than the expectation of product planning . Why is that ？

original , For Parallel Computing , There is an important law ——Amdahl Law of acceleration ratio .

Amdahl The content of the law of acceleration ratio is , The speedup ratio available to complete complex work is limited , Limited by the part of this work that must be executed serially . in other words , Even if there is a small amount of computing work that requires multiple processors to execute in series , The overall performance will also be greatly reduced .

actually , When multi-core processors process packets in parallel , In most cases , Each packet can be based on hash Algorithm , The load is shared to different processors for processing .

however , Notice the part framed by the red line ：

All processor cores share L2 cache , such , Once two or more processor cores share a cache area , This will cause multiple cores to queue up to access this cache area .

The default cache organization of the processor , So cache lines (cache line) Organized in a way . Such as MIPS The caching behavior of 32 byte , And packet caching (MBuf) The size is 4KB, yes 32 Bytes of 128 times . such , It is very easy to cause continuous MBuf Use conflict , When accepting the next packet , Just kick out the previous packet cacheline.

The solution is simple . Distribute MBuf when , With 4KB+32 Bytes are units . such , Can guarantee every two MBuf Between , The header will not map to the same one cacheline.

in addition , When implementing a router with multiple cores , There is another place that is prone to cache conflicts , Do you know where it is ？