As a general purpose OS, Linux is limited in its effective network throughput, but Intel's latest open source project could help better enable impressive network performance from it.
Linux is a general purpose operating system. This comment may sound like an obvious statement, but it's sometimes easy to forget. Because it's a general operating system, it is used across a variety of use cases.
The OS is used in Internet of Things (IoT) devices, smartphones, tablets, servers, and data center appliances. However, it sometimes takes a reminder that using Linux for specialized use cases, such as a network or even network function devices, takes some customization of the kernel or the acceptance that performance may be uneven or limited. The Intel-sponsored open source Data Plane Development Kit (DPDK) project hopes to extend the usefulness of Linux to include high-performance networking devices.
Linux high-performance networking
It's a challenge to achieve high-performance networking inside of the Linux kernel. A dangerous oversimplification would be to say that Linux networking is optimized for general purpose computing. According to this deep dive from CloudFlare, a common conception is that Linux maxes out at 50K packets per second (pps). Pps calculation is a bit different than bits per second (bps). While an interface on a server may have a speed of 10Gbps, it doesn't necessarily correlate to how much data gets processed.
See: Intel launches silicon photonics (ZDNet)
In the CloudFlare write-up, the experiment used the minimum Ethernet Frame size of 72 bytes or 576 bits. The author was able to achieve a peak throughput of 1.4Mpps, or about 8Gbps on a 10Gbps link. The challenge is primarily that of processor efficiency. The test worked hard to pin threads to specific processor cores to eliminate wasted cycles. According to the article, a realistic goal using the described process is achieving throughput of 1Mpps sustained. 1Mpps doesn't come close to consuming a 10Gbps link. So, why do we see multiple 10Gbps links, or even 40Gbps links in x86 servers?
Bypassing the kernel
To utilize processors more efficiently in Linux, developers bypass the Linux kernel and dedicate CPU cores to network processing. The CloudFlare example used Linux kernel tools to pin processes to a set of cores and actively managed memory queues. It took a bit of programming discipline to send unprocessed packets between two systems. Add any packet processing, such as IPS, and the performance should drop significantly.
DPDK bypasses the Linux kernel and manages the NIC and CPU assignment directly. On a single Intel processor, it's possible to achieve throughput of up to 80Mpps or roughly 46Gbps using the smallest Ethernet Frames. I want to make sure I caveat the above comment and note that it was an Intel processor. DPDK is an open source project and currently has both Intel and ARM distributions.
Enterprise use cases
The DPDK is a development kit. The audience is creators of network software and devices. However, these devices make their way to the enterprise. An intriguing use case is hypervisors and network functions virtualization (NFV). It's easy to envision an NFV-based firewall or IPS that leverages DPDK to achieve single instance processing of up to 10Mpps without using specialized hardware.
DPDK is yet another enabler for software driving the replacement of dedicated hardware in the data center. X86 and ARM servers continue to supplant dedicated hardware for both traditional storage and network use cases. For a deeper dive on DPDK, I suggest this presentation from a Tech Field Day event.
- Why Dell is disaggregating with OS 10 (TechRepublic)
- Intel to manufacture ARM processors in a bet on IoT and premium smartphones (TechRepublic)
- Intel launches new products to prepare the network for the future of cloud (TechRepublic)
- Intel previews its next processor geared for machine learning (ZDnet)
- Why Cisco's data center strategy depends on simplifying storage (TechRepublic)