Ethernet speeds might not be doubling every two years like what Gordon Moore predicted for processor transistors, but they have been skyrocketing. Case in point: At the Hadoop user meeting in March 2015 in the Twin Cities, two individuals who work at the University of Minnesota's Supercomputing Institute were talking about the new insanely-fast network at the Institute — 100 gigabit per second full-wire speed — that saves hours of compute time on the school's EDR InfiniBand large-scale cluster. Someone asked who made the equipment, and they said Mellanox Technologies. Until then, I had not heard of Mellanox.
Three months later, Mellanox, founded in 1999 by Eyal Waldman (now CEO) and a team of engineers in Yokneam, Israel, is once again making news. The maker of InfiniBand and Ethernet interconnection equipment — including adapters, switches, silicon, and software — introduced Spectrum, the industry's first 100 Gigabit Ethernet, Open Ethernet, non-blocking switch. The company also introduced ConnectX-4 Lx, a series of network adapters. Together, the equipment enables end-to-end 10, 25, 40, 50, or 100 Gigabit Ethernet connectivity.
The Spectrum line of switches offers the following features.
- Wire-speed switching: 6.4 Tb/s and 9.52 B packets-per-second
- High density: 32 100 GbE ports, 32 40/56 GbE ports, 64 10 GbE ports, 64 25 GbE ports, and 64 50 GbE ports
- Low latency: 300 nsec for 100 GbE port-to-port, and flat latency across L2 and L3 forwarding
- Low power: 7.5 watts per port
- Overlay gateway and tunneling support, including VXLAN, NVGRE, Geneve, MPLS, and IPinIP
- Embedded low-latency RoCEv2 support for storage and compute fabrics
I was curious about the industry firsts mentioned in the Mellanox Technologies press release — in particular non-blocking. According to the Computer Desktop Encyclopedia, non-blocking is the ability of a signal to reach its destination without interference or delay. In a non-blocking switch, all ports can run at full wire speed with no loss of packets or cells. I asked a few network engineers I know why non-blocking is important.
They all agreed non-blocking is essential if Spectrum's target market is data-center operators. As to why, one veteran engineer explained that data traffic used to be bursty, and a blocking switch would not cause enough delay to be noticeable. However, with today's more latency-prone traffic, delays or "blocking" are unacceptable.
Open Ethernet was also mentioned as a first. According to Mellanox, Open Ethernet is:
"An initiative based on a complete separation between the switch hardware and the switch software. In simple terms, Open Ethernet allows any software to be run on any hardware and allows either the hardware or the software to be changed without changing the other components."
The engineers are positive about Open Ethernet, reminding me open-source initiatives similar to the Open Compute Project are good for the industry. Open Ethernet networking equipment will let data-center operators select Application Programming Interfaces (API) that match their requirements.
"The need for increased speed and ability to customize data-center networks to meet expanding business needs is critical for the modern application-dependent enterprise," said Cliff Grossner, research director at IHS, in the Mellanox press release. "Open Ethernet delivering increased network programmability, choice of switch-hardware components and switch operating system will play an important role within the data center."
Bloomberg Business interviewed Waldman soon after he announced the Spectrum line of interconnection equipment. One of the first questions asked of Waldman was, "Who are your competitors?" Waldman responded, "I believe we are a generation ahead of Intel, Broadcom, and other competitors. And, I believe we can maintain the gap going forward."
A few days before the Spectrum announcement, Cisco introduced its new Nexus 3000-series switches, which also have 10, 25, 40, 50, and 100 GigE connectivity.
It will be interesting to see how a relatively new and smaller company fares against the industry giants. The Bloomberg Business interviewers seemed convinced that Mellanox is ripe for a takeover, which CEO Waldman adroitly denied.
Information is my field...Writing is my passion...Coupling the two is my mission.