I’ve spent 17 years in the IT field, and I’ve just barely scratched the surface of what’s out there; however, one constant that has stayed with me from the very beginning is Ethernet. In 1994, I was running thin coaxial (10Base2) and category 5 (10BaseT) cabling through K-12 schools in upstate New York. At the time, 10BaseT had come on the scene as “the next big thing” in networking with incredible speeds of up to 10 Mbps. At the same time, 10BaseT rid Ethernet of several of the serious limitations inherent in coaxial-based Ethernet networks:
- There was no more need to replace lost or damaged terminators.
- There was no more need for vampire taps.
- 10BaseT brought to Ethernet a “home run” simplicity that moved Ethernet from a continuous line of computers to a more reliable hub and spoke architecture.
At the time, the transformation was big, but it only portended what was to come. Over the past 17 years, Ethernet has come a long, long way from thick coax with DSL-like speed limits to fiber-based Ethernet and speeds of 10 Gbps common in the data center.
Enhanced Ethernet overview
In my Fibre Channel over Ethernet (FCoE) primer, I provided a 30,000-foot view of FCoE – I described why it’s compelling, its associated challenges, and the changes it brings to the operating environment. One of these changes — Enhanced Ethernet — brings with it major improvements to this sturdy and venerable technology.
- Data Center Ethernet (DCE). This is Cisco’s version of Enhanced Ethernet and includes enhancements to data center bridging standards.
- Convergence Enhanced Ethernet or Converged Enhanced Ethernet (CEE). The standards defined in CEE were developed by a representative group of networking and storage companies with an eye toward I/O convergence based on Ethernet. In particular, the group focused their efforts on ways to enhance Ethernet in a way that would make it a suitable transport mechanism for sensitive Fibre Channel-based traffic.
This article is an overview of what’s necessary for Enhanced Ethernet to operate.
Notes: For the purposes of this article, I’ll group all of this under the umbrella term Enhanced Ethernet. Although the terms aren’t always technically interchangeable, it’s close enough. This is an overview and not a technical deep dive.
Enhanced Ethernet features
Here are some of the features that comprise Enhanced Ethernet:
- Lossless. Through the use of traditional Ethernet technologies as well as the new ones described below, Enhanced Ethernet attains lossless characteristics that are critically important in environments that absolutely demand guaranteed packet delivery. A Fibre Channel environment is one such environment.
- Priority Flow Control. Ethernet already has some pausing capability, which helps to avoid congestion by temporarily halting transmission when a switch port buffer is full; however, it’s limited and takes an all or nothing approach — either all traffic is paused or no traffic is paused. Enhanced Ethernet takes it a step further through what’s called Priority Flow Control (PFC). PFC splits Ethernet communication into eight channels, each of which can be paused individually. This allows a switch that supports Enhanced Ethernet much more transmission flexibility, particularly when combined with Enhanced Transmission Selection.
- Enhanced Transmission Selection. Enhanced Transmission Selection (ETS aka priority grouping) enables further traffic prioritization and differentiation through the use of bandwidth and latency control mechanisms on a per-channel basis. This feature helps to ensure that traffic containing sensitive payloads is prioritized appropriately. Think of this as bandwidth shaping and management for each of PFC’s eight channels.
- Congestion Notification. Congestion Notification helps to smooth out traffic spikes by distributing congestion between core and edge services. In other words, as much as possible, Congestion Notification helps shape network traffic in an attempt to keep congestion closer to the edge, but is an end-to-end management tool that operates across the entire network spectrum.
- Data Center Bridging Exchange (DCBX). This protocol allows peers to exchange configuration and capability information. DCBX helps devices understand what is and what is not supported by other devices on the network. For example, if a device doesn’t support PFC, it won’t be used.
What this means to you
If you’re a server administrator or the person who administers the storage network, this information means that you need to enhance your Ethernet knowledge (get it? enhance?). Ethernet as a transport mechanism is here to stay and is being improved all the time in order to make it an even more ubiquitous technology.
It also means that you may need to start wrapping your head around converged network adapters (CNAs) and what they mean for your server systems. In FCoE-land, a CNA is a single network adapter that has both a Fibre Channel Host Bus Adapter (HBA) and a traditional network adapter built into one device with one connector.
Enhanced Ethernet brings with it very good things for Ethernet and for converged I/O in general. Introducing Enhanced Ethernet in your environment will require some new thinking and skills, but it will also result in lower costs and a simpler environment since there will be fewer pieces of equipment and fewer cables to deal with in the end.