There is nothing more frustrating than building out a solution only to determine that while the costs for servers and storage have been accommodated for, the switching costs have not. I personally think we are in a transition time where fibre channel still makes sense in some situations where 10 Gigabit Ethernet (10 Gig-E) doesn’t quite make sense. For small installations, there is an opportunity to skip the fibre channel switch and connect the servers directly to the storage. This can even be possible in multiple node cluster configurations of two, three or more servers.In the course of virtualizing small workloads, administrators may be presented with an opportunity to consolidate a number of servers to a small VMware or Hyper-V cluster. The most critical design element of virtualized servers is the shared storage implementation; however, it may appear to be wasteful to invest in a fibre channel switching infrastructure for a small cluster. By a small cluster, I’m referring to a cluster of two or three VMware or Hyper-V hosts. Depending on any number of factors, that could be between 10 and 80 virtual machines on the small cluster by widely accepted virtualization design strategies. A typical configuration is shown below in Figure A below:
This is a relatively straightforward storage configuration, but in a small configuration the switching components may not be needed. For smaller virtualization implementations that have the only consumer of the storage being the Hyper-V or ESXi hosts, the switching components can be removed. The obvious benefit is reduced cost. Fibre channel switch pairs can cost USD $30,000 or more for a new purchase plus support agreements. This reduced footprint configuration would look like Figure B below:
Click image to enlarge.
Click image to enlarge.
There are a few considerations that need to go into a design such as this. Primarily, the amount of ports is limited and where they go are important. Most storage processors in the modular storage space allow four or more fibre channel ports for connectivity. Consider a dual controller system without a switch that has two fibre channel ports, two servers can connect directly to each controller. This will still provide dual-path connectivity to each controller from each host. Additionally, storage processors can have additional fibre channel ports added to the storage processor to allow three, four or more servers to connect directly in lieu of a switch.
The clear issue here is scalability. If there are additional consumers to the storage fabric, a switch may need to be added. This could include a tape drive, additional hosts or additional storage processors.
Have you ever designed around avoiding the switch in small environments? Share your comments below.
Rick Vanover is a software strategy specialist for Veeam Software, based in Columbus, Ohio. Rick has years of IT experience and focuses on virtualization, Windows-based server administration, and system hardware.