Cisco UCS B-Series blade servers are increasing in popularity for various reasons. Some people like Cisco’s cabling innovations, reducing the ever-increasing rat’s nest of cables necessary within a data center. Others appreciate the amount of computing power you can fit into 6U (6 rack units), which is the size of one UCS chassis. There’s also the stateless computing to consider, which allows you to swap out blade servers if necessary, and we can add chassis without downtime.
Whatever your reason for choosing to go with blade servers, specifically Cisco UCS, there’s a lot to it if you’re just getting started. In this article I’ll concentrate on the racking of one chassis with the necessary components. (Note: This article is only meant to supplement the many books and Cisco documentation available about this topic.)
Let’s start with a brief overview of the components involved.
Chassis: This is a 6U chassis that can hold eight half-width blade servers or four full-width blades. It also contains four power supplies and the fabric extenders (FEX) (Figure A).
FEX (older terminology is IOM Module): There are generally two FEX in the back of the chassis for redundancy. This connects the chassis to the fabric interconnects (FIs) usually via 10 Gig Fibre Channel over Ethernet (FCoE) (Figure B).
FIs: There are generally two FIs that connect to the FEXs as well as to the existing IP and storage network (Figure C). FIs have unified ports, meaning we can specify within the GUI or CLI if we want them to connect to IP switches or storage switches.
UCS Manager (UCSM): This is the GUI interface where you administer all of the components connected to the FIs (Figure D). The UCSM is contained in the FIs, which is important to remember.
Racking the servers
The configuration within your rack is up to you. I commonly see the first chassis mounted at the bottom of the rack because it’s very heavy. Then the FIs are mounted at the top to leave room for more chassis within the rack for future growth. We’ll leave the servers out for now. Once the chassis and FIs are secured, we can start cabling. The first thing we’ll cable is the L0 and L1 ports together in each FI (L0 to L0 and L1 to L1)–this will create the cluster connectivity.
Though you can use normal fibre or copper cables with separate small form-factor pluggables (SFPs), it’s really easy to use what are called Twinax cables to connect the FEXs to the FIs. Twinax cables are generally cheaper and come with SFPs attached to either end; this will provide the 10 gig connectivity between the chassis and the FIs. Each FEX has up to eight ports, but you can use as little as two if you don’t need all of that bandwidth. Plug at least two cables into FEX A and connect those to the same FI–meaning, you want all the cables in FEX A to go to FI A, thereby creating one fabric–and then do the same with FEX B and FI B.
Once that is connected, you can connect the first few ports on your FI to your IP network. You can configure LACP on your networking switches to aggregate the ports and allow for more bandwidth and load balancing. You’ll also connect the FI to your storage switches–this can be iSCSI or Fibre Channel. For this example, I’m going to stick with Fibre Channel. If you’re connecting to Cisco storage switches, you can create a virtual port channel (VPC). If you’re using Brocade switches, we can’t exactly do that; however, it will use a sort of round robin load balancing algorithm between all the connections you create. For more information, see the Cisco UCS to Brocade Connectivity Guide white paper (PDF).
Now you’ll power the whole thing on and connect a console cable to your laptop and the first FI. Open a hyperterminal to the FI to see the initial configuration wizard; this wizard will basically take you through the network settings, such as IP address, subnet mask, default gateway, and DNS settings. The wizard will also ask if this FI will be part of a cluster; since we have two, answer Yes. We’ll supply an IP for the cluster as well here. When that configuration is done, move the console cable to the second FI. This FI will detect that it’s part of a cluster, so the initial configuration wizard will simply ask if you want to join that cluster and also ask for its own IP. When that’s complete, make sure you restarted both FIs.
When the FIs are completely up again, open a web browser and browse to the IP address you set for the cluster–this will connect you to the UCSM where we can continue the configuration. At this point, we can also insert our servers into the chassis.
More to come
In future posts I’ll walk you through more UCS blade configuration steps. If you have comments or helpful hints about this process, please feel free to share them in the discussion.