Two weeks ago I was helping a customer troubleshoot storage issues in their environment. Last week I was at Cisco Live and attended a session called Troubleshooting Storage Technologies with UCS in the Data Center, presented by Eric Austin, a UCS and Whiptail SME. Needless to say, Cisco UCS B-Series blade servers and storage have been on my mind a lot.
During the first part of the Cisco Live session, we covered three major block protocols: iSCSI, Fibre Channel, and Fibre Channel over Ethernet (FCoE). Then we went through common deployment issues for each of these protocols. Here are a few of the things we discussed in the session.
iSCSI common deployment issues and best practices
- The IQN/EUI is incorrect. This could mean using the wrong case or wrong date.
- Not having the correct VLAN configured throughout the entire stack can cause major issues. If the networking isn't set up properly, the UCS servers won't be able to communicate with the storage.
- The IP addresses for the initiators or targets are configured incorrectly.
- Firewalls are blocking connections.
- Insufficient bandwidth, particularly with 1GB deployments
- No Quality of Service
- MTU mismatches
- Multiple targets when installing Windows hosts due to multipathing issues within Windows
- Incorrect LUN IDs specified, especially when trying to use boot from SAN
- Use native VLAN for booting
Fibre Channel common deployment issues and best practices
- Improper zoning -- either not activated, misconfigured, or not configured on all the necessary storage switches
- Incorrect VSAN, UCS uplinks, or vHBAs
- Confusing the fabrics -- make sure fabric A and fabric B are kept separate
- Incorrect WWPN, WWNN, WWN, or LUN ID
- FC switch is using NPIV, and ports are type F.
- Avoid using soft zoning
FCoE common deployment issues
- MTU mismatches or Quality of Service mismatches
- Incorrect networking/VLAN settings
- Spanning tree not set up correctly (e.g., the port type edge trunk is not configured on the upstream Cisco switch to the UCS)
- Virtual Fibre Channel (VFC) interface configured incorrectly
- Disjointed layer 2 (recommended to only allow the VLANs you need over the links you're using for storage and UCS)
I've run into a lot of boot from SAN issues. Boot from SAN allows you to install an operating system, such as VMware ESXi, on a LUN on your storage array instead of on local storage. You may want to do this to use the concept of stateless computing. When using boot from SAN, you can have a server (physical host) crash and then replace it and have the new server boot the same operating system with minimal configuration.
The proper way to set this up (at least when using UCS servers and EMC arrays) is to create a storage group for each server. Each storage group will then contain a distinct boot LUN, usually specified as LUN 0, and then will contain the shared LUNs, which may map to datastores within a VMware environment. The shared LUNs in each storage group will be the same, but to reiterate, they will each have their own boot LUNs -- you cannot use one boot LUN for multiple servers. For more information on setting this up, read one of my previous articles.
Some tools we can use for troubleshooting discussed in the Cisco Live session include the command line and logs. On the fabric interconnect within the UCS, we can run the command:
Show logging logfile
This will show us a history of events that have happened recently. However, it's a circular file, so we can only see so many logs for a certain amount of time.
There are several CLI commands you can use at each part of the UCS as well that will give you some insight as to what might be configured incorrectly in your environment. You can download the documents from the session.
If you have tips, tricks, or questions about troubleshooting Cisco UCS with storage, please feel free to post them in the comments.
- Pro tip: Rack and stack Cisco UCS B-Series blade servers
- Demystify Cisco's intimidating UCS Manager with this basics rundown
- Pro tip: Configure ports for the Cisco UCS B-Series blades environment
- Create pools to maximize Cisco UCS blades' stateless computing features
- Create service profiles for Cisco UCS B-Series blade servers
Lauren Malhoit has been in the IT field for over 10 years and has acquired several data center certifications. She's currently a Technology Evangelist for Cisco focusing on ACI and Nexus 9000. She has been writing for a few years for TechRepublic, Tech Pro Research, and VirtualizationAdmin.com. As a Cisco Champion, EMC Elect, VMware vExpert, and PernixPro, Lauren stays involved in the IT community. Lauren has been a delegate for Tech Field Day and has also authored a book called VMware vCenter Operations Manager Essentials.