Follow Lauren Malhoit's step-by-step instructions on how to upgrade from vSphere 5.1 and View 5.1 and do a server refresh.
A couple of weeks ago I was helping a client move to new servers within their vSphere environment while also doing an upgrade of their VMware products. While this can be a really smooth transition, there are always things you need to consider if you don't want to shut down your VMs for a maintenance period.
Here's how I upgraded from vSphere 5.1 and View 5.1 while also doing a server refresh. The storage used was an EMC Clariion CX4 managed by Unisphere with fibre channel connections between the servers and the storage area network (SAN) and some EMC/Brocade fibre channel switches.
How to upgrade from vSphere 5.1 and View 5.1 and do a server refresh
The first thing you need to do is enable EVC on the current cluster. EVC allows you to put servers with dissimilar processors in the same cluster and still let you use vMotion and HA. Keep in mind that all of your servers (old and new) will have to be either Intel or AMD -- you can't mix and match. You also need to turn on EVC before you put a new server in the cluster; otherwise, you will be forced to restart your VMs in order to migrate them to one of the new servers.
You need to update your vCenter server to version 5.1 (you can find the documentation as a PDF on VMware's site). vCenter can manage ESXi servers on 5.0, but it will not work the other way around (i.e., vCenter 5.0 managing ESXi 5.1). To do the simple install option took me a little less than an hour by the time I was done configuring SSO with no real issues. The vCenter 5.1 install is a bit more involved than previous upgrades. I suggest familiarizing yourself with SSO and really going over the documentation before you start.
Now that vCenter is upgraded you can go through the process of adding a new host. You'll need to install ESXi 5.1 as you normally would (read this KB article about best practices), and then get it networked and connected to the storage. Keep in mind that in most configurations using fibre channel you will need to zone the switch properly so your ESXi server can see the datastores. You'll need to create aliases, map the WWNs and storage processors, and add these to the current zonesets (check your SAN's documentation for more information). As far as the virtual networking goes, you just need to match the networking of your current hosts. Follow the detailed steps below for this specific configuration.
1. Sign in to the fibre channel switch UI via your browser.
2. Click the Zone Admin link in the top left.
3. Click the Aliases tab.
4. Open the vSphere client.
5. Highlight your new host, go to the Configuration tab | Storage Adapters and click the vmHBAs to get the WWNs for those adapters.
6. Back in the switch, click New Alias.
7. Give it a descriptive name (e.g., ESX_HOST_1_HBA1).
8. Select the WWN that you found earlier and move it to the members column.
9. Click the Zone tab.
10. Click the New Zone button.
11. Give it a descriptive name according to which storage processor you're connecting it to (e.g., ESX_HOST_1_HBA1_SPA0).
12. Select the storage processor you're working with and the Alias you just created and move those into the members column.
13. Repeat steps 11 and 12 for the second storage processor.
14. Repeat steps 1-13 for the second fibre channel switch.
15. Log in to the Unisphere UI using your browser.
16. Add the server to the storage group set up for your ESXi hosts and make sure all four connections are listed for it.
Your new host should be part of your cluster, and you should be able to vMotion all your VMs off one of your old hosts to this new one. Once there are no VMs left on your old host, put it in maintenance mode and then shut it down. Once it's shut down, take it out of your storage group on your SAN, and take it out of the zoneset on your fibre channel switches. You may now disconnect it physically from your storage and networking. You've also freed up some fibre channel ports and Ethernet ports by doing this. If you use the same fibre channel ports on your next new host, be sure that you re-zone them to match the WWNs on your new host. Keep swapping out the physical hosts like this until you have all of the old hosts out of the cluster.
Be sure to follow best practices when installing and configuring vCenter and ESXi, and don't forget to configure NTP if you use it.