SolutionBase: Building a cluster with VMware GSX Server

How to build a test cluster on one machine using VMWare GSX Server

IT budgets rarely allow you to have a complete testing lab at your disposal to test various server configurations. This can be especially frustrating if you want to investigate complex network administrative issues such as clustering. Rather than making a big investment in various pieces of hardware, you can use VMware Workstation's big brother, GSX Server, to set up a test cluster in a matter of a few hours and without any investment in hardware.

Installing GSX Server
With GSX Server at just over $3,000 for a two-CPU license, you probably won't rush out and buy a copy just because you want to experiment with clustering or server virtualization. If your company already has GSX Server, you're all set to start building your first cluster. If not, you can download a 15-day evaluation copy from VMware's GSX Server Web site.

When you have that copy on hand, actual installation of GSX Server is a breeze. You only need to run the downloaded executable on a supported host system and step through the installation wizard. I chose Windows Server 2003 Enterprise Edition as the host operating system simply because I had it on hand. Among server operating systems, GSX Server supports the following:
  • Windows Server 2003 Enterprise
  • Windows Server 2003 Standard
  • Windows Server 2003 Web
  • Windows 2000 Server SP3 and greater
  • Windows 2000 Advanced Server Service Pack 3 and greater
  • Red Hat Linux
  • SuSE Linux
  • Mandrake Linux
  • Turbolinux

If you decide to use Windows Server 2003 as the host platform and want to use the VMware Management Interface package for managing virtual machines from a browser, you need to install IIS on the host server before installing GSX Server. IIS is installed by default on Windows 2000 Server, so this additional step is not required on that platform prior to installing GSX Server.

Finally, although Setup lets you install GSX Server inside a virtual machine running under VMware Workstation, you won't be able to power up a virtual machine under GSX Server in this configuration. You must install GSX Server on a physical computer, and VMware recommends a minimum of 512 MB of RAM for the host operating system. A host system with 1 GB of RAM would allow 512 MB for the host and 256 MB each for two clustered virtual machines.

Creating the first virtual machine
With GSX Server installed, you're ready to start creating the virtual machines that will comprise the cluster. Clustering Windows Server 2003 requires either the Enterprise Edition or the Datacenter Edition. Because of pricing and availability, I'll assume you're using the Enterprise Edition. But first, I'll give you a quick overview of virtual machines before we dive into creating the first one, in case you are new to VMware.

Each virtual machine resides in a folder on one of your host system's hard disks. The virtual machine is composed of several files including a virtual BIOS file, a configuration file, and one or more virtual disk files. (I won't cover using physical disks for virtual machines because of the relative complexity of it.) The virtual disk files contain the data that makes up the virtual machine's hard disk. You can add multiple virtual disks to a virtual machine, and we'll take advantage of that shortly to create the cluster's quorum disk. You can also easily add other virtual hardware components, such as network adapters, either by running a wizard in VMware or by manually editing the virtual machine's configuration file in Notepad. We'll use both approaches here.

To create the first virtual machine, choose Start | Programs | VMware | VMware GSX Server or double-click the GSX Server icon on the desktop. Enter the serial number if this is the first time you have run GSX Server. Then, click the New Virtual Machine icon to start the New Virtual Machine Wizard. Choose the Typical installation option, which will enable you to specify the guest operating system, virtual machine folder path, and network connection type. These are your options for networking:
  • Bridged networking: Choose this option if you have separate IP addresses you can assign to your virtual machines. You can't use DHCP for clustering, so make sure you have two additional valid IP addresses on your host computer's subnet if you choose this option.
  • Network address translation: This option performs network address translation between the guest and the host. It enables Internet access by the VM and the capability to share files between the two.
  • Host-only networking: This option limits network connectivity by the guest virtual machines to only the host.

Keep in mind that you don't necessarily need access to the outside world by the virtual machines unless you're going to have clients on the network access the cluster. The VMs will need two network interfaces each, with each interface on a different subnet. In my test environment I used 192.168.0.n and 192.168.1.n subnets for the two interfaces. For example, one VM used and, and the other VM used and I used for the public cluster IP address.

After the wizard finishes, install Windows Server 2003 Enterprise Edition into it. To do so, insert the Windows Server CD in the computer, open the VMware GSX Server console, open the virtual machine you just created, and click Start This Virtual Machine or click the Power On button on the toolbar. Boot the VM from the Windows Server CD and run Windows Server Setup to install the operating system. At this point, you can set it up as a stand-alone server.

After Windows Server is installed in the first VM, you need to add the second network interface to it. Shut down the VM and, with the VM open and selected (but not started) in the GSX Server console, click Edit Virtual Machine Settings or choose VM, Settings to open the Hardware tab, as shown in Figure A.

Figure A
Use the Hardware tab to view existing virtual hardware and add other virtual components.

Click Add to open the Add Hardware Wizard and then click Next. Select Ethernet Adapter from the list and click Next. Choose the desired connection type and click Finish.

Cloning the VM
With the first VM created, you have a few options for creating the second VM. First, you could simply create another new VM and run the Windows Server 2003 Setup in it to install another copy of Windows Server. A second option is to use Sysprep to clone the first copy. Another is to use the Template Deployment Wizard in VMware VirtualCenter to clone the VM. A fourth option is to simply copy the folder containing the VM to another folder.

The last option can lead to SID conflicts, although I was able to create a sample cluster using this method and tweaking the network adapter settings in the configuration file. However, I recommend that you use Sysprep to clone the VM.

To do so, boot the newly created Windows Server 2003 VM. Extract Sysprep.exe from \Support\Tools\ on the Windows Server 2003 CD. Next, run Sysprep, which removes the SID and makes other changes, such as resetting the network configuration. Shut down the VM and power it off. Copy the contents of the VM’s folder to a new folder. For example, if the first VM is located in C:\VMs\Windows Server 2003, copy the folder to C:\WMs\Windows Server 2003 Node 2.

When you're done, create other instances of the VM as needed. We’ll keep it to just two nodes in this example. Open the GSX Server console and create a new VM named Windows Server 2003 Node 2. In the wizard, choose the path to the location of the cloned VM and choose the option to use an existing virtual disk. Select the disk in the Node 2 folder and complete the wizard.

At this point, you have two virtual machines, each of which you'll have to configure the first time you boot them. Boot the original VM and configure its network and other settings as needed, then shut it down. Boot the second VM and configure its settings, then shut it down.

Configuring the cluster drives
In this test scenario, the cluster will be built with a shared quorum drive and a shared resource drive, just as if you had two physical computers connected to a common SCSI bus and shared physical storage. In other words, both VMs will each have their own virtual disks for the operating system but will share two virtual disks for clustering.

The next step is to add the shared quorum drive. Begin by selecting the first node’s VM in the GSX Server console, then click Edit This Virtual Machine. Click Add again to start the Add Hardware Wizard and then click Next. Choose Hard Disk from the Hardware Types list and click Next. When the Select A Disk screen appears, choose Create A New Virtual Disk, as shown in Figure B, and click Next.

Figure B
Add a new virtual disk to the first VM to serve as the quorum disk.

Choose SCSI and click Next. The quorum disk only needs to be about 500 MB, so set the size at 1 GB and then click Next. Use the filename Quorum.vmdk as the disk name and click Finish. The disk will be located by default in the folder containing the VM.

Repeat the above steps one more time to add a second virtual SCSI disk to the virtual machine, but name this one Resource.vmdk. Size the drive according to what you plan to put on it for testing purposes.

Boot this VM and use the Disk Management console to partition and format the two new drives using NTFS. Then, shut down and power off the VM.

At this point, you need to modify the VM’s configuration file to allow SCSI bus sharing. Close the GSX Server console and open the VMX file from the first VM’s folder into Notepad. Scan through the file for the lines that start with scsi. Unless you choose a different SCSI configuration from the default, you should see several lines that start with scsi0. This is the virtual bus on which the SCSI drives reside. Add the following two lines to the end of the configuration file:

Now, add the two SCSI drives to the second cluster node and configure it for shared bus. Open the GSX Server console and open the second node’s VM. Click Edit This Virtual Machine, run the Add Hardware Wizard, and add a hard disk to the VM. In the wizard, choose the option Use An Existing Virtual Disk and choose the Quorum.vmdk virtual disk that resides in the first node’s virtual machine folder. Add a second new SCSI disk to the second node, again choosing the option to use an existing virtual disk. Point it to the Resource.vmdk created for the first node. Close the GSX Server console and open the VMX configuration file for the second node in Notepad. Add the scsi0.sharedBus and disk.locking lines to it just as you did to the first node’s configuration file.

Get clustering
Now you have two virtual machines that share storage, and you're ready to start building your first cluster. Boot the first node and run dcpromo to promote it to a domain controller. Alternatively, you can make it a member server in your existing domain. Then, boot the second node and configure it as a member server or domain controller. When both virtual servers are up and running, open the Cluster Administrator on the first node and create the cluster. See our Cluster Administrator Guided Tour for details on how to create and configure the cluster.