At the time of writing the Brocade 2 port 10GbE Converged Network Adapter for IBM BladeCenter (81Y1650) is supported on VMware ESXi up to version 4.1.

To install VMware on the blades, a custom VMware ESXi 4.1 ISO image with Brocade CNA drivers is required.

The Brocade 2 port 10GbE Converged Network Adapter for IBM BladeCenter drivers are available on Brocade web site:

http://www.brocade.com/services-support/drivers-downloads/adapters/IBM_BLADECENTER.page

Download the latest Adapter Driver Package for WMware ESX 4.0/4.1 (brocade_driver_esx4x_v2-3-0-2.tar.gz at the time of writing).

I used the VMware KB Adding third-party device drivers to ESXi 4.1 to include the Brocade CNA drivers into the installation media.

First, download and install the VMware VIBDDI Image Customization Tool for ESXi 4.1. I recommend using 32-bit version for Red Hat Enterprise Linux 5. Anyway, vtar binary is 32-bit only inside the 64-bit package.

Copy the ESXi 4.1 installation media ISO image to /opt/vmware/vibddi/iso.

Mount (loop) the ISO:

# mount -o loop /opt/vmware/vibddi/iso/<VMware-VMvisor-Installer-iso> /mnt

Decompress the imagedd.bz2 file:

# bunzip2 < /mnt/imagedd.bz2 > imagedd

Umount the ISO image:

# umount /mnt

View the contents of the ESXi image:

# vibddi -i imagedd -q
< oem-vmware-esx-drivers-net-vxge (400.2.0.28.21239-1OEM) >
< oem-vmware-esx-drivers-scsi-3w-9xxx (400.2.26.08.036vm40-1OEM) >
< vmware-esx-firmware (4.1.0-1.4.348481) >
< CMPI-1.0 >
< VMW_CMPI_CPP-1.0.5 >
< vmkapi_1_1_0_0 >
< vmknexus1kvapi-1-4 >
< vmkepsecapi_1.0.0.0 >
< DriverAPI-9.0 >
< DriverAPI-9.1 >
< vmware-esx-tools-light (4.1.0-1.4.348481) >


Extract the network and Fibre Channel HBA offline bundle ZIP files (BCD-bna-*offline_bundle-*.zip and BCD-bfa-*-offline_bundle-*.zip) from the two ISO files inside the Brocade Adapter Driver Package for WMware ESX 4.0/4.1.

Add the Fibre Channel HBA driver (bfa) to imagedd:

# vibddi -i imagedd -o BCD-bfa-*-offline_bundle-*.zip
Setting up configuration for esxupdate
This may take a few seconds...
Checking bootbank filesystems (pre-install)
Installing offline bundle: BCD-bfa-*-offline_bundle-*.zip
Update operation completed successfully
Checking bootbank filesystems (post-install)

Confirm the driver is in place (look for vmware-esx-drivers-scsi-bfa) in the query output:

# vibddi -i imagedd -q

Add the CNA network driver (bna) to imagedd and generate the custom installation image (-g flag):

# vibddi -i imagedd -o BCD-bna-*-offline_bundle-*.zip -g
Setting up configuration for esxupdate
This may take a few seconds...
Checking bootbank filesystems (pre-install)
Installing offline bundle: BCD-bna-*-offline_bundle-*.zip
Update operation completed successfully
Checking bootbank filesystems (post-install)
Generating customized ISO: /opt/vmware/vibddi/iso/VMware-VMvisor-*.x86_64.iso
This may take a few minutes...
Generated new ISO: /opt/vmware/vibddi/iso/custom/VMware-VMvisor-*.iso


Confirm both drivers are in place (look for vmware-esx-drivers-scsi-bfa and vmware-esx-drivers-net-bna) in the query output:

# vibddi -i imagedd -q

The VMware ESXi 4.1 custom installation image with Brocade 2 port 10GbE Converged Network Adapter for IBM BladeCenter is in /opt/vmware/vibddi/iso/custom/VMware-VMvisor-*.iso. Use the image to install ESXi 4.1 on the blades with Blade Tasks / Remote Control / Remote Console under BladeCenter H AMM.

Check about vHBAs and other PCI devices may stop responding in ESX/ESXi 4.1 when using Interrupt Remapping.

Install Solaris 11 Express HVM domU with virt-install:

virt-install -n sx11 -r 2048 --vcpus=4 --os-type=solaris --os-variant=opensolaris \
-v -c /path/to/sol-11-exp-201011-text-x86.iso --disk path=sx11.img,size=20 \
-b br1 --vnc --noautoconsole

Of course, adjust domU memory (-r), vcpus, path to Solaris 11 Express DVD, disk path, network bridge (-b).

After installation has started, you can connect to the VNC display. First, determine which port:

virsh vncdisplay sx11
:8

Finish the installation as usual. After reboot, you can start the Solaris 11 Express domU:

virsh start sx11

or have the domain autostarted at dom0 boot:

virsh autostart sx11

Or disable autostarting

virsh autostart --disable sx11

Enjoy!

Install Fedora 12 domU:
pfexec virt-install -n f12 -r 512 --vcpus=4 -f /dev/zvol/dsk/rpool/f12d0 -b e1000g0 --os-type=linux -p --nographics --os-variant=fedora11 -l http://fedora-12-mirror/fedora/linux/releases/12/Fedora/i386/os/

-n f12 – domU’s name
-r 512 – allocate 512 MB memory
–vcpus=4 – number of virtual CPUs allocated (make sure this number is lower or equal to the number of CPUs available)
-f /dev/zvol/dsk/rpool/f12d0 – ZVOL block device
-b e1000g0 – bridged networking to e1000g0 interface
-p – paravirtualized guest
–os-variant=fedora11 – “hack” until fedora12 OS-Variant will be integrated into xVM
-l http://fedora-12-mirror/fedora/linux/releases/12/Fedora/i386/os/ – replace fedora-12-mirror with a near Fedora 12 mirror, replace i386 with x86_64 for 64-bit domU

After you bring up the network, do a VNC installation.
Until xVM supports ext4 boot, make sure to create an ext3 /boot partition (or just a big ext3 / filesystem)

After installation, start the domU:
pfexec xm start -c f12
You will be presented the pygrub menu, just press Enter to boot.

Log into the domU and change the default timeout=0 to a different value (e.g. 5 seconds) in /boot/grub/grub.conf. Now you can use virsh start / shutdown commands.

To auto-start the guest on host boot:
virsh autostart f12

Tested on OpenSolaris 2010.02 preview snv_127.

Mount the new Boot Environment:
pfexec mkdir /mnt/devel
pfexec beadm mount devel /mnt/devel


Use the dev publisher:
pfexec pkg -R /mnt/devel set-publisher -O http://pkg.opensolaris.org/dev opensolaris.org

Update to the latest bits:
pfexec pkg -R /mnt/devel image-update -v

If everything went fine (after you read the Release Notes), activate the new BE:
pfexec beadm activate devel

Reboot:
pfexec shutdown -y -g1 -i6

Did this remotely. After reboot, system was up and running, including all xVM domU autostarted.