"Boot from SAN" step by step with Windows 2012 R2 and Cisco UCS using Brocade and EMC VNX.

“Boot from SAN” step by step with Windows 2012 R2 and Cisco UCS using Brocade and EMC VNX.
UCS:
  • Create service profile for windows server.
  • Create “Boot from san” boot policy
    • Setup SAN primary and secondary target.
    • The WWN required are that of your VNX array ports.

Brocade:
  • Login to create an initial zone for one of the ports.
  • Create new Alias
    • Type in the Alias name and select the WWN from blade
  • Create zone
    • Select the blade Alias and VNX Storage processor
  • Add to Zone configurations
  • Activate

VNX:
  • Start EMC Unisphere
  • Create Initiator
    • WWN/IQN can be obtained from UCS director
      • Open properties window for service profile of server
      • Select storage tab
      • At top copy the World Wide Node Name  (this is the first part of WWN/IQN)
      • Under vHBAs copy the WWPN
    • Now combine the WWNN and WWPN and with “:” as separator paste into WWN/IQN
  • Select “New Host” radio button
    • Type in the server name and IP address
  • Create LUNs
  • Create Storage Group per server
    • Associate the hosts
    • Associate the LUNs
Server:
  • Start the server and boot from Windows disk
  • Load the UCS disk drivers when asked for during installation and selection of the installation disk.
  • Verify disks show up and select where it will be installed.
  • After installation is completed and Windows is up and running, go ahead and install EMC Powerpath!

Cisco UCS – step by step configuration

As mentioned I don’t go into too much details on my post since i think there are a lot of other great blogs and vendor documentation out there.  Here is my short bullet point task list.   If I am missing anything please let me know.
Set equipment policies:
  • Equipment tab -> equipment – > policies tab
    • Chassis/fex discovery policy
      • Action = 4 ports
      • Link grouping preference = port channel
    • Power policy = grid
Configure server/uplink port:
  • Equipment tab -> select FI-A/B -> expand -> fixed modules
    • Configure the appropriate unconfigured ports as “Server” (connections between IOM and Fabric Interconnect) and “Uplink” (connection to network)
Configure FC storage ports
  • Equipment tab
  • All the way bottom, select FI A
    • Right hand side select Configure unified ports
    • Run slider to end of fiber storage ports you need
    • This will reboot FIA, after reboot re-login.
  • Select FI B
    • Perform same steps
Create Port Channels:
  • Setup ports as uplink ports
  • LAN TAB
    • Fabric – Port Channels
    • Setup port channel ..set same PORT ID on both Fis
  • SAN TAB ( will not be creating port channel due to connection to Brocade)
    • San Cloud -> Fabric A -> Under general tab select “create Port Channel”
Create VSANs: (brocade):
  • SAN > SAN Cloud > Fabric A > VSANs (both Fabric A & B)
    • Create VSAN
    • Select the specific Fabric A or B (not common)!
  • Assign VSAN to FC uplinks
    • Equipment tab -> Fabric interconnect A & B -> Fixed modules -> FC ports
      • Select FC port
      • Under general tab click drop down for VSAN.
        • Select VSAN which is associated to FI.
Upgrade firmware
  • . An “*.A.bin” file and a “*.B.bin” file. The “*.B.bin” file contains all of the firmware for the B-Series blades. The “*.A.bin” file contains all the firmware for the Fabric Interconnects, I/O Modules and UCS Manage
  • Equipment tab -> Eqiupment -> Firmware management
  • Download firmware
  • Update firmware (view progress under Firmware auto install -> general tab, or press Apply to view status in same window))
    • Adapters
    • CIMC
    • IOMs
  • Activate firmware in the following order:  Choose “Ignore Compatibility Check” anywhere applicable.
    • Adapters
    • UCS manager
    • I/O Modules
    •  Choose “skip validation” anywhere applicable. Make sure to uncheck “Set startup version only”, since this is an initial setup and we aren’t concerned with rebooting running hosts
  • Activate subordinate FI and then primary FI
Create sub-organization
This is optional to create specific organizational servers/pools/policies for instance ESXi, SQL, Windows etc
  • Right click and root directory, select Create organization
  • Specify name
Create KVM IP pool:
  • Lab tab -> pools -> root -> IP Pools -> IP Pool ext-mgmt
  • Create block of IPv4 Addresses
    • Specify IP range
Create Server pool
  • Servers tab -> Pools -> Sub-Organization -> -> Server pools
  • Create server pool
Create UUID suffix pool
  • Servers tab -> Pools -> Sub-Organization -> -> UUID Suffix Pool
  • Create UUID suffix pool
  • Create Suffixes
Create MAC pool
  • For each suborganization create 2 groups of MAC pools. 1 for FI-A and 1 for FI-B
  • LAN TAB: -> Pools -> Root -> MAC Pools
    • Create new pool for A
    • Create block
    • Create new pool for B
Create HBA pools:
  • SAN TAB:
    • Pools -> root -> sub-organization -> WWNN Pools
      • Create WWNN pool
        • Add double the amount since each server will have two HBA’s
    • For WWPN we will again create separate pools for FI-A and FI-B:
      • Pools -> root -> sub-organization -> WWPN Pools
        • Create WWNN pool for FI-A
        • Create WWNN pool for FI-B

Create VLANS:
  • LAN TAB -> Lan -> Lan Cloud -> VLANs
    • Create new VLANs
    • Provide name and ID
Create vNICs templates:
  • LAB TAB -> LAN -> Policies -> root -> Sub-organization -> vNIC templates
    • Create vNIC template (this is again done for each FI-A and FI-B
Create VHBAs templates:

  • SAN TAB -> Policies -> root -> sub-organizations -> vHBA templates
    • Create vHBA Templates for both FI-A & FI-B

Create a Service Profile Templates:
Servers tab -> Servers -> Service Profiles -> root -> Sub-organizations
  • Create service profile template
Under networking select expert.
Click Add
Select Use vNIC template
Storage, select Local storage SD card policy
Select WWNN assignment policy
Select Expert connectivity
Create vHBA
Next zoning, leave defaults since we using Brocades
Set PCI ORDER
Select vMedia to use, default
Server boot order, select boot policy create for SDCard

Select Maintenance policy create earlier
Select server assignment
Operational Policies
Set Bios policy
Deploy service profile from template
Servers tab -> Service profile template -> root -> sub-organizations
Right click server profile template and select “create service profiles from template”
Select naming prefix
Configure call home:
Admin tab -> Communication Management -> call home
Turn on and fill in the requirements
In profiles tab add “callhome@cisco.com” to Profile CiscoTAC-1
Under call home policies add the following to provide a good baseline
Configure NTP:
Admin tab -> Time zone management
Add NPT servers
Backup configuration:
Admin tab -> ALL -> Backup configuration on right hand side pane
Select “create backup operations
Admin state = enables

Select location = local file system



For setting policies i created another blog:

Cisco UCS – configure policies

Set Policies:
Network control policies (enable CDP)
  • LAB tab -> Policies -> root -> sub-organizations -> network control policies
    • Create network control policy
    • Enable CDP
Bios Policy:
  • Servers tab -> Policies -> root -> sub-organizations -> Bios Policies
  • Create bios policies
    • Mostly setting cpu settings
Host Firmware:
  • Servers tab -> Policies -> root -> sub-organizations -> Host Firmware Packages
  • Create host firmware package
    • Set simple and only blade package version.
Local disk configuration:
  • Servers tab -> Policies -> root -> sub-organizations -> Local disk config policies
    • Create local disk configuration policy
      • This is to setup SD card
        • Disable protect configuration
        • Enable flexflash state
        • Enable flexflash RAID reporting state
      • For SAN boot
        • Set mode to No local storage
Maintenance policy:
  • Servers tab -> Policies -> root -> sub-organizations -> maintenance policies
    • Create Maintenance Policy
Boot policy:

  • Servers tab -> Policies -> root -> sub-organizations -> boot policies
    • Create boot policy
      • Expand local devices and add to boot order
        • Start with Local CD, then remote virtual drive then SD card

vCloud director – running MAC OS X and Windows VM in same vApp.

Recently we installed in Mac Pro 6,1 hardware and provided MAC OS X virtual machines to our vCloud director environment.

This was configured in a separate cluster in vCenter server which provides all the regular capabilities like HA, DRS and vMotion which is great. Also created a separate storage cluster and assigned a new MAC storage profile to this cluster which was made available within vCD.

MAC OS X templates was created and added to catalog with storage set to pre-provision on default storage profile create for MAC cluster.

Problem: 

In vCloud director a new Provider VDC was created and linked to the new vCenter server cluster.
Within the existing Organizations we created an additional virtual datacenter with MAC provider VDC selected.  This created new resource pool in the cluster.

The users where now able to deploy MAC OS templates to this VDC, however a request came back quickly that users need to deploy both MAC OS X and Windows VM within the same vApp.

Troubleshooting: 

The configuration as explained above obviously does not allow for this situation since during deployment of vApp you can only select a single VDC to deploy too as well for adding a VM you can only specify a single storage Policy, which will also be the one assigned to vApp’s VDC .  So all the VMs would need to run on the Apple hardware cluster which is not idea.

Solution:

Solution was pretty simple and can be accomplished by merging your provider VDC’s which was introduced in vCloud director 5.1.1.

  1. Login to vCloud director as system admin.
  2. Select Manage & Monitor
  3. Under Cloud resources select Provider VDCs
  4. Right click the MAC provider VDC and select Merge
  5. Select the Provider VDC that you want this merge with.
  6. After completion you will now see your Provider VDC has  additional resource pool, datastores and ORG VDCs
  7. I then went ahead and deleted the VDC I initially created for the MAC deployments since only need the original VDC.
Now when you deploy a vApp select the existing VDC which contains the MAC Provider VDC and storage profile.  Same can be accomplish for deploying VM within a vApp.

Mac Pro 6,1 rack environment running VMware ESXi 5.5 (with Fiber connectivity to VNX)

With the recently addition of Mac Pro 6,1 to VMware’s hardware compatibility I was eager to replace our exiting old Power Mac G5 towers in our environment.
Prerequisites:

  1. Mac Pro bios version MP61.88Z.0116.B05.1402141115
If your Mac Pro has an older boot ROM then just upgrade the Mac Pro to Yosemite (OS X 10.10) which contains the update to be applied to the Mac Pro.
  1. vSphere 5.5 P03 is required
Currently the latest version of ESXi available on VMware download is only 5.5 update 2 so you have to include the required patch version onto the update 2 ISO.  To do this perform the following steps:


  • Download the latest ESXi 5.5 Update 2 Driver rollup
  •  Download the offline bundle for ESXi Update 2 patch 3
  • Next you need to convert the offline bundle zip file into an ISO file to be placed on bootable USB stick.  To do this I used the VMware Image Builder which is available as part of PowerCLI.
    • After you installed PowerCli open the application
    • Change to folder location where zip file resides
    • Run command to add the offline bundle: 
      • ‘add-esxsoftwaredepot .\ESXi550-201412001.zip’
    • Run command to see the image profile: 
      • ‘get-esximageprofile’ 
    • Select the ESXi-5.5.0-20141204001-standard which include VMware tools and security patches.
      • Run Command:  
        • ‘New-EsxImageProfile -CloneProfile “ESXi-5.5.0-20141204001-standard” -name “ESXi55u2-P03-MACPRO” -Vendor MACPRO66’
    • Now you can create the ISO file with running command:
      • ‘Export-EsxImageProfile -ImageProfile “ESXi55u2-p03-MACPRO” -ExportToISO -filepath H:\VMware-ESXi-5.5u2-P03-MACPRO.iso’
    • This file can now be places on a bootable USB.
      • I use Universal-USB-installer or UnetBootin to place the ISO on the USB.
  • Boot ISO from MAC
    • press and hold the “ALT” key on keyboard to boot the USB. 
  • Rest is the same basic installation as with any regular Intel based server
Here is a list of all the hardware items used to in our environment:
  • Sonnet xMac Pro rackmount enclosure.
    • This the most valuable piece of equipment and I highly recommend this if you planning on placing your Mac Pro’s in a server rack. 
    • Comes with 3 x PCIe slots available through thunderbolt which provide 20Gbps throughput and flexibility which is unmatched and can now add extra network and even Fiber connections for storage.
    • Do yourself the favor and check them out:
    • UPDATE:
  • APPLE Mac Pro 6,1  
    • 12GB memory
    • Intel Xeon CPU E5-2697 v2 @ 2.70GHz
    • only purchase small memory size to be replaced with Transcend
  • Transcend 128GB Kit for Mac Pro
  • Intel I350-T4
    • 4 port network card.  We actually have two card installed in Sonnet.
    • This card is VMware compatible but not Sonnet however it works great without issues
  • 1 x Atto Celerity FC-81EN Fiber Channel adapter
  • APC AP7731
      • Since there are no dual power supplies on Mac Pro we purchased this APC switched rack PDU which takes two single-phase 220V drops and can switch power if you have a failure on one of over This provides redundancy even though you only have one cable.  However if the power adapter fails on hardware you are out of luck. 
Some gotchas experienced:
– We tried to run the updates for ESXi through VMware update manager and this caused the onboard NICs on Mac Pro to not be recognized anymore.  Re-installed the old version to resolve this.  Current build is 2302651
– To add storage on VNX a rescan does not seem to work so we had to restart the Mac Pro in order to pick up the LUNs.
– We initially installed all the PCI cards and then installed ESXi.  This cause the network cards numbering to go out of whack.  What we had to do was remove all the cards and power on ESXi and let it complete the startup.  Then shutdown and add a single PCI card  and power on again, Do this one at a time in order you want starting from bottom.    Should fix the network port order.
UPDATE:  created a new blog to show how within vCloud Director to run a MAC OS X and Windows VMs within the same vAPP

http://virtualrealization.blogspot.com/2015/04/vcloud-director-running-mac-os-x-and.html

Here are some photos of our build:

Started installation

ESXi installed and ready for use

Internal and external casing

Internal casing housing the Mac Pro

work bench

rear of Sonnett chassis. very nice

Mac Pro housing

More Mac Pro housing

Mac Pro ready to be installed

Now for some pictures in the rack:
APC PDU

Rear of Rack

Front of Rack..so nice and clean!

Cisco UCS error: will_boot_fault sensor failure asserted

After replacing a faulty UCS blade the following error presented itself after inserting the new blade into chassis:  will_boot_fault sensor failure asserted
Troubleshooting:
Ran the following –
Tried to run the board controller activation
# scope server
# scope boardcontroller
# show version
Showed version 13 (2.2.3d)
# show image
Did not display version 13 but latest was version 8 (2.2.1d)
# activate firmware .0 force
# commit-buffer
Received an error message that commit cannot proceed.
Solution:
The problem turned out to be the new blade had a newer version of firmware installed that what I had loaded in UCS manager. 
To fix this upload the latest firmware version ( in our case 2.2.3d).
Verify the new version is available by running show image under scope for chassis/server/boardcontroller.

  • Run the same process as listed above

Dell DRAC and Internet Explorer 11

I don’t think i have ever had more problems that i have had with my IE browser and Dell DRAC interface 🙂
Seems that there is always something broken and can either never get logged in, or show me the turning circle of death or just a blank white screen.
I am still a big IE browser user, yes yes I know, mainly due to compatibility but I think it is getting worse with newer versions and the whole implementation of their compatibility view is just crazy. 
Already downgraded from IE 11 back to IE 10, hence why i am writing this email since for the life of me I could not get the DELL DRACs to work. Alas I am back on IE 11 and have this working..
Problems:
If you cannot login and you know you typing the correct username and password:
Delete your internet browser history, trust me for some reason this works.
If you see blank white screen:
Add the site address to trusted site and check your activeX settings in IE
Add site address to the compatibility view list in IE
If you see the turning circle of death as I call it:
Make sure that server address as added to your Java control panel.
Open control panel
Select security tab
Set security level to appropriate level
Add the site address at bottom to exception site list to allow running after security prompts.
IF you still get stuck my final recommendation would be the following:
  • Upgrade your DRAC to latest firmware version
  • In IE 11 press F12 to bring up the debug bar.  
    • Scroll down to Networks button and select
    • Click on the “Always refresh from server” button (3rd from left).