Month: May 2015

vRealize configuration Manager – unable to run reports

/ johannstander / Leave a comment

When trying to run reports we receive the following error:

An error has occurred during report processing. (rsProcessingAborted)
Cannot create a connection to data source ‘ECM’. (rsErrorOpeningConnection)

For more information about this error navigate to the report server on the local server machine, or enable remote errors
Troubleshooting:
Login to data source on report server to verify authentication which was successful
Worked with VMware support on solution
Solution:
add basic to the end of the AuthenticationTypes string in the rsreportserver.config file of the report server.
C:\Program Files\Microsoft SQL Server\MSRS10_50.MSSQLSERVER\Reporting Services\ReportServer

vCenter Server – lookup service failed

/ johannstander / Leave a comment

Recently did a rack migration for which we had to shutdown SQL cluster and vCenter server including all Virtual machines.

After powering I was unable to login with my domain account and got the following error:

“Failed to connect to VMware Lookup Service. SSL certificate verification failed”

Troubleshooting:

  • Login to vCenter server (SSH or RDP depending on vCenter server platform) and verified that the vmware-sso service is running.
  • Could not login with local admin@vsphere.local so logged into VA configuration https://:5480.  Verified all services was running.
  • Verified SSO server and Active directory domain controllers time is not off by more than 5 minutes.
  • Found KB article on this error.

Solution:

Found KB that provides information on this issue which seems to get triggered when you had change the IP or hostname of the vCenter server after its initial creation.

To fix the issue login to VA configuration: https://:5480.
Select admin tab
Check the “yes” radio button for “Certificate regeneration enabled:”
Click Submit.
Restart vCenter server

Link:

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2058430&sliceId=1&docTypeID=DT_KB_1_1&dialogID=638751240&stateId=0%200%20638753748

Unable to power on VM with SCSI bus sharing enabled – error: "Could not open/create change tracking file"

/ johannstander / Leave a comment

So recently i was asked to assist with setting up a new SQL 2012 AlwaysOn Cluster for POC.
During the configuration of the VM’s we had to setup a quorom disk which is shared between both servers for MSCS.
The shared disk I configured and select virtual device node on newly created SCSI controller, LSI Logic SAS with SCSI bus sharing set to Physical.
On the 2nd VM i created the same SCSI controller as above and instead of creating new disk the existing disk was select.

With Primary powered on and trying to power on the 2nd SQL server i receive the following error:
“Could not open/create change tracking file”

Troubleshooting:

The error eludes to problem with CBT on the Virtual server where i cannot access the cbt file due to the primary VM already running and locked the file.

I could not find much information in documentation regarding setting up clusters or shared vmdk disk and making use of CBT.

It does however seem that CBT has some restrictions:

  1. The virtual machine version earlier than 7
  2. The virtual machine has a Raw Device Mapping (RDM) in the physical compatibility mode.
  3. The virtual machine has a disk attached to a shared virtual SCSI bus.  Use of CBT only possible when SCSI bus sharing is set to none.

Following disk formats supported:
•Thin and Thick virtual disks
•VMDK and RDM (virtual only)
Solution:

Disable CBT for VM: (this can be set for all devices or just specific devices)
  • Edit settings of virtual machine
  • Select options tab
  • Click General under advanced -> configuration parameters
  • Click Add row
  • Add “ctkEnabled and set value “false”  (this will set for all devices)
  • OR
  • Add “scsi0:0.ctkEnabled and set value “false” (this will set for specific devices)

 To prevent any third-party applications from enabling Change Tracking on the virtual machine:
  • SSH to ESXi host
  • browse the datastore where VM resides
  • Open the .vmx file of the virtual machine using a text editor. 
  • Add this entry to the file:
    • ctkDisallowed=”true”

vCenter Server – Host Profiles – set NTP

/ johannstander / Leave a comment

Setting your host with NTP services using host profiles requires a few changes to be made in different settings on host profiles.

Solution:

Edit the host profile

  • Lets first set the NTP servers:
    • General system settings -> Date and Time configuration
    • Click Date and time configuration
    • Time settings – “Configure a fixed NTP configuration”
    • List of time servers – add server IP address or FQDN for NTP servers.  Use comma to list multiple servers
  • Now lets enable the NTP service:
    • Security and services -> Services configuration -> Services configuration
    • Select NTPD (make sure this is enabled)
    • On service verify the dropdown box is set to “Establish a fixed service configuration”
    • Policy that dictates how the service will be handled – type “on”
  • Now lets set the firewall rule:
    • Security and services -> Firewall configuration ->  Firewall configuration -> Ruleset Configuration
    • Select ntpClient (make sure this is enabled)
    • Verify dropdown “Specify the configuration of a fixed ruleset.
    • Check the box for enabled  on “Flag indication where ruleset should be enabled”
    • Check the box for enabled on “Flag indication where ruleset should allow all ip addresses”  (this is self explanatory)

vCenter server – host profiles error "host state doesn’t match specification…"

/ johannstander / Leave a comment

Setup host profiles and when applied on other hosts i get receive the error message:

Seems that ESXi treats SAS disk as shared disks instead of local disks which causes these errors when you create your host profile from an existing ESXi host.

Solution:

The only workaround I found is to disable the following settings in the host profile –

  • Storage configuration – > Pluggable Storage Architecture (PSA) configuration
    • disable “PSA device settings”
    • disable “PSA device configuration”
  • Storage configuration -> Native Multi-Pathing NMP
    • disable “PSP and SATP configuration for NMP devices
Links:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2002488

Cisco UCS – step by step configuration

/ johannstander / Leave a comment
As mentioned I don’t go into too much details on my post since i think there are a lot of other great blogs and vendor documentation out there.  Here is my short bullet point task list.   If I am missing anything please let me know.
Set equipment policies:
  • Equipment tab -> equipment – > policies tab
    • Chassis/fex discovery policy

      • Action = 4 ports
      • Link grouping preference = port channel
    • Power policy = grid
Configure server/uplink port:
  • Equipment tab -> select FI-A/B -> expand -> fixed modules
    • Configure the appropriate unconfigured ports as “Server” (connections between IOM and Fabric Interconnect) and “Uplink” (connection to network)
Configure FC storage ports
  • Equipment tab
  • All the way bottom, select FI A
    • Right hand side select Configure unified ports
    • Run slider to end of fiber storage ports you need
    • This will reboot FIA, after reboot re-login.
  • Select FI B
    • Perform same steps
Create Port Channels:
  • Setup ports as uplink ports
  • LAN TAB
    • Fabric – Port Channels
    • Setup port channel ..set same PORT ID on both Fis
  • SAN TAB ( will not be creating port channel due to connection to Brocade)
    • San Cloud -> Fabric A -> Under general tab select “create Port Channel”
Create VSANs: (brocade):
  • SAN > SAN Cloud > Fabric A > VSANs (both Fabric A & B)
    • Create VSAN
    • Select the specific Fabric A or B (not common)!
  • Assign VSAN to FC uplinks
    • Equipment tab -> Fabric interconnect A & B -> Fixed modules -> FC ports
      • Select FC port
      • Under general tab click drop down for VSAN.
        • Select VSAN which is associated to FI.
Upgrade firmware
  • . An “*.A.bin” file and a “*.B.bin” file. The “*.B.bin” file contains all of the firmware for the B-Series blades. The “*.A.bin” file contains all the firmware for the Fabric Interconnects, I/O Modules and UCS Manage
  • Equipment tab -> Eqiupment -> Firmware management
  • Download firmware
  • Update firmware (view progress under Firmware auto install -> general tab, or press Apply to view status in same window))
    • Adapters
    • CIMC
    • IOMs
  • Activate firmware in the following order:  Choose “Ignore Compatibility Check” anywhere applicable.
    • Adapters
    • UCS manager
    • I/O Modules
    •  Choose “skip validation” anywhere applicable. Make sure to uncheck “Set startup version only”, since this is an initial setup and we aren’t concerned with rebooting running hosts
  • Activate subordinate FI and then primary FI
Create sub-organization
This is optional to create specific organizational servers/pools/policies for instance ESXi, SQL, Windows etc
  • Right click and root directory, select Create organization
  • Specify name
Create KVM IP pool:
  • Lab tab -> pools -> root -> IP Pools -> IP Pool ext-mgmt
  • Create block of IPv4 Addresses
    • Specify IP range
Create Server pool
  • Servers tab -> Pools -> Sub-Organization -> -> Server pools
  • Create server pool
Create UUID suffix pool
  • Servers tab -> Pools -> Sub-Organization -> -> UUID Suffix Pool
  • Create UUID suffix pool
  • Create Suffixes
Create MAC pool
  • For each suborganization create 2 groups of MAC pools. 1 for FI-A and 1 for FI-B
  • LAN TAB: -> Pools -> Root -> MAC Pools
    • Create new pool for A
    • Create block
    • Create new pool for B
Create HBA pools:
  • SAN TAB:
    • Pools -> root -> sub-organization -> WWNN Pools
      • Create WWNN pool
        • Add double the amount since each server will have two HBA’s
    • For WWPN we will again create separate pools for FI-A and FI-B:
      • Pools -> root -> sub-organization -> WWPN Pools

        • Create WWNN pool for FI-A
        • Create WWNN pool for FI-B

Create VLANS:
  • LAN TAB -> Lan -> Lan Cloud -> VLANs
    • Create new VLANs
    • Provide name and ID
Create vNICs templates:
  • LAB TAB -> LAN -> Policies -> root -> Sub-organization -> vNIC templates
    • Create vNIC template (this is again done for each FI-A and FI-B
Create VHBAs templates:

  • SAN TAB -> Policies -> root -> sub-organizations -> vHBA templates
    • Create vHBA Templates for both FI-A & FI-B

Create a Service Profile Templates:
Servers tab -> Servers -> Service Profiles -> root -> Sub-organizations
  • Create service profile template
Under networking select expert.
Click Add
Select Use vNIC template
Storage, select Local storage SD card policy
Select WWNN assignment policy
Select Expert connectivity
Create vHBA
Next zoning, leave defaults since we using Brocades
Set PCI ORDER
Select vMedia to use, default
Server boot order, select boot policy create for SDCard

Select Maintenance policy create earlier
Select server assignment
Operational Policies
Set Bios policy
Deploy service profile from template
Servers tab -> Service profile template -> root -> sub-organizations
Right click server profile template and select “create service profiles from template”
Select naming prefix
Configure call home:
Admin tab -> Communication Management -> call home
Turn on and fill in the requirements
In profiles tab add “callhome@cisco.com” to Profile CiscoTAC-1
Under call home policies add the following to provide a good baseline
Configure NTP:
Admin tab -> Time zone management
Add NPT servers
Backup configuration:
Admin tab -> ALL -> Backup configuration on right hand side pane
Select “create backup operations
Admin state = enables

Select location = local file system



For setting policies i created another blog:

Cisco UCS – configure policies

/ johannstander / Leave a comment
Set Policies:
Network control policies (enable CDP)
  • LAB tab -> Policies -> root -> sub-organizations -> network control policies
    • Create network control policy
    • Enable CDP
Bios Policy:
  • Servers tab -> Policies -> root -> sub-organizations -> Bios Policies
  • Create bios policies
    • Mostly setting cpu settings
Host Firmware:
  • Servers tab -> Policies -> root -> sub-organizations -> Host Firmware Packages
  • Create host firmware package
    • Set simple and only blade package version.
Local disk configuration:
  • Servers tab -> Policies -> root -> sub-organizations -> Local disk config policies
    • Create local disk configuration policy

      • This is to setup SD card
        • Disable protect configuration
        • Enable flexflash state
        • Enable flexflash RAID reporting state
      • For SAN boot
        • Set mode to No local storage
Maintenance policy:
  • Servers tab -> Policies -> root -> sub-organizations -> maintenance policies
    • Create Maintenance Policy
Boot policy:

  • Servers tab -> Policies -> root -> sub-organizations -> boot policies
    • Create boot policy
      • Expand local devices and add to boot order
        • Start with Local CD, then remote virtual drive then SD card

DPA – Automate the conversion of report from CSV to Excel and upload to Sharepoint

/ johannstander / Leave a comment

Data Protection Advisor is a great application providing alerts and reports.

I created a scheduled custom report to show daily backups from Avamar.  The report job however only saves to CSV and cannot export file to a share.  You can publish to Sharepoint but this only compatible with Sharepoint Server 2013 so this was not an options for us.

I create a batch file which copies the CSV file from the website using wget.
I then run a powershell script to convert the CSV file to XLSX.
Thanks to Boe Prox – https://gallery.technet.microsoft.com/office/7c56c444-2476-4625-b1d9-821f30280e44
After conversion is completed the file is copied to a Sharepoint document library.
I then setup a windows task scheduler to run the batch file.

A couple of problems I ran into:

On a windows 2008 R2 server when trying to open the Sharepoint document library in windows explorer gives you the following error message:

“your client does not support opening this list with windows explorer”
To resolve this:
Install Desktop Experience on Windows 2008 R2
Go to features -> Add features -> Select “Desktop experience”
When i ran the batch file from windows task scheduler I got the following error “Microsoft Excel cannot access the file”. 
To resolve this:
You have to create a folder (or two on a 64bit-windows):
(32Bit, always)
C:\Windows\System32\config\systemprofile\Desktop
(64Bit)
C:\Windows\SysWOW64\config\systemprofile\Desktop
Task scheduler failed to copy the file to mapped network drive
To resolve this:
Changed the copy to UNC path
Solution:  (can copy and paste into batch file)

“C:\Program Files (x86)\GnuWin32\bin\wget” -N –http-user= –http-password= –no-check-certificate –secure-protocol=sslv3 https://:9002/dpa-api/scheduledreport/results/DailyBackups.csv
sleep 5
powershell -NoProfile -command “. .\ConvertCSV-ToExcel.ps1”; “” > E:\DPAreports\convert.log
sleep 5
copy /Y “E:\DPAreports\Daily Backup Report.xlsx” “\\@SSL\DavWWWRoot\DPA Reports” > E:\DPAreports\copy.log