vCloud director 5.5: Catalogs problem #3 – Assigning catalog media files to different VDC within an ORG?

We have a public catalog which is shared to all ORGs within vCloud.  This also contains some ISO media files for users to mount to cdrom of virtual machines.
However in order for a user to make use of a media file from a public catalog, that media file firstly needs to be copy to the local catalog of that ORG.

Within this ORG we have multiple vDCs.  This is where things get interesting.

Scenario:

When a vAPP is deployed from catalog you assign this vAPP to a specific vDC.
When you then try to mount an ISO from the local catalog media files, you are unable to do so since this media file is associate to a different vDC…mmm

Troubleshooting:

I believe the problem lies with how the Catalogs are done in 5.5.  When creating or copying files into a Catalog, there is no method to accurately control where the item will be stored. This is more critical when there are multiple Org vDCs, and a file could be placed in an Org vDC to which the file is not intended for.

This is a departure from vCloud Director 5.1, where importing media allowed you to select a specific Organization vDC. Even when you could select a vDC, you still needed to have the ISO image in the vDC where the VM resides to be allowed to insert it.

I had a lot remote sessions and discussions with support on this issue since we could not find a way to assign a media file to specific VDC.

Workarounds:

Option 1 –  Temporarily disable storage profiles before doing an ISO import. This would eliminate certain vDCs as an option, and force the selection process to go to the remaining vDC.  this would be my recommendation.

Option 2 – Set the catalog to use a specific Storage Profile, from the desired Organization vDC and import the media again. This should send the imported files to the correct ORG vDC. You will need to import Media files multiple times to accomplish this.  This only possible if you have multiple storage profiles which I currently do not have.

Option 3 – Have a catalog specific to an individual ORG vDC, name the catalog relative to the ORG vDC, and select the Storage Class/Profile from the same ORG vDC. This means that any ISOs will be sent up to the correct vDC when you import to a given catalog.  On this option here are the steps to setup more catalog for more than one vDC:
1. Create a “master” catalog in the ORG on the Storage Profile vDC A
2. Publish the Catalog created in Step 1 externally
3. Create a “slave” catalog in the ORG on the Storage Profile vDC B, and have this catalog sync from the catalog created in step 1.
4. Repeat this to have Catalog C on Storage Profile C, and sync to Catalog A.
5. Whenever you add a new ISO image to the Master Catalog, it will automatically update the others in 24 hours or when a sync is requested.

vCloud director 5.5: Catalogs problem #2 – Error mounting catalog media ISO files to CDROM – "This media type is incorrect. Media type ISO was expected." (known issue)

Catalogs has had quiet a big revamp in vCloud 5.5 with some enhancements.
This can be read in the “whats’s new” pdf:
http://www.vmware.com/files/pdf/products/vCloud/Whats-New-VMware-vCloud-Director-55-Technical-Whitepaper.pdf

We have a public catalog which is shared to all ORGs within vCloud.  This also contains some ISO media files for users to mount to cdrom of virtual machines.
However in order for a user to make use of a media file from a public catalog, that media file firstly needs to be copy to the local catalog of that ORG.

Problem:

A users copies the media ISO file to local catalog however the file is not copied over correctly.
When users tries to attach the newly copied file from media to CDROM, the following error is observed:
“This media type is incorrect. Media type ISO was expected.”
Troubleshooting:

The original media file for public catalog I am able to mount to CDROM on all VM’s within that catalog.
Once that same media file is copied to local catalog I experienced the problem.
When I uploaded the same media ISO file directly to users catalog I was able to mount the file.
Solution:
No solution as of yet but workaround is to update the tables directly in database.
I just went ahead and uploaded the needed ISO files directly to the local ORGs catalog media files.
I was also able to locate the knowledge base article for the issue mounting ISOs after copying them between catalogs. This article includes the required process on the Database side to correct the ISO/Media bits for the catalog items.

vCloud director 5.5: Catalogs problem #1 – browser crashes on "copy to catalog" (resolved)

Catalogs has had quiet a big revamp in vCloud 5.5 with some enhancements.
This can be read in the “whats’s new” pdf:
http://www.vmware.com/files/pdf/products/vCloud/Whats-New-VMware-vCloud-Director-55-Technical-Whitepaper.pdf

We have a public catalog which is shared to all ORGs within vCloud.  This also contains some ISO media files for users to mount to cdrom of virtual machines.
However in order for a user to make use of a media file from a public catalog, that media file firstly needs to be copy to the local catalog of that ORG.

Problem:

When a user log in directly to their respective ORG
Select the public catalog and go to media tab
Right click on media file select to “copy to catalog”
The browser crashes and restarts without any error messages.

Troubleshoot:

This was tested on MAC and Windows with each of the following browsers including Chrome, Firefox and Internet explorer.
Same problem appears when logged in as ORG or system admin.

However it works fine when I login as system admin to main vCloud Director login page.  Only seems to crash browser when logged in direclty to any ORG site.

Solution:

Do not share the catalog with the radio button “all organizations”, but rather select all the ORGs individually from the list.

I do not experience the same results as listed the following KB but same solution solves both problems:
http://kb.vmware.com/kb/2063431

VMware 5.5 error – Invalid datastore path ”. – When creating new Virtual Machine

A new problem I found in VMware 5.5 is when you try to create a new Virtual Machine.

Debugging the problem:

I have never had any issues with creating new Virtual machines from scratch.  However i had to create a new OS template and started the process but received error:  Invalid datastore path.

Looking online I did seem to find this error popping up alot but not related really to my problem.
I did test the following:
•Verify that datastore mentioned in the error is connected and accessible.
•Verify that the file exists in the datastore.

•Ensure that the file name does not contain non-Basic Latin characters.

Kind of figure the problem storage or permission from error message so  I went through my custom VM steps and under storage selection I always choose the Storage cluster and keep storage DRS enable so VMware make the recommendation of where to place the disks.  This is where the problem lies for some reason….



Resolution:

When you create the VM and under storage selection if you choose the storage cluster, make sure you disable Storage DRS for the VM and select the datastore manually.

This seems to have fix my issues and I was able to create the VM successfully.


Migration to host failed with error Already Disconnected (195887150)

This is the first time i ever ran into this problem after installing a new ESXi host.  Added the host to our current vDS, moved into cluster and disabled maintenance mode.
Thought might be related to our newly upgrade 5.5, however seems a lot of people ran into same problem.

ERROR:

Migration to host failed with error Already disconnected (195887150).
vMotion migration [169942327:1389120124781890] failed writing stream completion: Already disconnected
vMotion migration [169942327:1389120124781890] failed to flush stream buffer: Already disconnected

vMotion migration [169942327:1389120124781890] socket connected returned: Already disconnected

VMware KB found on problem:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1030267

I tried all the regular fixes:

  • vmkping to and from the vmkernel port used for vmotion
  • verify DNS
  • restart the management agent on host given error.
    • /etc/init.d/hostd restart
    • /etc/init.d/vpxa restart
  • Verified time on the both hosts
  • Turned off Enable Migration in the Advanced Settings of the host and re-enabled it.
    • vSphere web client -> Manage tab -> Settings tab -> Advanced System Settings
    • Search for “migration”
    • Find Migrate.Enabled and set to 0(disabled)
  • Unchecked vmotion on the vmkernel port and re-enabled it.
Still this did not resolve my problem.
RESOLUTION:
Change the IP address of the MVkernel adapter for vMotion traffic.  This seems to fix the problem.  I then  also changed the IP back to original and problem was still resolved.
The vMotion VMkernel adpater IP address was previously used but not on this server since new server.

vCloud Director 5.1.2 – Bug – Retain IP/MAC resources does not apply when you use the “Move to” task to move to new Storage profile.

In previous blog post I mentioned the usefulness of this setting, but during our storage migration to new profile for vCloud director we ran into a bug where this is not applied.
I have opened a case with VMware and they verified this as a bug and now has a SR.  Hopefully get this fixed within the next build.
My current vcloud director version where this applies:
vCloud director 5.1.2.1068441
Debugging the problem:

When you have to move vAPPs to new storage profile the easiest way is  to shut down the vAPP and select “move to”.

However when you perform this task the vAPP will actually release the Org VDC NAT’d address for the VM.
If you have any NAT’s configured on the Edge gateway, this will now be out of sync.

Workaround:
I will discuss this in next blog post.