Month: May 2016

vCSA & PSC: Update/Patching options available

/ johannstander / Leave a comment
The update of either vCSA or PSC can be achieved through the VAMI interface which was introduced back in 6.0U1 or through the appliancesh command-line interface.
Method 1: VAMI and URL: 
This method requires internet access from your appliances.
  • Login to VAMI
  • https://vcenterserver.domain.com:5480/
    • U: root / P: password
  • From navigator select Update
  • This will display the current version details
  • Select Check Updates -> Check URL
  • This method will go out to VMware’s repository https://vapp-updates.vmware.com/vai-catalog/valm/… and verify you are on latest.
  • If available updates then select the Install updates -> Install all updates
  • Accept EULA
  • Wait for updates to complete.

Method 2: VAMI and custom URL: 
This method can be used if you do not have internet access from your appliances by setting up a local repository.
  • Setup a webserver on your network (ISS or Apache) that will be accessible to the vCSA and PSC.
  • Create a directory called PSC_updates or VCSA updates.  Names can be changed.
  • On VMware support site now download update but make sure to download the zip update bundle.
  • Extract the zip update bundle to the folder you created earlier.
  • Login to VAMI
  • https://vcenterserver.domain.com:5480
    • U: root / P: password
  • From navigator select Settings
  • Select “Use specified repository”
  • Click OK
  • Select Check for updates
  • If available updates then select the Install updates -> Install all update
  • Accept EULA
  • Wait for updates to complete
Method 3: VAMI and CDROM:

This method is pretty straight forward.

  • On VMware support site download the ISO for latest vCSA and/or PSC.
  • Login to vCenter Web client
  • Select vCSA or PSC appliance VM
  • Launch remote console
  • Select VMRC -> Removable devices -> CD/DVD drive 1 -> Connect to Disk Image File (iso)
  • Mount the ISO downloaded from VMware support site
  • Login to VAMI
  • https://vcenterserver.domain.com:5480/
    • U: root / P: password
  • From navigator select Update
  • Select Check Updates
  • Select Check CDROM
  • If available updates then select the Install updates -> Install all update
  • Accept EULA
  • Wait for updates to complete
Method 4: Using appliancesh command line

This method was the only way to update the appliance when vCenter 6 was released since the VAMI was only introduced in 6.0U1.  You can either mount and ISO or point to a URL for updates.  The URL can be the default VMware or 
  • On VMware support site download the ISO for latest vCSA and/or PSC.
  • Select vCSA or PSC appliance VM
  • Launch remote console
  • Select VMRC -> Removable devices -> CD/DVD drive 1 -> Connect to Disk Image File (iso)
  • Mount the ISO downloaded from VMware support site
  • SSH to VCSA or PSC
  • type # appliancesh
  • enter root password
  • To use CDROM
    • type # software-packages install –iso –acceptEulas
  • To use default VMware URL 
    • type # software-packages –url (This will use the default vmware URL to check)
  • to use local ropository URL

Product patches can be downloaded from this VMware site:
https://my.vmware.com/group/vmware/patch#search
Log files to review for updates:

/var/log/vmware/applmgmt/software-packages.log




vRealize Log Insight: Configuring agents

/ johannstander / Leave a comment

The vSphere content pack provides powerful insight into your vSphere logs, allowing you to make informed and proactive decisions within your environment.  For the exercise I am just reviewing some of the VMware products and providing notes I took during installation. Sorry if they seems a bit all over the place 🙂

Log Insight agent now gets pre-installed on some of the appliances which is great and means no need to install agents manually.  Some of the VMware products that has agent pre-installed:
vRealize Business
vRealize Operations Manager (beginning from 6.1)
vRealize Orchestrator (beginning from 7.0.1)
vRealize Automation (beginning from 7.0.1)

vRealize Log Insight

Here are some basic functions which will help a lot for instructions on content packs:

Install Content Packs:

  1. Login to vRealize Log insight.
  2. Select the stack menu button in top right hand corner
  3. Select Content Packs
  4. Installation has been simplified a lot since you do not have to go to VMware solution exchange anymore to download and manually install the content packs, it is available straight from Marketplace window.  Super awesome!
  5. Just click on Install for which ever content pack you want to install.

How to view setup instructions?

  1. Select the stack menu button in top right hand corner
  2. Select Content Packs 
  3. Select Installed content pack
  4. Click the cog wheel -> Setup instructions

To verify if agent configuration from Log Insight was pushed successful to server:

Check the affective file to see if the correct agent configuration file logs has been pushed to the liagentd.

Linux:

\etc\liagent-affective

Windows:

C:\ProgramDATA\Vmware\Log insight agent\liagent-affective

View the agent configuration settings:

 

  1. Login to vRealize Log insight.
  2. Select the stack menu button in top right hand corner
  3. Select Content Packs 
  4. Select Installed content pack
  5. Select Agent Groups tab
  6. Find group name and review the Notes and Configuration

Agent Groups

Agent Groups comes as part of the content packs you installed. This is required for dashboard to work correctly. If you use syslog-ng, you will still receive the events but the vSphere content pack dashboards will not work.

  • I would always recommend making a copy of the original
  • Provide a new name
  • Save it
  • Provide a filtered list of hosts which could be by name, IP address or wildcards.  These hosts should already have been already registered to Log Insight via their Agent configuration.
  • Save the Agent Group.

The configuration is automatically pushed out to the selected hosts and log messages will begin flowing in.

Install agents on linux:

This is of course not part of VMware products but providing the steps to manually install the agent on a linux box which you still need to do sometimes.

http://pubs.vmware.com/log-insight-30/index.jsp#com.vmware.log-insight.agent.admin.doc/GUID-83976956-C16C-42BD-9950-C6EDDF983086.html

  1. Make sure the hostname is set under /etc/hosts, /etc/HOSTNAMES, hostname   (otherwise server will show up with localhost hostname)
  2. Copy the bin file to appliance (this is SUSE so have to copy the bin)
  3. Chmod +x .bin
  4. ./.bin
  5. Vi /etc/liagent.ini
  6. http://pubs.vmware.com/log-insight-30/index.jsp?topic=%2Fcom.vmware.log-insight.agent.admin.doc%2FGUID-D245F706-BC99-46D0-87E3-584D9D250529.html
  7. (/etc/init.d/liagentd status/stop/restart)

 

In order to download the agent from server and install the agent I use following commands:

# curl -o /tmp/liagent-current.rpm http://LOGINSIGHT-SERVER:9000/api/v1/agent/packages/types/rpm ; rpm -Uvh /tmp/liagent-current.rpm

NSX:

NSX Manager

Sends all audit logs and system events from NSX Manager to the syslog server.

Steps

  1. Log in to the NSX Manager virtual appliance.
  2. Under Appliance Management, click Manage Appliance Settings.
  3. From the Settings panel, click General.
  4. Click Edit next to Syslog Server.
  5. Type the IP address of the syslog server.
  6. Required Type the port and protocol for the syslog server.  If you do not specify a port, the default UDP port for the IP address/host name of the syslog server is used.
  7. Click OK.

 

NSX Edge

NSX Edge events and logs related to firewall events that flow from NSX Edge appliances are sent to the syslog servers.

Steps

  1. Log in to the vSphere Web Client.
  2. Click Networking & Security and then click NSX Edges.
  3. Double-click a NSX Edge.
  4. Click the Manage tab and then click the Settings tab.
  5. In the Details panel, click Change next to Syslog servers.
  6. Type the IP address of both remote syslog servers and select the protocol.
  7. Click OK to save the configuration.

NSX Controllers:

The only supported method on configuring the syslog server on the NSX controllers is through the NSX API which is described in the KB below:

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2092228

https://jreypo.wordpress.com/2015/09/30/how-to-configure-a-remote-syslog-server-for-nsx-controller/

I did however found another way to perform this through SSH but use at own risk and I still recommend using the NSX API!

  1. SSH into NSX controller:
  2. Change Controller cluster password
  3. vCenter server -> Networking & security -> Installation -> Management
  4. NSX controller select
  5. Actions -> Change controller cluster password
  6. 12 character min
  7. Login
  8. # show syslog-exporters
  9. add syslog-exporter
    1. This will add a syslog exporter
  10. add syslog-exporter-facility
    1. This will add a facility to a syslog exporter

Example:  # add syslog-exporter nsx-controller-syslog INFO kern,user,mail,deamon,auth,syslog,lpr,news,uucp,cron,security,ftp,ntp,logaudit,logalert,clock,local0,local1,local2,local3,local4,local5,local6,local7,api,api_request,api_request_content,api_request_header,logical_net,system,transport_net

  • 514 UDP

 

vRA 7:

  1. Install content packs:
    1. Vra7
    2. Vrealize orchestrator
    3. Apache
  2. Download windows agents from administration -> Management -> Agents -> Right at bottom of screen!
  3. Install agents on windows servers  (management, DEM, Web)
  4. From drop-down agents select vRealize 7 – Windows and create filter for only the windows server for instance hostname = wdvra*.domain.com

Update:

Vra-dem, vra-dem-metrics, vra-deo, vra-deo2

Under agent configuration update the paths where necessary like for instance vra-deo where the directory is normally -DEO after Distributed Execution Manager folder  “C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\DEO\Logs\”  BUT SHOULD BE C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\FQDN-DEO\Logs\

If you have multiple DEM servers or management servers then create  another file log called vra-dem2 to add the 2nd server file location.

For vRA appliances:

  1. Just update the \etc\liagent.ini with the hostname for vrealizeloginsight server.
  2. Restart service \etc\init.d\liagentd restart

vRealize Orchestrator:

Some good information from VMware blog on Orchestrator for vRealize Log insight.

http://blogs.vmware.com/management/2016/04/vrealize-orchestrator-7-0-content-pack-log-insight.html

 

  1. Login to vrealize orchestrator control center.
  2. https://10.10.30.133:8283/vco-controlcenter/
  3. Select Log -> Logging Integration
  4. Check box for “Enable logging to a remote log server”
  5. Currently only Log4j is supported but upcoming release after 7.0.1 should support Log Insight Agent
  6. Enter Host, Port and protocol.
  7. Test Connection
  8. Save

 

Problems experienced:

This did not work and got an error “HTTP Status 500 – Failed to edit Log Insight Agent configuration file!”

I create another blog to show how to fix this problem:

http://virtualrealization.blogspot.com/2016/05/vrealize-orchestrator-control-center.html

Agents group template does not show up and had to uninstall and reinstall the agent.

 

vRealize Orchestrator control center : HTTP Status 500 Failed to edit Log insight configuration file

/ johannstander / Leave a comment

With latest vRealize Orchestrator 7.0.1 I was configuring syslog logging integration in control center, to send logs to vRealize Log insight, but ran into error “HTTP Status 500 Failed to edit Log insight configuration file”.

Troubleshooting:

Testing on a fresh install and did no run into the problem so came to the conclusion that this error only appears when you upgrade from 7.0 to 7.0.1

SSH into Orchestrator appliance and reviewed the logs.
/etc/var/log/messages

2016-04-27T17:19:32.013813+00:00 ldvro01 sudo:      vco : a password is required ; TTY=unknown ; PWD=/var/lib/vco/configuration/bin ; USER=root ; COMMAND=/var/lib/vco/app-server                          /../configuration/bin/config_liagent.sh /var/lib/vco/configuration/temp/liagent.tmp /var/lib/loginsight-agent/liagent.ini
2016-04-27T17:20:10.075308+00:00 ldvro01 sshd[20887]: rexec line 79: Unsupported option KerberosAuthentication
2016-04-27T17:20:10.075376+00:00 ldvro01 sshd[20887]: rexec line 85: Unsupported option GSSAPIAuthentication
Found the script that gets executed to be /var/lib/vco/configuration/bin/config_liagent.sh which actually resides on /usr/lib/vco/configuration/bin/config_liagent.sh
Listing the folder shows that vco:vco has rwx permission.
:/usr/lib/vco/configuration/bin # ls -ll
-rwx—— 1 vco vco  218 Feb 19 15:09 config_liagent.sh
-rwx—— 1 vco vco  230 Feb 19 15:09 controlcenter.sh
-rw-r–r– 1 vco vco 6718 Feb 19 15:09 log4j.dtd
-rw-r–r– 1 vco vco 3315 Feb 19 15:09 propagate.sh
-rwx—— 1 vco vco 1321 Feb 19 15:09 setenv.sh
A password is required is throw in the error message which leads me to think the vco user does not have the necessary permissions when trying to execute the command.
Looking in /etc/sudoers file and found the vco missing the path to the config_liagent.sh file.
Resolution:
Add the path to config_liagent.sh for vco user.
# visudo
scroll to bottom of file.
you will see the following:
vco     ALL=(root) NOPASSWD: /etc/init.d/vco-server, /etc/init.d/vco-configurator
update the line as follows:
vco     ALL=(root) NOPASSWD: /etc/init.d/vco-server, /etc/init.d/vco-configurator, /var/lib/vco/configuration/bin/config_liagent.sh

EMC UnityVSA with SRM configuration

/ johannstander / Leave a comment

I am not going to get into the details of setting up SRM and ECM Unity this is very well documented so the information I will provide is after SRM is installed and configured on vCenter and EMC Unity is installed and configured.

Previous blog post shows UnityVSA setup:
https://virtualrealization.blogspot.com/2016/05/how-to-emc-unityvsa-installation-and.html

EMC UnityVSA:

I already have my pools and LUN’s configured on both Unity virtual storage appliances.
Firstly we want to setup an interface for replication on both Unity VSA’s.
In Unisphere select Data protection -> Replication
Select Interfaces
Click + sign

Select Ethernet Port and provide IP address information.

click OK

Now lets configure the remote connections between Unity arrays.
In Unisphere select Data protection -> Replication
Select Connections
Click + sign

Enter Replication connection information for your remote Unity VSA.
Asynchronous is the only supported method for the Unity VSA.

Click OK.
Select the remote system and click “Verify and Update” to make sure everything is working correctly.

Now lets go ahead and setup the Consistency groups.
In Unisphere select Storage -> Block
Select Consistency Groups
Click + sign

Provide name

Configure your LUN’s.  You have to create a minimum on 1 LUN but you can later add your existing LUN’s to this consistency group if that is required.

Click + to Configure access

Add initiators

Create Snapshot schedule

Specify replication mode and RPO

Specify destination

Click Finish

Now that we have replication configured we can go to vCenter and configure SRM.

SRM:
I already have my EMC Unity Block SRA installed on my SRM server. My mappings is also configured within each site so we will skip this.

Open vCenter server and select Site recovery.
Select each site -> Monitor -> SRA’s
Select rescan all SRA’s
Verify that EMC Unity Block SRA is available.

Let’s configure Array Base Replication.
Select Site recovery
Select Inventories -> Double click Array Base Replication
Select “Add array manager”
On popup wizard select “Add a pair or array managers”

Select location

Select Storage replication adapter, EMC Unity Block SRA

Configure Array manager

Configure array manager pair for secondary site.

Enable the pairs

Click Finish

Verify Status is OK

Click on each storage array and verify no errors and that you can see the local devices being replicated.

Now we can setup the protection group
Select Site recovery
Select Inventories -> Protection Groups
Select “Create Protection group”
Enter name

Select protection group direction and type. For this we will select array base replication with datastore groups.

Select datastore groups

This will provide information on the VM’s which will be protected.

Click Finish
Verify protection status is OK

Finally you can configured your Replication plan:
Select Site recovery
Select Inventories -> Recovery Plans
Select “Create Recovery plan”
Enter name


Select recovery site
Select protection group

Select network to be used for running tests of the plan.
Click Finish

You can now test your recovery plan.

EMC UnityVSA : adding LDAP authentication

/ johannstander / Leave a comment

Here are the steps to setup LDAP authentication for EMC UnityVSA.

Login to Unisphere.
Select the cog in top right hand corner to open settings.
Select Users and Groups -> Directory Services
Enter LDAP server information

Click Apply
Click Verify Connection
If successful, Select Advanced
This is recommended since otherwise you will run into problems with assigned users and groups since a default user and group search path created for domain is “cn=Users,dc=domain,dc=com” which in most cases will not fit in your company’s AD structure.

Click Ok
Click Apply
Select Users and Groups -> User management
Click + Sign

Select User or Group type
Select LDAP User

Enter username.  Wish validation could have taken place for LDAP user or group at this point and not after entering all information.
Enter role
Click Finish

EMC UnityVSA installation and configuration

/ johannstander / Leave a comment

I am currently testing SRM and installed Nimble as my virtual storage array with Nimble SRA 3.0 but having to many problems with getting the array pairs working correct so decided to setup UnityVSA community addition which is available for free with up to 4TB of data.  At the bottom of the page I provided some useful links:

Installation:
first off lets review requirements:

  • vCenter 5.5 update and later.
  • ESXi 5.x and later
  • 12GB Memory
  • 2 vCPU
Deploy the OVA downloaded “UnityVSA-4.0.0.7329527.ova”.
I am not going to provide the steps to deploy and OVA since this is pretty straight forward and nothing really to configure except for management and data ports and management IP address.
After deployment is completed and VM powered on, open a browser and point to IP address specified during OVF deployment.
You will be presented with a login screen.
Type admin / Password123#
Wizard will appear for initial configuration.
Specify password to replace the default.
You need to request the license file by providing the System UUID to the following website: 
Download the license file and install it.
Enter DNS information
Enter NTP information
Pools can be configured here but you do you require a manually created VM disk.  If you have not added the new disk within vCenter for the VM then I would recommend just skipping this step for now.
Enter SMTP information
Create iSCSI network interface.  
This can also be performed later but I created this on the data network ports i specified during the OVF deployment.
Creating NAS server but this can be done at later time.
Initial setup is now completed, yay!

Setup Pool:
Next step is to setup your storage to used by the UnityVSA.  
This is very easily accomplished through vCenter server. 
Edit settings on the UnityVSA VM
Select new device “New Hard disk” and click Add
Create hard disk with following recommended settings:
  • SCSI controller 0 which is VMware Paravirtual
  • Thick provision eager zero
  • Max size of 4TB
  • Min size of 10GB
  • Connect up to maximum of 12 disks for user data
Wait 60 seconds for UnityVSA to recognize the new storage.
Now we can setup our Pool
Select Storage -> Pools
Select + to create new pool
Select the newly create disk, and make sure to select the storage tier.  After select the Storage tier, press either enter or anywhere else on screen to make the Next button available.
check box for storage tier
Select virtual disks
Create Capability profile.  This is a set of storage capabilities for VVol datastore.  The capabilities are derived from the underlying pools so best practices is to configure it during pool creation.  Capabilities needs to be created before you can create a VVol datastore.
Specify Tag.  Usage tags can be applied to capability profiles to designate them and their associated VVol datastores for a particular use
Setup initiators:
Select Access -> VMware
You have some options here to either directly connect and configure the ESXi hosts or connect directly to vCenter server and select which ever ESXi hosts within the environment you want to setup initiator access for.  I selected the latter since easier to connect to vCenter.
Enter vCenter information
Select the ESXi hosts.
Click Finish
To verify the added ESXi hosts you can select Access -> Initiators.  Here you can review your which will import both FC and iSCSI protocols if configured on hosts.
Setup LUN:
Enter name
Select the Pool previous created and size of LUN
Click + to Select initiators for access
Create snapshot schedule  (this is a very welcome addition since was lacking in VNX)
Setup replication.  I will be adding another blog shortly to setup replication between two UnityVSA and using VMware SRM.
Finish

Hopefully get some time here shortly to work on setting up SRM with Unity so stay tuned.

Links:

vCenter Server SMTP authentication not supported – how to guide on getting alerts

/ johannstander / Leave a comment

I recently updated a customer from 5.1 to 6.0 and a couple of days later received a question on how to setup a mail server with SMTP authentication.

This of course is not possible as described in the following KB 2063147
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2063147

This got me interested to setup a solution that would provide the functionality to allow outgoing email through SMTP relay service in stead of setting up a full fledged local email server.
Here are the steps I took to setup postfix on CentOS to relay outgoing email to 3de party which requires authentication.

  • Install and configured a linux operating systems.
  • Now we need to install and update the packages required for our configuration which includes postfix as well as cyrus-sasl-plain which is not installed by default on CentOS 6+
    • sudo yum install postfix cyrus-sasl cyrus-sasl-plain
  • To make postfix the default MTA in our system lets remove sendmail
    • sudo yum remove sendmail
  • Postfix setup:
    • vi /etc/postfix/main.cf
    • Configure server FQDN:
      • mydomain =
      • myhostname =
    • Configure relayhost to email provide smtp server.  Verify the port since might not be default 25 to prevent spamming.
      • relayhost =
      • relaydomain =
    • Configure cyrus-sasl-plain:
      • smtpd_sasl_auth_enable = yes
      • smtpd_sasl_path = smtpd
      • stmpd_sasl_password_maps = hash:/etc/postfix/sasl_passwd
      • smtpd_sasl_type = cyrus
      • smtp_sasl_auth_enable = yes
    • Configure receive mail so that communication can be established from all networks.  If you select inet_interfaces = localhost then can only send from local server.
      • inet_interfaces = all
      • inet_protocols = all
    • Configure additional trust and relay control
      • mynetworks_style = subnet  (if you want to specify specific network subnets)
      • mynetworks_style = host  (if you want to specify specific host names)
      • mynetworks = 127.0.0.0/8, 192.168.1.0/24
  • Now since our SMTP server requires authentication we need to setup username and password.
    • vi /etc/postfix/sasl_passwd
      • yourisp.smtp.com:2525 username:password
      • the servername should match exactly what you have entered for relayhost in /etc/postfix/main.cf
  • Generate a postfix lookup table
    • postmap hash:/etc/postfix/sasl_passwd
  • Test lookup table which should return username and password
    • postmap -q yourisp.smtp.com:2525 /etc/postfix/sasl_passwd
  • Verify sasl_passwd and sasl_passwd.db files are read/write enabled for root only to protect the plain text password.
    • chmod 600 /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db
  • Add postfix to be started at boot
    • chkconfig –add postfix
  • start service
    • /etc/init.d/postfix start
  • Send test email.
    • # sendmail -t
    • TO: addressto@test.com
    • From: addressfrom@test.com
    • Subject: Test
    • Did you get this email?
    • .
Troubleshooting:
If you check the status of service and get error:  “Master is Dead ButPid File Exists”, verify that you have removed sendmail successfully.
Connection refused when trying to send from vCenter, verify that port 25 is listening on host with # netstat -nlp | grep 25.  If it shows with 127.0.0.0/8 then it will only allow local connection.  This needs to show 0.0.0.0:25 so make sure you have inet_interfaces = all.
Some useful links: