VMC on AWS: What you need to know to get started

First off, VMC on AWS is fantastic.

With pretty much “the swipe of your credit card,” you can get a login and start building your SDDC and be up and be running in a matter of hours.

It has some great use-cases that I think customers should really consider, especially if you have a cloud initiative, disaster recovery requirements or looking at building out new data centers. Here are some examples of those different use-cases:

  • Datacenter Consolidation
  • Datacenter Evacuation
  • Datacenter Expansion
  • Disaster Recovery (DRaaS with vSphere Replication and SRM)
  • Cyclical/Burst Capacity
  • Development and Testing

I do not want to go into crazy detail here since there are already a lot of guys in the community, who are much smarter than I am, who you should follow if you want to get in-depth knowledge of VMC on AWS. To name only a few:

Frank Denneman

Brian Graf

Emad Younis


What I do want to chat about it is how you get started and what do you need to think about from a requirement and architecture perspective during the initial setup.

Let’s start off with due diligence.

As mentioned there are a lot of good use cases for VMC on AWS but what do you need to consider to make the decision if this is the right solution for you. Here are a few things that I can think of, and please provide comments if you can think of any additional ones I can add.

  • How large is your team and can they handle the management and maintenance of the hardware and software of an additional data center?
    • VMC takes care of all of this
  • If moving to a Cloud Provider, does your team have the necessary skills to manage the workload and consume the services?
    • VMC requires no additional skill sets
  • Do you require the use of AWS services in your environment?
    • With VMC on AWS, you can deploy or migrate your VMs to the SDDC and consume the services without any egress costs. AWS service cost still applies.
  • How much would it cost to run your current Virtual Machine workload in VMC on AWS or a different Cloud Provider?
    • vRealize Business for Cloud provides you with cost comparisons between different Cloud Providers
    • vRBC also provides a VMC Assessment which provides a comparative analysis of utilization based cost on Private cloud vs. VMware Cloud on AWS.  Sweet!
  • How do I take care of billing?
    • SDDC consumption will be billed through VMware. Fixed monthly costs which are great because it can be a roll of the dice using a Service Provider.
    • Your egress charges from SDDC Edge Gateways will be billed to VMware because the VPC is actually owned and managed by VMware as part of the service.
    • Any AWS related services will be billed to your AWS account.
  • If you make a decision on using a colo, then you have the following cost considerations:
    • Rack or Floor space / Power consumption
    • Compute
    • Storage
    • Licensing
    • Networking
    • ISP
    • Dark Fiber
    • Cabling
  • If you make a decision on using your own physical datacenter, then you have these cost considerations:
    • Compute
    • Storage
    • Licensing
    • Networking
    • ISP
    • Dark Fiber
    • Electricity
    • HVAC/Cooling
    • UPS Systems
    • Cabling
    • N+1 redundancy on physical infrastructure
    • This list can go on and on and on…

Now, let’s say you performed your due diligence and make the smart decision to purchase VMC on AWS. What’s next?

To sign up, please contact your partner or VMware account team.

After you receive your login for console access, you can start the onboarding process, which has some requirements and upfront considerations.

  • AWS requirements and decisions
    • You require an existing account on Amazon Web services, and if you do not have one, you can connect to a new AWS account.
    • The AWS account will be linked to your Cloud organization and as mentioned earlier and any AWS services consumed will be billed through this account.
    • The default VPC assigned subnet might conflict with your internal networks and create a problem when the VPC is linked to your SDDC. Please review your existing subnet and if necessary create a new non-conflicting subnet.
    • Make sure your subnet is in the same region as your planned SDDC
    • Subnet should have at least 64 IP addresses (/26) in each AZ in your VPC.
  • How many hosts do you need?
    • This will depend on your use case and workload requirements.
    • When you create the SDDC, you have the option to set your number of hosts, with a minimum of 4, which has a total capacity of 8 Sockets, 144 Cores, 2 TB RAM, 42.8 TB Storage.
    • Also, think about future growth and the maximum amount of hosts that you might deploy.
  • Cost considerations for your subscription
    • Select the number of hosts you want as part of the subscription and for what period (on-demand, 1 year, 3 years).  Longer-term reservation of hosts gives you up to 50% cost saving!
    • Through the Hybrid Loyalty Program discount, you get an additional discount of up to 25% based on the number of eligible on-premises product licenses you own.
  • You have to make a decision on the Management IP address schemes that you want to use within your SDDC.
    • During the creation of your SDDC, you can configure the management subnet so make sure to pick a range that will not conflict with your existing on-premise networks as well as the AWS subnet that will connect to your SDDC.
    • Depending on the maximum future amount of hosts that you might deploy, you have to select the appropriate range.   13 hosts = /23, 125 hosts = /20, 160 hosts = /16.
    • Once the management network is created, it cannot be changed, and you would have to destroy your SDDC and start from scratch. Make an informed decision!
  • In which AWS region should your SDDC run in.
    • This will depend on your use case and current location.
    • At the time of writing US East (N. Virginia), US West (Oregon) and London (England) is available.
  • After the SDDC is created, you also have to make a decision on how you want to connect to your Management and Compute SDDC networks externally through the Edge Gateway. The following options are available.
    • IPsec VPN (L3)
      • This is available for both your management and compute network
    • L2VPN
      • This is only available for your compute network, and provides a secure communications tunnel between an on-premises network and one in your cloud SDDC.
    • Direct Connect
      • Service provided by AWS that allows you to create a high-speed, low latency connection between your on-premises data center and AWS services with no additional egress charges.
      • Either select a colo at AWS Direct Connect locations or will need APN Partners to establish network circuits between the AWS Direct Connect location and on-premises environment.
  • What ports do you require to be open on your Management and Compute network Firewall?
    • By default, your gateways are configured with a deny all policy.

The onboarding process is as follow:

  • Setup VMware Cloud Organization
    • Create an account by signing up
    • Log in to your Cloud console
    • Click on Invite users
    • Add users and pick their Organization role.
    • Set your theme, I like mine dark.
    • If you are a member of more than one Organization, set your default.
  • Create a subscription
    • This allows you to save money by committing to buy a certain amount of capacity for a defined period of time.
    • Choose wisely
  • Create your SDDC
    • Link your AWS account and choose your VPC and Subnet.
    • Choose the appropriate AWS region
    • Enter a name for your new SDDC
    • Select the number of hosts
    • Set a private subnet range for your management subnet
      • This will be used for vCenter Server, NSX Manager, and ESXi hosts.
  • Configure access to your Management cluster
    • This is required for access to the vCenter Server server as well as ESXi vMotion VMKernel network, for when you want to migrate your VMs into your VMC SDDC.
    • Configure the IPsec VPN
      • You need to decide if you want to use your existing physical networking gear for connectivity, or if you have NSX implemented you can use an Edge GW, or you can install a standalone Edge GW for free.
    • Routing
      • Don’t forget routing, your networking team would need to route your on-prem management network to this new network.
    • Firewall rules
      • Decide on what ports need to be opened for network connectivity to vCenter Server, NSX, ESXi remote console, vMotion, etc
    • DNS
      • Set DNS IP addresses to resolve computer names.
  • Configure access to your Compute cluster
    • Configure your logical networks
      • Need to decide what type of network you need, routed (default) or extended
      • Decide on an IP range and size of the subnet.
      • Routed network use the SDDC compute gateway as the default gateway, which means it has connectivity to other logical networks in the same SDDC, as well as to external networks like the Firewall and NAT.
      • Extended networks use an on-premises gateway as the default gateway and require L2VPN connectivity to be configured.
      • As a note, logical networks can be changed from routed to extended and extended to routed.
    • Routing
      • If you select a routed logical network, you need to decide which of your local compute networks you want to route for connectivity.
    • Firewall rules
      • Decide on what ports need to be opened for network connectivity to
      • This will probably require a bit more brain power because it will include all your workload application and however you decide where your workload will run.  Knowing your mapped application dependencies is crucial and vRNI, as well as the Service discovery management pack for vROPS can help in understanding how your applications talk to each other and provide the necessary firewall rules that need to be created.
    • NAT
      • Do you need to connect public IP addresses directly to deployed VMs?
    • Public IP address
      • You need to decide if you require public IP address for maybe web front-end applications running in your SDDC. If so you need to request them and beware there are costs involved.


Configure Hybrid linked mode

  • Do you want to see both your onsite and VMC vCenter Server through a single pane of glass?  This is a one-way connection so you always need to connect to your VMC on AWS vCenter Server to see both!
  • Definitely worth it and provides that live Cross-vCenter vMotion awesomeness.


  • DRaaS
    • If your use-case is to protect your on-prem workload and recover them in your SDDC, you can activate Site Recovery.
    • The service consists of Site Recovery Manager (SRM) and vSphere Replication
  • HCX
    • This add-on SaaS offering provides large-scale migration between your on-premises environments running vSphere 5.0+ and VMC on AWS with no re-platforming, retesting, or change in tooling. Cool!
    • It also provides high-performance Layer 2 extensions, data synchronization, traffic analysis, WAN optimizations, and built-in IPsec VPN connectivity that will enable secure and efficient cloud migration with no impact on application uptime.


Useful links:

vExpert 2018 achievement

So this just happened!

After years of sharing the good and sometimes bad experiences in my profession, I finally decided to apply for the vExpert award.  When I started, I never even knew of this community and only blogged to improve my English writing skills (still a work in progress), as well share some of my knowledge that I have all written down in my MS OneNote.

2 years ago I started noticing the vExpert badge worn so proudly and I started researching on what this is and what it stands for.  I don’t really care much for the recognition, but what did peak my interest, is the community and exclusive opportunities that this program provides.

So here we are today and after my first application, I was accepted into the community and cannot be more happy and honored.

To everyone in the vExpert community, we appreciate the time and effort you put in day in and day out, sharing your knowledge and making our lives easier.  Keep up the good work.


VMware EMPOWER 2018: Technical Partner Event of the Year

VMware has finally decided to right their wrongs and move Partner Exchange back out of VMworld and into its own conference. This is great news for all partners and looking forward to some great technical deep dive sessions into VMware productions and solutions.

The Global Conference takes place this year in my hometown of Atlanta, GA from April 16 – 19, 2018.  Let me know if want to meet up.

Content catalog available here

Register here

FAQ here

VMware Cloud on AWS: New features

VMware delivered its first ever Cloud Briefing today. It was kind of weird that the event was pre-recorded and not live since their marketing emails made it sound like it would be live.  Nothing to fuss about but just worth mentioning.

This is already the 4th update to VMC on AWS within the last 6 months, so what is new?

  • New UK based service offering in London.
  • Introduction to upcoming Germany based service offering in Frankfurt.
  • Preview support for stretched clusters across AWS Availability Zones within AWS regions.
    • Synchronous replication
    • Zero RPO high availability across AZs
    • During failure vSphere, HA will restart the VMs on surviving AZ
  • Multi-Cluster support
    • Add additional clusters to your SDDC
    • Maximum of 10/SDDC
  • vSAN Compression and deduplication
  • RESTful API now available.
  • vROPS support for VMC on AWS
    • Predictive DRS not yet supported
    • Service delivery management not yet supported
  • VMware Cloud services updates:
    • HCX Enterprise
      • On-premise offering and supports for any-to-any vSphere migration.
    • HCX for Clouds
      • Migrated workload between different Clouds.
    • Cost Insight
      • Now supports VMC on AWS
      • Calculates VMC on AWS capacity required to migrate from on-premises to VMC
      • Integration with Network insight to calculate networking costs during migration as well.
    • Wavefront
      • Now supports VMC on AWS
      • Monitoring and analytics to optimize cloud-native and enterprise applications
      • Wavefront for vRealize Operations
        • Complete visibility into your enterprise applications as well custom apps.
    • Log intelligence
      • Think of it as vRealize Log insight delivered as a SaaS offering
      • IT troubleshooting and log management across multiple clouds including VMC on AWS






VMware Skyline:  Making support easy!

You barely take a sip of your first cup of coffee after just arriving at the office.  Your mail client pops up with an email notification, your phone rings, you look up and see people walking towards your desk.  Augh, you know this is not going to be a good day.

We have all been in this situation, and it’s never fun but also inevitable. We work in IT. Things don’t always go as planned and stuff breaks.  How successful you are at fixing the issues depends how you deal with the situation. Some of us open our favorite tools (vROPS, vRLI, PowerCLI) and go straight into troubleshooting mode, while others pick up the phone and call support. Regardless of how you troubleshoot, it can be time-consuming and at times frustrating.

Well, VMware understands our pain and is trying to limit the frequency of these bad days by introducing automated proactive actions to prevent problems and resource contention before they even occur. Minority Report sci-fi stuff, right? Well not really, but the new features mentioned below are a game changer.

  • ProActive HA provides an additional layer of availability and avoids the need to use vSphere HA, so when your hardware starts showing signs of failure, DRS will kick in and evacuate your running VMs onto another host and place the faulty host into quarantine.
  • Predictive DRS provides placement and balance decisions using DRS and vROPS by predicting future demand and determine when and where hot spots will occur.

With all these new proactive features, we can certainly sleep better at night, but what if you still run into a problem after exhausting your troubleshooting skills and need to call support?

VMware did not stop with just proactive features in their product but took this one step further by introducing proactive support, as well as making support a more user-friendly experience with VMware Skyline.

What exactly is VMware Skyline?

VMware describes it the following way “It is a proactive support technology which uses automation to securely collect data and perform environment-specific analytics on configuration, feature, and performance data.  The resulting information may radically improve visibility into a customer’s environment, enabling richer, more informed interactions between customers and VMware without extensive time and investment by support administrators.”

In layman’s terms, what does this mean to you when you open a case with GSS?

  • Well first off, you might not even need to place a call. Skyline might proactively identify an abnormality from your system logs and contact YOU with the recommended fix and avert an outage. (Go ahead and read that again…customer support will contact YOU. Like I said, game changer!)
  • You do not have to upload any log files, GSS already has this information available. The manual process is very time consuming to perform and wastes precious time to collect, upload, and wait for VMware to analyze.
  • No need to provides information on your VMware products and versions, or changes that were introduced to the environment before the problem occurred, GSS already has this information available. Again, this can otherwise be a very time-consuming exercise.

You see a pattern here?

  • Reduced time-to-resolution for service requests
  • Identification of potential product bugs and guided resolution before problems occurs

How does VMware Skyline work?

The solution includes customer site and VMware cloud components.

Customer site:

A VMware Skyline Collector standalone appliance is required, which will automatically and securely collect usage data and then listen for changes, events, and patterns which are then streamed back to VMware.

VMware cloud component:

The platform receives the data from the onsite collector and performs analyses to as determine the following:

  • Alignment with VMware best practices
  • Comparing deployed products with licensing history
  • Determining if a problem is a known issue that can be addressed with an automatic remediation solution.

VMware Skyline advisor engine then delivers rich insight and recommendations, and uses rules to perform the following checks:

  • Checking of data such as configurations and patch levels
  • Cross-product, cross-cloud checks

VMware Technical Support Engineers

Takes the analyzed data and proactively reviews your environment, performs research analysis for service requests, and provides prescriptive recommendations back to you to improve your environment’s health and performance.

VMware will also provide regularly scheduled reports, which include information on observations and insight from ongoing analytics as well as provide prioritized recommendations.

Which products does VMware Skyline support?

With future plans to support all core products, VMware Skyline is initially available for Premier Support customers in North America.

  • VMware vSphere 5.5 and up
  • VMware NSX 6.1 and up
  • vSAN 6.6

Spectre and Meltdown – How to check your VMware environment for vulnerabilities

Updates added to the blog

Unless you have been on a very long vacation without internet access (The BEST type of vacation!) you should know of the Spectre (CVE-2017-5753 & CVE-2017-5715) and Meltdown (CVE-2017-5754) vulnerabilities that affect nearly every computer chip manufactured in the last 20 years.

I am not going to provide any specific details on these vulnerabilities since there are more than enough material already available, which you can read here:

I do however want to provide more detailed information related to VMware specifically, as well as different ways on how you can verify what in your VMware environment is vulnerable to these exploits:

VMware responded to the Speculative Execution security issues with KB 52245, which I highly recommend you read and subscribe to.

Intel and AMD released microcode updates that provide hardware support for branch target injection mitigation, for which VMware released KB 52085. The KB provides instructions on how to enable Hypervisor-Assisted Guest Mitigation, which is required in order to use the new hardware feature within VMs.  The KB also provides manual verification instructions for the following:

  • ESXi – Verify that the microcode included in ESXi patch has been applied
  • VM – Verify that the VM is seeing the new microcode ( VM needs to on HWv9 or newer)

ALERT: VMware also released KB 52345, which rollback the recently issued security patch recommendation (ESXi650-201801402-BG, ESXi600-201801402-BG, and ESXi550-201801401-BG). The rollback is due to customers complaining of unexpected reboot after applying Intel’s initial microcode patch on Intel Haswell and Broadwell processors.

UPDATE 01.24.18: VMware updated KB 52345 to include updated list of all Intel CPUs affected by Intel Sightings

  • VMware provides some manual workarounds for these specific processors that have already been patched.
  • For ESXi hosts that have not yet applied one of the patches, VMware recommends not doing so at this time and using the patches listed in VMSA-2018-0002 instead.

That is a lot of information to take in, and the rollbacks just add complexity to IT teams who are trying to secure their customer’s data.

UPDATE 02.15.18: VMware security advisory for VMware Virtual appliance mitigation available here

UPDATE 03.20.18: VMware provided an update to KB 52085 for patching the vSphere vCenter server to latest 6.5U1g, 6.0U3e, 5.5U3h and Hypervisor to ESXi 6.5: ESXi650-201803401-BG* and ESXi650-201803402-BG**, ESXi 6.0: ESXi600-201803401-BG* and ESXi600-201803402-BG**, ESXi 5.5: ESXi550-201803401-BG* and ESXi550-201803402-BG**.

* = Framework to allow guest OSes to utilize the new speculative-execution control mechanisms

** = Applies the microcode updates


Option 1: (The best of the best)

However, to make things a bit easier we have William Lam to the rescue who wrote an excellent script that automates the verification for both the ESXi and Virtual Machines. as well as provide ESXi microcode versions.

The PowerCLI script is called VerifyESXiMicrocodePatch.ps1 and performs the following validations

  • Verify that VM’s are running at least HWv9
  • Verify that VM completed a power cycle to see the new CPU features
  • Verify ESXi microcode has been applied
  • Verify that one of the three new CPU features are exposed to the ESXi host.
  • Verify if CPU is affected by Intel Sighting
  • Show the current Microcode version for each ESXi (requires SSH to be enabled)
  • UPDATE 01.24.18: Script was updated to validated the affected CPUs

All the detail regarding the script can be read on virtuallyGhetto here.

Option 2:  (Acceptable, but limited)

Although not nearly as thorough as William’s Script, with RuneCast Analyzer latest 1.6.7 you can detect ESXi hosts that are not protected and patched against these vulnerabilities.

Runecast Analyzer enables you to scan and detect the CPU chip vulnerabilities on your VMware infrastructure.  It detects which ESXi hosts are not protected and advise on how to patch them against such security vulnerabilities.  This solution is continuously updated as new guidance from VMware is released.

Currently only supports VMSA-2018-0002.2

Update 01.26.18: New 1.6.8 release updated to support VMSA-2018-0002.3

Screen Shot 2018-01-18 at 6.34.09 PM.png

Update 01.21.18: Option 3: (Coolest of them all)

This option does not only show what in your VMware environment is impacted but it will also assess the performance impact of both Spectre and Meltdown patches using vRealize Operations Manager (vROPS). We already know the patches will impact the speculative execution capabilities of the processor, which will lead to higher CPU utilization in your cluster due to each OS slower processing times.

The questions that come up then before patching:

  • Will I have enough resources available in my cluster to support these patches?
  • How will my ESXi host resources be impacted?
  • Should I roll out the patches in stages or all at once?

These are hard questions that are not easy to answer, or is it?

If you are using vROPS 6.6.x Advanced or Enterprise, which allows the creation of custom dashboards, then you can download and install the Spectre Meltdown Specific Dashboard kit created by Sunny Dua.  The download is available here.

The Dashboard kit consists of 3 Dashboards:

Screen Shot 2018-01-24 at 10.59.56 AM.png

  • Performance monitoring dashboard
    • Track resources utilization of your environment and will provide valuable information on the impact of patching as it relates to your Clusters, ESXi hosts  VMs.
    • Screen Shot 2018-01-24 at 11.59.56 AM.png
  • VM Patching dashboard
    • Provides views showing which VMs are running idle and can potentially be patched first since it should not have a large overall impact on performance.  Evaluate the resource utilization with the performance monitoring dashboard after the idle VM’s are upgraded, and then make a decision to continue patching or first add additional resources to the cluster.
    • Screen Shot 2018-01-24 at 11.11.41 AM.png
  • vSphere Patching dashboard
    • Shows the ESXi hosts that have been patched and also affected by Intel Sighting.
    • Shows the ESXi hosts that still needs to be patched.
    • Show the Virtual Machines that required Hardware versions upgrade since the recommended version is 9 or higher.
    • I recommend keeping an eye on VMware’s advisory site since this problem is still ongoing and the build numbers will change as new patches are released.  This will then required that you make a manual update in the filters of this dashboard
    • Screen Shot 2018-01-24 at 11.51.08 AM.png

The Performance monitoring dashboard can also be accomplished by just using the default dashboards available in vROPS standard, which means you can download the evaluation version and have that piece of mind that you can track the performance impact while going through these tough times.





Free VMware related eBooks available!

Lately, we have seen a lot of excellent guys releasing their valuable books for free! The time and effort they put into these must be crazy, and we thank them for it.  I highly recommend you pick them up and give them a read!

vSphere HA Deep Dive 6.x:

This book was made available some time ago but still relevant, and all the thanks must go to Duncan Epping. You can download it here

vSAN Essentials:

Lots of thanks and praises must go to Cormac Hogan and Duncan Epping for making their Essential Virtual SAN (vSAN) book available for free. You can download it here

vSphere 6.5 Host Resources Deep Dive:

Lots of thanks must go to Frank Denneman and Niels Hagoort for writing this book which was a big hit at VMworld 2017. The book has been made available for free by Rubrik and VMUG, which you can download it here


If you ever want to learn more about use cases for NSX then these PDF documents are a must read.

VMware NSX Micro-segmentation Day 1 by Wade Holmes

VMware NSX Micro-segmentation Day 2 by Geoff Wilmington

Operationalizing VMware NSX by Kevin Lees

Automating NSX for vSphere with PowerNSX by Anthony Burke


Vladan Seget wrote a study guide for VCP6.5-DCV which Nakivo sponsored and made available here

VMware on AWS:

AWS  made available an E-Book for VMware Cloud on AWS which covers the challenges of Hybrid Cloud adoption, how VMC on AWS is optimized for common uses cases and also the available AWS services. Available to download here

vSphere Upgrade

Emad Younis has made available his Upgrading to VMware vSphere 6.5 book which you can download here.