Spectre and Meltdown – How to check your VMware environment for vulnerabilities

Unless you have been on a very long vacation without internet access (The BEST type of vacation!) you should know of the Spectre (CVE-2017-5753 & CVE-2017-5715) and Meltdown (CVE-2017-5754) vulnerabilities that affect nearly every computer chip manufactured in the last 20 years.

I am not going to provide any specific details on these vulnerabilities since there are more than enough material already available, which you can read here:

I do however want to provide more detailed information related to VMware specifically, as well as different ways on how you can verify what in your VMware environment is vulnerable to these exploits:

VMware responded to the Speculative Execution security issues with KB 52245, which I highly recommend you read and subscribe to.

Intel and AMD released microcode updates that provide hardware support for branch target injection mitigation, for which VMware released KB 52085. The KB provides instructions on how to enable Hypervisor-Assisted Guest Mitigation, which is required in order to use the new hardware feature within VMs.  The KB also provides manual verification instructions for the following:

  • ESXi – Verify that the microcode included in ESXi patch has been applied
  • VM – Verify that the VM is seeing the new microcode ( VM needs to on HWv9 or newer)

ALERT: VMware also released KB 52345, which rollback the recently issued security patch recommendation (ESXi650-201801402-BG, ESXi600-201801402-BG, and ESXi550-201801401-BG). The rollback is due to customers complaining of unexpected reboot after applying Intel’s initial microcode patch on Intel Haswell and Broadwell processors.

  • VMware provides some manual workarounds for these specific processors that have already been patched.
  • For ESXi hosts that have not yet applied one of the patches, VMware recommends not doing so at this time and using the patches listed in VMSA-2018-0002 instead.

That is a lot of information to take in, and the rollbacks just add complexity to IT teams who are trying to secure their customer’s data.

Option 1:

However, to make things a bit easier we have William Lam to the rescue who wrote an excellent script that automates the verification for both the ESXi and Virtual Machines. as well as provide ESXi microcode versions.

The PowerCLI script is called VerifyESXiMicrocodePatch.ps1 and performs the following validations

  • Verify that VM’s are running at least HWv9
  • Verify that VM completed a power cycle to see the new CPU features
  • Verify ESXi microcode has been applied
  • Verify that one of the three new CPU features are exposed to the ESXi host.
  • Verify if CPU is affected by Intel Sighting
  • Show the current Microcode version for each ESXi (requires SSH to be enabled)

All the detail regarding the script can be read on virtuallyGhetto here.

Option 2:

Although not nearly as thorough as William’s Script, with RuneCast Analyzer latest 1.6.7 you can detect ESXi hosts that are not protected and patched against these vulnerabilities.

Runecast Analyzer enables you to scan and detect the CPU chip vulnerabilities on your VMware infrastructure.  It detects which ESXi hosts are not protected and advise on how to patch them against such security vulnerabilities.  This solution is continuously updated as new guidance from VMware is released.

Screen Shot 2018-01-18 at 6.34.09 PM.png

 

Free VMware eBooks available!

Lately, we have seen a lot of excellent guys releasing their valuable books for free! The time and effort they put into these must be crazy, and we thank them for it.  I highly recommend you pick them up and give them a read!

vSphere HA Deep Dive 6.x:

This book was made available some time ago but still relevant, and all the thanks must go to Duncan Epping. You can download it here

vSAN Essentials:

Lots of thanks and praises must go to Cormac Hogan and Duncan Epping for making their Essential Virtual SAN (vSAN) book available for free. You can download it here

vSphere 6.5 Host Resources Deep Dive:

Lots of thanks must go to Frank Denneman and Niels Hagoort for writing this book which was a big hit at VMworld 2017. The book has been made available for free by Rubrik and VMUG, which you can download it here

 

VMware Cloud on AWS

At the recent AWS re:invent conference in Las Vegas, VMware announced a bunch of new features for VMC on AWS.   Here is a complete list of the new features with some already being available and others in preview, which means they might not apply to all customers or AWS regions:

  • VMware site recovery service
    • This new service provides a great use case for an end-to-end DR solution, which simplifies DR operations, faster time-to-protect and removes the requirements for a second data center.
    • Build on top of VMware Site Recovery Manager with vSphere Replication, the service protects workloads between an on-premises data center and VMC on AWS, as well as between different instances of VMC on AWS.
  • 1 and 3-year subscriptions
    • Provide significant cost savings
    • Additional cost savings available base on the number of eligible on-premise product license you own (Hybrid loyalty program)
  • VMware Hybrid Cloud Extension (Preview)
    • In short, this is an add-on SaaS offering which will provide large-scale migration between your on-premise environment running vSphere 5.0+ and VMC on AWS.
    • Provides built-in high-performance layer 2 extensions so you will be able to keep the same networks, IP addresses, and routing policies in place during migration.
  • Layer 2 VPN (Preview)
    • Extending Layer 2 networks from an on-premise data center to VMC on AWS, which allows you to migrate VMs to your cloud SDDC without having to change their IP addresses.
    • Only one Layer 2 VPN is supported per cloud SDDC
    • Hybrid Linked Mode is optional for configuring Layer 2 VPN but is required for cold migration and migration with vMotion between your on-premises data center and cloud SDDC.
    • In your on-premise data center, you can use NSX or configure a Standalone Edge.
  • L3 VPN Generic Download (Preview)
    • This will reduce configuration issues with IPsec deployments since you will be able to download a generic configuration after VPN is configured, which provides all the parameters that need to be set on remote VPN device.
  • AWS Direct Connect
    • High speed, reliable and private network connectivity to an on-premise data center.
    • Single or Multiple DX links option available.
    • While connecting to an SDDC, customers can choose a Private VIF, Public VIF, or both VIF options.
      • Private VIF – carry vMotion and ESXi management traffic
      • Public VIF – optional, and used to establish VPN tunnel and carry management appliance and workload VM traffic.
    • VMC on AWS scale
      • Supports 32 host clusters
      • Multiple SDDC per organization
      • 10 Clusters per SDDC (future)
    • VMC on AWS regions
      • New region US East (N. Virginia)
    • Support for Wavefront by VMware
      • Collects data from application metrics collectors (Java, Ruby, Python, and more) as well as service metrics collectors (MySQL, Pivotal, Kubernetes, AWS, and more)
      • Allows customer to visualize and troubleshooting applications as well as receive alerts.
    • Scripting support
      • API
        • You can use NSX APIs and Power CLI for the Day0 and Day2 automation activities.
      • PowerCLI (preview)
        • A new module has been added since PowerCLI 6.5.4, a which enables the automation and scripting of VMware Cloud on AWS features
      • AWS SDKs (preview)
        • Existing vSphere Automation SDKs for both Python and Java will include functionality for access to VMC on AWS
      • Datacenter CLI (preview)
      • VMC on AWS API is available via a multi-platform simple command line interface
    • AWS service access enhancements
      • You have the choice to access S3 buckets over the internet or over the AWS Connected VPC.
    • VM template support in MVP
      • You can now add VM templates to Content Library, as well as delete and deploy them
    • Live migrations!! (This is a biggy, but still in preview)
      • Live vSphere vMotion will be supported over L2VPN and Direct Connect
      • Need to setup Hybrid Linked Mode (HLM) and L2VPN for this to work
    • vCenter HLM
      • Hybrid link mode sounds similar to enhanced linked mode but differs in requirements, how they work, and what problem each solves. William wrote a great blog describing the differences.
      • Supports vCenter Servers with an embedded or external PSC.
      • Support a single on-premise vCenter Server or multiple on-premise vCenter Servers that are joined to the same SSO domain.
    • External Storage access from inside Guest VM
      • NFS, SMB and iSCSI storage protocols are validated over following networks:
        • AWS Elastic Network Interface (ENI)
        • VMware Cloud on AWS Compute Gateway (CGW)
        • VMware Cloud on AWS Internet Gateway (IGW)