vSphere/vCenter 6.5 released

So post VMworld, I wrote a long article about what’s new for vSphere 6.5 which I was hoping would be published on SearchVMware.com…. unfortunately I’m still waiting on it to be published, last I heard the article was too long and they were splitting it up into two articles! ¬_¬”

Anyways, whilst I wait for the article to be published, I’ll give a quick summary of things I’ve learnt about the new vSphere/vCenter 6.5 that was released 2 days ago.

  • New HTML5 vSphere Client
  • Fully Integrated vSphere Update Manager and AutoDeploy with vCenter Server Appliance
  • Native High Availability for the vCSA
  • Native backup/restore for vCSA
  • Built-in monitoring web interface for the vCSA
  • Over 2x increase in scale and 3x in performance
  • Easy to migrate from Windows vCenter to vCSA
  • Client Integration Plugin for the vSphere Web Client is no longer required
  • The vCSA deployment installer can be run on Windows, Mac and Linux
  • The installer now supports install, upgrade, migrate and restore
  • vSphere API Explorer
  • VM Encryption / Encrypted vMotion
  • Secure Boot (for ESXi host and VM)
  • VMware Tools 10.1 and 10.0.12 (for older guest OSes that are out of support)
  • Multi-factor authentication with Smartcard or SecurID
  • VMFS-6 (4k drive support in 512e mode – emulating 512 sectors)
  • Automatic Space Reclamation – VAAI UNMAP now automatic and integrated it UI
  • VVOLs 2.0 plus VASA 3.0
  • vSphere HA is now known as vSphere Availability, enhancements to Admission Control
  • HA Orchestrated Restarts (adding in dependencies when HA restarts a VM)
  • Proactive HA (when host components are failing they are put into a quarantine mode)
  • Enhancements to DRS (VM distribution, CPU Over-commit, Network aware)
  • Predictive-DRS if vRealize Operations 6.4 is deployed (forecasted trends will kick off DRS)
  • vSphere Replication enhancements (now 5min RPOs like vSAN)


To find out more information, head along to the following:


In addition to the GA of vSphere/vCenter 6.5 there were a load of other releases on the same day:


I’m still waiting on the launch of vRealize Automation 7.2 and NSX 6.3….. those should be imminent as well!

As always, all downloads are available via the My VMware Portal.

Modifying VMware Site Recovery Manager – Windows 2012 UAC error

I first came across this issue when helping a customer uninstall Site Recovery Manager last year and wanted to blog about it but because I was pretty busy it totally slipped my mind….. until today!! I’ve been cleaning up the Solution Centre at MTI and tried to uninstall SRM for a new build…. and came across the same Windows User Access Control error. =)


Turns out that in Windows 2012. even when you go into User Accounts to turn off the UAC, it doesn’t disable it.


There’s a Microsoft Technet article which explains how to edit the Windows Registry in order deactivate UAC.

  1. Go to Start > Run, type regedit and click OK. The Registry Editor window opens.
  2. Navigate to HKEY_LOCAL_MACHINE > SOFTWARE > Microsoft > Windows > CurrentVersion > policies > system.
  3. Right-click EnableLUA and select Modify.
  4. In the Edit DWORD window, change the Value data from 1 to 0.
  5. Restart the Windows machine and re-run the SRM uninstall program.


Upgrading to SRM 5.8 and vSphere Replication Appliance 5.8

There were new versions released earlier this month for a number of VMware products, and one of the things I like to do is keep my demo environment up to date! =)

Upon upgrading SRM from 5.5 to 5.8 and vRA from 5.5 to 5.8 (I’m using vSphere replication with SRM in my demo environment), I noticed that the usual process of upgrading the vRA from the admin page doesn’t work if you point it at the online repository!

From the Release Notes, it turns out that the downloadable ISO image is the only means of upgrading from vRA 5.5.x or 5.6 to 5.8!
You’re not able to use VUM or the VAMI (Virtual Appliance Management Interface – the vRA admin page) to do the upgrade! =(

So what you have to do is download the full vRA ISO image, mount the ISO to the vRA and then change the update repository to use CD-ROM updates:


NOTE: Ensure you follow the VMware Upgrade method to the DOT…….!!
The first update went wrong as I mounted the ISO file from my desktop to the vRA via the vSphere Client, and when it updated and rebooted it lost the connection to vCenter and I didn’t have the option to re-register the vRA…. >_<”
I pretty much had to delete the vRA and install a brand new one and I’m still none the wiser as to what went wrong during the upgrade!!

vRA Upgrade Procedure

  1. Download the vRA 5.8 ISO image and copy the ISO file to a datastore that is accessible from the vCenter Server where your vRA is currently deployed.
  2. In Web Client, right-click the vSphere Replication virtual machine and select Edit Settings.
  3. In Virtual Hardware, select CD/DVD Drive > Datastore ISO File.
  4. Navigate to the ISO image in the datastore.
  5. For File Type, select ISO Image and click OK.
  6. Check the box to connect at power on and follow the prompts to add the CD/DVD Drive to the vSphere Replication virtual machine.
  7. Restart the vSphere Replication virtual machine.
  8. In a Web browser, log in to the virtual appliance management interface (VAMI).
    If you are updating vSphere Replication 5.1+, go to https://vr_appliance_address:5480.
  9. Click the Update tab.
  10. Click Settings and select Use CDROM Updates, then click Save Settings.
  11. Click Status and click Check Updates.
    The appliance version appears in the list of available updates.
  12. Click Install Updates and click OK.
  13. After the updates install, click the System tab and click Reboot to complete the upgrade.
  14. Log out of the vSphere Web Client, clear the browser cache, and log in again to see the upgraded appliance.


Once you’ve upgraded the vRA to 5.8, you can now go ahead and upgrade your SRM server to 5.8…… simply run the SRM installer and follow the installation wizard!

Note: if you didn’t upgrade the vRA before trying to upgrade the SRM server, when the installer prompts to connect to the vCenter Server, you will get the following error:


BIG WARNING HERE: As soon as you upgrade your SRM to 5.8, you can only manage it via the Web Client…..!!!
And it is completely re-written……. took me a while to work out where all the settings were located!
Same with vRA 5.8, you can’t set replication within the old vSphere client.

Note: I still have no idea where the SRM alarms are set……. ¬_¬”


PS: One of the cool things about vRA 5.8 is the ability to replicated to the cloud! =)


Edit (15/10/14) – The alarms for SRM can be found under the alarms of the vCenter Server that you connected the SRM server to:
In web-client, select vCenter the vCenter Server and under the “Manage” tab, you will find a tab called “Alarm Definitions”. This is where all the SRM alarms are located. =)

srm alarms

Site Recovery Manager SSL error

Recently had to regenerate the SSL certificates on my vCenter Server Appliance as I had noticed they were originally created with localhost.localdom as the DNS name.

Quite easy to regenerate the SSL certs, just log onto the admin page of the vCSA (http://ip-address:5480) and log in as root. Click on the Admin tab and by Certificate regeneration enabled select Yes.
When you next reboot the vCSA, the SSL certs will be regenerated – just be sure to change it back to No after it’s complete otherwise it will regenerate a new certificate every time you reboot!

A problem I discovered after regenerating the SSL certificate of my vCSA was that I could no longer connect to Site Recovery Manager using vSphere Client.
It would try to connect and then fail with a Connection Error.

As I knew this was working prior to regenerating the SSL certificate, I guessed that SRM was still trying to authenticate with the vCenter Server using the old SSL certificate.

Checking through the SRM logs (\ProgramData\VMware\VMware vCenter Site Recovery Manager\Logs\) confirmed that my assumption was correct:

The SRM logs show a certificate error:
Failed to connect: std::exception 'class Vmacore::Ssl::SSLVerifyException' "SSL Exception: Verification parameters:
The remote host certificate has these problems:

Quickest way I could think of in order to solve my issue was to modify the installation of SRM to update the certificates.
Log into the SRM server, open up Programs and Features from the Windows Control Panel. Select the entry for VMware vCenter Site Recovery Manager and click Change.
At the SRM wizard, select Modify.

You won’t be able to change the vCenter Server details but you will be able to change the authentication method. Regenerate the certificate by selecting Automatically generate certificate.
Ensure you select Use existing database, otherwise you will lose all your protection groups and recovery plans.

Once the SRM installer was finished, I was able to reconnect to SRM using vSphere client.

vCenter Site Recovery Manager – Shared Recovery Site

So a lot of customers use Site Recovery Manager as the tool to automate their Disaster Recovery Policy, usually it’s deployed on an one-to-one relationship – i.e. a single protected site and a single recovery site where the SRM/vCenter server on the protected site is paired to a SRM/vCenter server on the recovery site.

Not many people are aware that since SRM 4.0 you could actually set up a Shared Recovery Site which allows you to ‘fan-in’ your recovery from several protected sites (ie N-to-1 relationship).
This could be very useful if an organization has several remote/branch offices and requires them to be protected by a single shared site. Another example is where a service provider offers business continuity services to multiple customers – DR-as-a-Service.

In a shared recovery site configuration, you install one SRM Server instance on each protected site. On the recovery site, you install multiple SRM Server instances to pair with each SRM Server instance on the protected sites (so if you had two protected sites, in the recovery site you need to deploy two SRM servers).
All of the SRM Server instances on the shared recovery site connect to the same vCenter Server instance.

Similar to how standard SRM is deployed, you can use either array-based replication or vSphere replication.

In the diagram above, you can see that there are two field offices protected by the their Head office.

SRM Server A (Field Office 1) is paired to SRM Server C (HO)
SRM Server B (Field Office 2) is paired to SRM Server D (HO)

So what are the limitations of a Shared Recovery Site?

  • You can’t reconfigure an existing standard SRM deployment to use a shared recovery site, this is because a custom installer is used when deploying SRM which allows you to set the SRM Identifier (which must be identical for the SRM pair).
  • Each instance of the SRM Server at the shared recovery site must be deployed on its own host machine, ie you can’t deploy multiple SRM servers on the same VM.
  • Each instance of the SRM server requires its own database.
  • You’re limited to a maximum of 10 protected sites per shared recovery site.
  • When accessing SRM through vCenter Server on the shared recovery site, you can view all the SRM extensions (and pairs) and hence see all the VMs, protection groups, recovery plans, etc.

One thing you need to be very careful with is how your solution is licensed if you’re thinking about using the Reprotect functionality of SRM (or bi-directional protection).
As with standard 1:1 SRM deployments, you can re-use your protected site SRM licenses at the recovery site (assuming they are no longer in use at the protected site) and invoke reprotect/failback.
However, if you use a new set of keys at the recovery site, then you need to ensure you have enough licenses to cover the VMs that need to be reprotected.

For example:
Site A – licensed to protect 20 VMs.
Site B – licensed to protect 10 VMs.
Shared Recovery Site – licensed to protect 25 VMs.

If you don’t transfer the licenses from the protected site to the shared recovery site, then you can only perform reprotect on 25 VMs. If you recover all of the VMs from sites A and B to the shared recovery site and attempt to perform reprotect, you have sufficient licenses to reprotect only 25 of the 30 VMs that you recovered!

The Operational Limits for SRM in a shared recovery site can be found here:

At some point before Christmas I’m hoping to get this all up and running in MTI’s demo centre…… =)

Updating VMware vCenter Site Recovery Manager to 5.5

So after updating my vCenter Server Appliance to 5.5, the next obvious choice was to update Site Recovery Manager and the vSphere Replication Appliance.

Note: the way I’m upgrading my demo environment is not according to best practice…. it’s merely a quick way to document and try out the upgrade process!

If you’re planning an upgrade of your production environment, then there is a recommended order for the upgrades:

  1. Upgrade vCenter Server on the protected site.
  2. Upgrade SRM Server on the protected site.
  3. Upgrade the storage replication adapters (SRA) on the protected site.
  4. Upgrade the vSphere Replication appliance on the protected site.
  5. Upgrade any additional vSphere Replication server instances on the protected site.
  6. Upgrade vCenter Server on the recovery site.
  7. Upgrade SRM Server on the recovery site.
  8. Upgrade the storage replication adapters (SRA) on the recovery site.
  9. Upgrade the vSphere Replication appliance on the recovery site.
  10. Upgrade any additional vSphere Replication server instances on the recovery site.
  11. Configure the connection between the SRM sites and vSphere Replication appliances.
  12. Verify that your protection groups and recovery plans are still valid.
  13. Upgrade ESXi Server on the recovery site.
  14. Upgrade ESXi Server on the protected site.
  15. Upgrade the virtual hardware and VMware Tools on the virtual machines on the ESXi hosts.

When you upgrade Site Recovery Manager, there’s no real need to do anything to your database as the upgrade preserves all information in the current SRM DB (so basically preserving your protection groups, inventory mappings and recovery plans, plan run history, etc).
However, if you’re not running 5.1 or later then you will need to create a 64bit ODBC system DSN (SRM 5.0.x used a 32bit ODBC DSN). Any deployment of SRM earlier than 5.0 will require a two-step upgrade, first to 5.0.x then to 5.5.
(Note: Don’t upgrade vCenter Server directly from 4.1.x to 5.5, as this will cause the upgrade of SRM to fail when moving from 4.1.x to 5.0.x… this is because SRM 5.0.x cannot connect to vCenter Server 5.5)

Fortunately I’m going to step through an upgrade from SRM 5.1, so my ODBC is already in place!

1. On the SRM server, start the installer.

2. Choose where you wish to install the new version of SRM.

3. Choose to install vSphere Replication if you’re using it (Which I am).SRM3

4. Enter the vCenter Server credentials (the installer will notice SRM is already installed and pick up the IP address of the vCenter Server).

5. Accept the SSL certificate warning.

6. Once the installer connects to your vCenter Server, it will identify that you already have a registered extension for SRM. Click Yes to overwrite the old extension.

7. Select the type of authentication to be used (in my case I’m letting SRM auto-generate a self-signed certificate).

8. Enter the Organisation and OU in order for SRM to generate the certificate.

9. Enter an administrator’s email address in order to obtain SRM generated alerts.

10. The installer will pick up the ODBC connection and request you to enter the existing SRM DB user information.

11. Select Use existing database, choosing the other option will wipe everything from your existing SRM deployment!

12. Click Install to kick off the upgrade.

13. Once installation is complete, you will need to upgrade the vSphere Client Plug-in for SRM. You will need to uninstall the old SRM plug-in from within Windows Control Panel->Programs.

14. Once complete. Fire up vSphere Client and install the new plug-in via Plugins->Manage Plug-ins


Job done…… now to upgrade the existing vSphere Replication Appliance…. You can upgrade the vRA either using vSphere Update Manager, via the online repository, or via an offline repository (like an ISO image).
Unfortunately like the VCA, I couldn’t find an 5.5 update using the online repository so had to mount the vRA 5.5 ISO image to the vRA and update via CDROM.

1. Log into the vRA management console (https://vra-IP:5480 or http://vra-IP:8080 if its vRA 5.0.x) and select the Update tab. Select Settings and then choose Use CDROM Updates and click Save Settings.

2. Click on Status and it should display any update files it has found in the mounted ISO image. Click Install Updates.

3. Once update is complete, you will need to reboot the vRA…. don’t forget to unmount the ISO image…..


Voila….. it’s as easy as that!

Protecting your Cloud (vCloud & SRM)

So one of the BIG problems at the moment is that SRM does not fully support protecting your vCloud environment.

It supports protecting your management cluster (so the vCenter servers, vCD cells, vCNS manager, vCM, DBs, etc), but it doesn’t yet protect your resource cluster….. so all those VMs you’ve deployed in your organisations under vCD – well they’re not protected by SRM!

Definitely NOT COOL if your primary site goes tits up!!

From what I can gather, this is mainly due to the way SRM work….. When you setup SRM for DR, you have to ‘pre-create’ resources at the recovery site in order to map the resources from the protected site to them (stuff like resource pools, folders, network, placeholder VMs). Unfortunately vCD likes to have full control of a resource cluster and manages all the resource itself – this basically means that the vCD cells are not aware of the objects that have been created in the recovery site for SRM. It doesn’t matter if the names are the same, what matters is the Management object Reference IDs (MoRef ID) have changed and this is what vCD uses to construct its environment…..

MoRef IDs are used to correlate objects between vCD and the underlying vSphere/vCenter layer. Any changes to these identifiers will result in the loss of functionality because vCD will not be able to manage these objects as it will not be aware of them (ie the MoRef IDs will not exist inside the vCD DB).
The use of SRM would result in a change of the MoRef ID on the vCenter Server layer, resulting in an incorrect reference in the vCD database – and so leaving the object (eg. a VM) unmanageable from a vCD perspective. I believe SRM also re-signatures the storage volumes which will also confuse vCD.

About a year ago Chris Colotti and Duncan Epping wrote an article on how vCloud DR could be achieved, this involved the clever idea of putting the resource ESXi hosts at the recovery site into the same resource cluster as the resource ESXi hosts at the protected site (but in maintenance mode as obviously it won’t see the storage located at the protected site so can’t be used by vCD). Then using vSphere HA to take the ESXi hosts out of maintenance mode to handle the recovered workloads…. However, this solution did involved manual intervention to fail over the vCD resources correctly:

Earlier this year, another white paper was released which described how the majority of this manual process (ie the VMware bits) could be automated using PowerCLI:

However, what’s missing is the automation of the whole storage piece – breaking the replication and making the volumes read/write….. but then I guess this is really more storage-vendor dependent! =)
I guess if the storage vendor has exposed the array to VMware using VASA then it could be possible to script the storage steps as well….! =)

Anyways, it’s been an interesting read…… and definitely a problem I see VMware sorting out for the next release of SRM!

Given how powerful PowerCLI is, I really need to find some time to learn how to use it!!

VMware Partner Exchange on Tour 2013 – Reading

Wow…. it’s exactly a month since I last posted on my blog! Not good signs….. =P

TBH, I’ve been some-what busy with work, exams, training courses, webinars…… plus the last thing I’ve wanted to do after a long day at work is to log on and blog (which can be read “I’ve been lazy!”).

It’s been a busy month – finally got round to taking and passing the HP0-S35: Implementing HP BladeSystem Solutions exam for the course that I took last November (… procrastination…), so finally have my HP ATP – BladeSystem Solutions Integrator certification! Although I can’t actually check because my HP SmartPortal access has been revoked as HP have killed off anyone with access linked with 2e2! So I can’t even check my HP certs now! ¬_¬”

Also been busy trying to complete the training and certifications required to enable MTI to achieve the VMware competencies! Currently working my way through the Management Competency as I already have a few clients in place that we can submit as case studies (this will be MTI’s 4th competency out of the 6 key ones! Although we’re also eligible for Cloud IaaS).

Did quite a lot of delivery work the past month…. upgrading VMware environments to the latest and greatest…. showing clients the pitfalls of SSO and informing them to ‘get use’ to the new web-client.

Been also busy hosting a MTI Webinar on vCenter Operations Manager – what it is and what it can do to your business! Went well, loads of new leads and quite a number of clients are trialing the product. Should be hosting a technical webinar soon to ‘deep-dive’ into vCOPs! =)

Finally I’ve been very busy deploying several versions of SRM…… VMware SRM and EMC SRM….. =)

VMware SRM = Site Recovery Manager (http://www.vmware.com/products/site-recovery-manager)

EMC SRM = Storage Resource Management (http://uk.emc.com/data-center-management/storage-resource-management.htm)

What I don’t understand is why choose the same acronyms?!? Coming from a VMware background, I always think SRM is going to be Site Recovery Manager – and so will pretty much most of the market!! So why EMC had to call their new management suite SRM is beyond me!!

Anyways, it’s quite a powerful suite of products – consists of:

  • ProSphere (great for discovering end-to-end topology of your whole infrastructure…. as long as you set it up correctly! Twas a painful process),
  • Watch4Net (great dashboard which allows you to create custom reports with a ton of solution packs allowing you to connect to NetApp & EMC storage, Cisco MDS/Nexus, Brocade, VMware, etc…..)
  • Storage Configuration Advisor (great for compliancy – checking aginst EMC best practice as well as setting up other baselines that you can use to check your environment against)

The only issue is it’s really really expensive!! And not to mention a pain to setup and configure….. it would be much better when they finally put it all into one product! Also did I mention it is Stupidly expensive? I mean you’re looking at 5-6 figures!! When you consider products like EMC Storage Analytics or even VNX Monitoring & Reporting give you great functionality for monitoring and managing your EMC Storage for a couple hundred dollars, you really have to think twice when breaking open the chequebook for SRM! Anyways, EMC are trying to push us into selling this product…. but it’s just not priced for SMBs/Mid-markets…..!

Moving swiftly on….. today was VMware’s Partner Exchange on Tour 2013 @ Reading (http://www.partnerexchangeontour2013.com/reading), held at the Madjeski Stadium…… home to the recently relegated Reading FC! I thought it was ironic that the venue was the home of the Royals and that the theme of PEX was “Take Charge”….. Couldn’t help but think the theme came too late for Reading! No one really took charge of keeping them in the premiership last season! Lol….. =P

As always, some good sessions…… especially the keynote speech by Joe Baguley, describing the Software Defined Data Centre (SDDC) to be like ‘chicken farming’….. =)

TBH, the keynote was pretty similar to what was heard at VMware Forum 2013, although more cats/chickens in the story this time round…. plus a large african snail!

I found it interesting that VMware have now focused their strategy into 3 key areas:

  • Software Defined Data Centre
  • Hybrid Cloud
  • PC->Mobility

I vaguely remember that a year ago this who area was very wishy-washy with the terms End-User Computing, Application Transformation and Infrastructure Transformation….. tbh, those terms could mean anything!! Glad that they’ve now got some good headlines (and some focus) that us partners can build on!

At least this year the venue was decent….. last year it poured with rain and we all had to park on the grass at Wokefield Park….. suffice to say there was a lot of damage to the grass at the end of the day when people tried to drive off in their cars! (Like the huge skid marks I left trying to get my 3-series going – 19″ rims don’t offer much grip on soggy grass!!)

Over the past year at both 2e2 and MTI, I’ve been trying to push home my own thoughts at how a customer’s journey to the cloud should look like:

  • Adoption – starting the virtualisation journey.
  • Evolution/Optimisation – virtualising business critical apps, look into managing the virtual environment, SRM, vCOPs.
  • Revolution – private cloud.

It was nice to see a keynote slide that detailed a similar path that VMware see as the journey from server virtualisation to SDDC!

  1. Virtual Servers
  2. vSphere Operations Management (vCOPs)
  3. Software Defined Storage and Availability (SRM, VAAI/VASA, VDP)
  4. Virtual Networking and Security (VCNS, Nicira)
  5. Cloud Service Provisioning (vCD, vCAC)
  6. SDDC

Ok, so my idea was pretty much squashed into 3 steps and had fluffy marketing spin on the names, but my thought process was pretty similar…. quite chuffed that I was heading in the right direction as the bods at VMware.

Anyways, enough of an update…… roll on VMworld 2013…..!

Oh, and before I sign off….. I’ve finally succumbed and registered myself on Twitter….. my handle is @anthony_poh (https://twitter.com/anthony_poh)……. no idea what I’ll be tweeting….. I’m hoping it will be IT related, but it may end up descending into tweets of my random thoughts and moans! =P