VMworld 2013 US ends…… so here’s my round up!

So VMworld 2013 has come to an end in the US….. and with no real big announcements, many people may have come away a bit disappointed.

But as I mentioned in a previous post, “VMware-the-company” has gone through quite a bit over the past year or so…. restructuring/redundancies, new CEO, new focus….. so you can kind of understand why nothing BIG was going to be announced.

However, that said, I believe they’ve still managed to excite the VMware community with the announcements made….

  • Official launch of NSX – details of the Nicira product has been floating around for months and it’s good to finally see what it can do!
    Announcement here: http://cto.vmware.com/introducing-vmware-nsx-the-platform-for-network-virtualization/
  • vSphere 5.5 – whilst no real ground-breaking announcements, the 5.5 release builds on what vSphere 5.1 achieved.
    I think some people might have hoped for a new version release (The “Tock” to 5.1s “Tick”), given how in the past it was always 1 major release before a new version, but given what 5.1 brought to the table (SSO, web-client, VDP, etc) I guess it didn’t make much sense to bash out a brand new version before the new functions had bedded in.
    But what’s great is that every time VMware announce a new release, there are a ton of new functionalities and features to go with it! =)

    • SSO – Glad to see they’ve listened to some of their customer frustrations and improved SSO to make it more streamlined and easier to deploy!
      I’ve read that it’s a complete rebuild/rewrite of the SSO architecture to enable it to scale and serve vCloud Director and Horizon View.
      SSO replication has been redesigned and it sounds like multi-SSO deployments will become easier to implement (in part due to no longer requiring an external DB which was a major PiTA to make available between multiple SSO instances!).
      It also has an improved AD authentication system which speeds up AD searches (especially if you have a super-large AD).
    • vSAN – a software based distributed storage solution built directly IN the hypervisor – so no virtual appliance required! Whist not GA with 5.5, this will probably show up at the next update (Q1 2014??).
      From what I have read, it takes all the disks on a host (at least 1 SSD and 1 magnetic HDD) and uses them to create the virtual SAN (it is very much like the vSphere Storage Appliance). It’s easy to deploy, scalable (although the beta limits to 8 hosts) and allows granular polices to be set on the VMs rather than on the LUN/datastore.
      Seems the SSD will be used as a read cache/write buffer and the HDDs as the actual storage.
      Hosts in the vSAN cluster can either be storage nodes (ie their local disks are put into the storage pool) or compute nodes (which just leech off the storage pool).
    • vFlash Cache – Much like how VMware pools CPU and RAM, this new feature does the same with local host flash devices (like SSDs). It’s a local host caching solution (using local SSD drives to offload IO), although currently it only operates as write-through caching (hmm… EMC vfcache or XtremeSW cache anyone?!?)
      VMs can be configured to use this pool of Flash resource to accelerate the read performance (so acceleration of applications) – it can’t be used as storage!
      I believe a write-back mode is in the pipelines for a future release.
    • vCenter Server Appliance – embedded Postgres DB has been re-engineered and supports up to 500 hosts or 5000 VMs. Now a GREAT alternative to deploying vCenter Server on Windows! Sounds production ready now (rather than just for PoCs and test/dev)!
    • App HA – application level HA. vSphere HA only currently recovers a VM, it isn’t application aware! vSphere App HA has the ability to monitor certain applications and can be configured to restart an application service when issues are detected. (Although it requires vFabric Hyperic to be deployed in order to gain the insight into the application)
    • Other updates:
      • vCO – new debugging mode, updated workflow schemas, integrations with vCAC
      • Web Client – faster UI, drag & drop
      • VDP – disk level granularity (backup/restore of VMDKs), replication to the cloud.
      • VUM – integrated with Web Client (although baseline creations still in vSphere client)
      • VMDK size – now up to 62TB
      • VM Hardware version 10
      • Increased maximums
  • vCloud Suite 5.5:
    • vCloud Director – improvements to the content catalog, provisioning vApps, vApp lifecycle management, importing/exporting vApps to OVFs.
    • vCloud Networking & Security – improvements to LACP (bonding uplinks together to increase bandwidth) and flow based marking & filtering (ability for granular filtering of traffic). Improvements to Firewall rule management.
    • Site Recovery Manager – Looks like it now supports Storage DRS, Storage vMotion and the new vFlash and vSAN feature (using vRA).
      • vSphere Replication – improved UI in the web client. Improved feature to allow multiple Points-in-Time snapshots (allowing you to select earlier snapshots rather than just the last known snapshot – max 24 replicas). Support for Storage DRS, Storage vMotion and vSAN. Scalability improvements allowing up to 10 vRAs per vCenter Server. Ability to replicate (vRA) to a remote site that doesn’t have a vCenter Server deployed.

And lastly, vCloud Hybrid Services (vCHS)…
Pretty well known that VMware have been looking at taking on Amazon and Azure with their own Public Cloud offering, however they’ve been taking a slightly different approach where they are engaging service providers to deploy vCloud Suite and join their ‘vCloud Hybrid Service’ (vCHS) program. So it’s announced that from Sept 2013, the vCHS will be available in 6 datacenters in the US.

The BIG difference between vCHS and the other competitors is that it’s designed to be a true extension of a clients private enterprise cloud running on vSphere and vCloud. VMs can easily be moved to and from the vCHS without any modifications – so pretty much allowing customers to burst out of their private cloud into the vCHS!

So the recent press release also mentions that they will be offering Desktop as a Service (DaaS), Platform as a Services (PaaS) and Disaster Recovery as a Service (DRaaS) in addition to IaaS!
This pretty much means you can run VDI in your private cloud and then burst out to the vCHS, allowing you to run both server and desktops either on-premise or in vCHS.

Whilst not yet being rolled out in the UK, I can only assume that Saavis will be one of the first to roll it out (given they’re involved in the US roll out).


So…. that’s my summation of VMworld announcements…… there was a whole load of other stuff (like EMCs VNX2 and ViPR), but I would be here all day writing this blog if I had to digest all the sessions that occurred!

Roll on Barcelona!! =)


VMworld 2013: Next-Gen Backup

So during my searches online for all things VMworld related, I stumbled across this interesting blog entry by VirtualGeek regarding backups….. always a bane of any VMware admin!

It seems that EMC and VMware have shacked up once again and the new features on Avamar 7 shows a tighter integration with vCenter…. it even looks like it’s using a vCenter web-client extension that we saw with VDP (not surprising since VMware took Avamar’s algorithms, so why shouldn’t EMC take stuff from VDP?).
The self-service recovery portal looks surprisingly similar to the VDP version (albeit with more functionality).

The demo showing Avamar working together with Data Domain to fire up a VM from within the DD storage was pretty impressive – EMC call this “VM Instant Access”!
Basically DD will be used as a backup target for Avamar (with Avamar itself determining whether to push the data to DD or to an Avamar grid – usually high changing data is sent to the DD), the VM will be restored via a NFS datastore mount from within DD itself.
Looks very simple and very effective…. especially since after firing up the VM in the mounted datastore, you can just simply Storage vMotion it back into your production environment!
Simple, Instant Access! =)

The Backup-as-a-Service demo was pretty interesting regarding how they use the vCHS and Avamar for BaaS…. whilst not fully released yet for production, it gives a good insight into what’s possible and the direction both VMware and EMC are taking.

Finally the Avamar and vCOPs/LogInsight integration demo was very cool……. vCOPs is a great management tool which I try to encourage all my VMware customers to buy and use…… having the Avamar integration would be very beneficial as you get a single dashboard which will show not only the health of your VMware environment, but also the health of your backups!
And with the integration you will also get to see all the metrics for Avamar (GB protected, dedupe, VM backup related information, etc) alongside all the usual VM metrics that comes as standard with vCOPs.


BTW, no keynote session on Day 3 of VMworld….. so pretty much most blogs are re-enforcing what was talked about during the first 2 days….. Seems day 3 was pretty much vendor related with a lot of people checking out what NetApp and EMC had to offer. A lot of blog entries about VNX 2 and MCx…..!

VMware Desktop as a Service will be powered by Desktone

VMware Desktop as a Service will be powered by Desktone.

More stuff coming out of VMworld in the US….. interesting how vCHS (vCloud Hybrid Service) will actually offer Desktop as a Service where you can push your VDIs into a public cloud provider.

As always, the US gets all the goodies before the rest of the world!
Will be really interesting to see which providers in the UK jump on board the vCHS offering…..


VMworld 2013 announcements – vCloud Suite

So there are times when a blogger becomes just pure lazy (ie this is one of those times), so rather than regurgitate everything I’ve been reading today that came out of the first 2 days of VMworld, I thought it would just be far easier to link to a blog that pretty much sums up what has been announced.

http://www.wooditwork.com – Justin Wood’s Blog

Flash Storage

So Duncan Epping (http://www.yellow-bricks.com) has put together 3 short blog posts about the numerous Flash Storage start-ups that are springing up…… seems every man and his dog wants to start up a flash storage company these days or get in on the high IO/Performance that can be gained by using flash memory or SSDs! =P

Anyways, whilst he hasn’t covered every start-up yet (Notable absences: Nimble, Whiptail, Pure and Tintri)…. it still makes very interesting reading!


Will be very interesting to see what happens in this market over the next few years…. especially with the likes of Fusion IO looking to enter the market with their NexGen acquisition, and the big vendors – EMC (XtremeIO), HP (3Par Flash array), NetApp (EF540) – all looking to jump on the flash/hybrid bandwagon……
The market’s gonna get saturated by arrays, and it’ll be the software and application integration that separates the winners from the losers!

VMworld 2013 just round the corner!

So with VMworld 2013 kicking off in San Francisco on Sunday, the internet (or at least the VMware community) are awash with what could be announced…..
With it being the 10th Anniversary of VMworld, you would expect some big announcements to go alongside a big shindig……!
However, from what I’ve read we might be in for a bit of a disappointment (that or VMware have been taking a leaf out of JJ Abrams’ books by keeping their cards really close to their chest!).

It’s been interesting reading what’s happening within VMware’s ranks this year…..
Their new(ish) CEO – Pat Gelsinger – decided quite early on in his reign to refocus the company’s route to market – focusing on their core server virtualisation (vSphere) and their cloud business (vCloud Suite).
There’s been a big push on the Software Defined Data Center (SDDC) messaging as it’s a key foundation to the direction VMware are moving…..
As such, they went through a bout of restructuring in Q1 (even though their revenue increased over last years), and cut a lot of their workforce….
And they’ve sold off some of the smaller apps/solutions which now fall out of scope:
Zimbra – which let’s be honest never got anywhere!
WaveMaker – which was a Java development tool that they didn’t really build upon or integrate (obviously bought to try and integrate with the vFabric solutions)
Shavlik – patch management tools which didn’t quite get rolled into vCenter or other management products….
(Just to name a few)

VMware have refocused on 3 key strategies:

  • SDDC (vSphere)
  • Hybrid Cloud (vCloud)
  • Mobility (Horizon)

Whilst already established in server virtualisation (vSphere being market leaders and probably the best hypervisor in the market – IMO), they’re still quite new into cloud computing and still struggling to go up against the likes of Amazon.
vCloud Suite, for all it’s advancement over the past year or so, still falls short of usability……
It’s great for deploying and managing the underlying cloud virtual infrastructure (albeit not great with DR of cloud resources – see my previous blog posts), but it is very lacking on the orchestration and self service portal front…… something they’re trying to address with vCloud Automation Center (the old Dynamic Ops product). But this is yet another layer upon a layer upon a layer….. It’s becoming like Microsoft!!
End users love Amazon because it’s EASY to use and administer……. something you can’t really say about vCloud Director.

Cisco have also waded into the cloud management market by buying out Cloupia and re-branding it UCS Director (lol)……. whilst the re-branding doesn’t go down well with all, it’s undeniable that the Cloupia product itself is quite a good tool!
It’s easy to use, easy to administer, easy to setup, a single pane of glass to manage virtual AND physical infrastructure, plus a decent end-user portal!
(Probably something I’ll post about at a later date)

Then there’s the spin off by EMC and VMware – The Pivotal Initiative – chucking big data Greenplum, Pivotal Labs, and Cloud Foundry together (headed by the old VMware CEO – Paul Maritz)…. pretty much a cloud PaaS offering.
(Which I won’t go into as I haven’t read up enough about them)


Anyways, I’ve digressed away from VMworld……. So to summarise what I’ve read/heard:

  1. vSphere 5.5 release (not the big v6.0 that most people were expecting)
    • A re-written, simplified SSO (yay!)
    • Enhancements to vSphere Replication
    • Increased maximums
    • Possibly another new VM hardware version (v10)
    • Improved Web Client (given that the vSphere client will probably be dropped at the next big version release)
  2. Tighter integration of the vCloud Suite products (possibly doing away with all the different products by rolling some of it into vSphere – like vCNS)
  3. Enhancements to Horizon Suite

Unfortunately I couldn’t find much info on the feature upgrade/enhancements on the latter two…. but then again, it’s all rumours at present. We’ll find out next week!

A few things I would like to see announced at VMworld:

  1. Product launch of NSX which they’ve been working on since the acquisition of Nicira.
  2. Re-engineered version of SRM which will protect cloud resources.
  3. Integration of vCAC and vCD into a single product. Or at least making vCAC easier to deploy with vCloud.

Anyways, should be a good show next week…… and I’m still hoping to head on out to Barcelona when VMworld hits Europe in October!

DRS Invocation Not Completed

So came across an issue with my VMware cluster today where vCenter was overloading one of my hosts…..

Had a quick look at the cluster and under the summary tab it displayed the following error:


Hmmm……. upon further inspection it turns out that the hosts weren’t displaying their utilisation properly – both CPU and memory displayed 0% utilisation:


Turns out that DRS may have got its knickers in a twist and it wasn’t able to load balance across the cluster, possibly because it could not contact the other two hosts to determine the available resources…. as you can see from the picture above, everything ended up on my second ESXi host!

A quick google shows that I’m not alone in experiencing this issue…. but there didn’t seem to be any reference to VMware KBs or an official line from VMware regarding this issue.

Turns out most people just ‘disconnected’ and ‘reconnected’ the offending host and it fixed the issue! I did just that and it cleared the DRS configuration issue and started to display the host resource utilisation again….. and after a while DRS kicked in and re-balanced my cluster!



I can only assume something caused the DRS or HA configuration of the cluster to go a bit funny – how or what caused it, I’m not sure……


Disconnecting and reconnecting an ESXi host is non-disruptive and doesn’t turn off VMs, all it does is remove the HA agent from the host and un-protect the VMs, and then re-enable the HA agent and re-protect the VMs.

I’m sure VMware are aware of this issue, but given it hasn’t been addressed in the latest release of vCenter Server (5.1.1b Aug 1st 2013), I can only assume they are none the nearer of discovering what’s causing the issue!

Troubleshooting a vCloud Director Installation

The problem about working full time is it’s really hard to find time to blog, and also to find topics to blog about! =)

One of the great things about my job is we have a solution centre in the office which allows me to play around with kit! =)
Our solution centre is based around an EMC VSPEX architecture….. so EMC VNX storage, Cisco UCS blades and VMware virtualisation!!

I’ve been busy the last week or so putting together a vCloud solution for some of the engineers to play around with, as well as finally completing the detailed installation guide for deploying the vCloud Suite (one of these days I promise I will post it up).

Anyways, so I ended up installing two RHEL 6.2 VMs as my vCD cells on a MS SQL 2008 R2 DB, load-balanced using a vCNS edge….. but when I tried to start the vCD services on my linux VMs, they would say they’ve started (simple service vmware-vcd status command) but wouldn’t give me the vCD web console/UI….. all I got was a Blank Grey Webpage and after a while it would error out saying it couldn’t connect to the website!! Hmmmm……

Anyways, this gave me a good opportunity to test out my troubleshooting skills and offer a topic for my blog! =)

So here goes……

Troubleshooting vCD….

The Log files for vCloud Director are located at /opt/vmware/vcloud-director/logs. There are three main files to look at (well there’s more than 3 but these are the ones I usually use and 99% of the time I can work out what’s wrong):

1. cell.log

This log file provides information on the status of the vCloud Director cell services and the application as it starts up.
Use tail -f cell.log to view the live status when starting a vCloud Director Cell.
A successful start up will allow you to access the vCD web-console/UI and will display a started status for each service, plus 100% for Application Initialization.

Usually if there is an issue with accessing the web-front end UI then it is more than likely that the services are still waiting to complete, as below:


If you’re seeing lots of services showing a “WAITING” status, then check the other logs to determine what could be causing this issue.

2. vmware-vcd-watchdog.log

This log file shows any alerts, errors or information that the vCloud Director cell services maybe experiencing. A healthy vmware-vcd-watchdog.log looks similar to the below:


If there’s an issue, then you could get an ‘Alert’ entry, similar to the one below:


I believe vCloud Director will automatically try to re-start the services as I didn’t see a time stamp for an entry when I manually restarted the service. Also this log looks very similar to what you would get if you typed in ‘service vmware-vcd status‘ as that command reports on both the vmware-vcd-watchdog and vmware-vcd-cell services.

3. vcloud-container-info.log

This log file shows the status of the initial installation of vCloud Director and will log how the application is currently functioning. If you have any errors or failures during installation, this log file will provide you with the details required to troubleshoot the cause of the failure.
In addition, this log will also provide information on any errors that may cause the vCloud Director services to fail to start.
In my case, after doing a cat vcloud-container-info.log | more I discovered the following error:


Turns out that the error shows that the vCloud Director cell could not resolve its hostname in DNS.

When I went through the pre-reqs before installation, I realised that I had only put in DNS entries for the two IPs used for the HTTP and the Remote Console access….. I forgot to put an entry into DNS that resolved the hostname of the Linux VM to the HTTP IP address.
A quick edit to DNS and then a restart of the vCD services fixed the problem I experienced.

4. vcloud-container-debug.log

This log file shows the debugging information. The detail in this log file will be dependant upon the level of debugging set. I didn’t actually end up looking at this log as the error was discovered in the -info.log…. However, it’s another port of call if you can’t work out what’s causing your vCD services to fail.

Rights….. blog entry over…… I’m off to eat my dinner! =)