Troubleshooting a vCloud Director Installation

The problem about working full time is it’s really hard to find time to blog, and also to find topics to blog about! =)

One of the great things about my job is we have a solution centre in the office which allows me to play around with kit! =)
Our solution centre is based around an EMC VSPEX architecture….. so EMC VNX storage, Cisco UCS blades and VMware virtualisation!!

I’ve been busy the last week or so putting together a vCloud solution for some of the engineers to play around with, as well as finally completing the detailed installation guide for deploying the vCloud Suite (one of these days I promise I will post it up).

Anyways, so I ended up installing two RHEL 6.2 VMs as my vCD cells on a MS SQL 2008 R2 DB, load-balanced using a vCNS edge….. but when I tried to start the vCD services on my linux VMs, they would say they’ve started (simple service vmware-vcd status command) but wouldn’t give me the vCD web console/UI….. all I got was a Blank Grey Webpage and after a while it would error out saying it couldn’t connect to the website!! Hmmmm……

Anyways, this gave me a good opportunity to test out my troubleshooting skills and offer a topic for my blog! =)

So here goes……

Troubleshooting vCD….

The Log files for vCloud Director are located at /opt/vmware/vcloud-director/logs. There are three main files to look at (well there’s more than 3 but these are the ones I usually use and 99% of the time I can work out what’s wrong):

1. cell.log

This log file provides information on the status of the vCloud Director cell services and the application as it starts up.
Use tail -f cell.log to view the live status when starting a vCloud Director Cell.
A successful start up will allow you to access the vCD web-console/UI and will display a started status for each service, plus 100% for Application Initialization.
Image

Usually if there is an issue with accessing the web-front end UI then it is more than likely that the services are still waiting to complete, as below:

Image

If you’re seeing lots of services showing a “WAITING” status, then check the other logs to determine what could be causing this issue.

2. vmware-vcd-watchdog.log

This log file shows any alerts, errors or information that the vCloud Director cell services maybe experiencing. A healthy vmware-vcd-watchdog.log looks similar to the below:

Image

If there’s an issue, then you could get an ‘Alert’ entry, similar to the one below:

Image

I believe vCloud Director will automatically try to re-start the services as I didn’t see a time stamp for an entry when I manually restarted the service. Also this log looks very similar to what you would get if you typed in ‘service vmware-vcd status‘ as that command reports on both the vmware-vcd-watchdog and vmware-vcd-cell services.

3. vcloud-container-info.log

This log file shows the status of the initial installation of vCloud Director and will log how the application is currently functioning. If you have any errors or failures during installation, this log file will provide you with the details required to troubleshoot the cause of the failure.
In addition, this log will also provide information on any errors that may cause the vCloud Director services to fail to start.
In my case, after doing a cat vcloud-container-info.log | more I discovered the following error:

Image

Turns out that the error shows that the vCloud Director cell could not resolve its hostname in DNS.

When I went through the pre-reqs before installation, I realised that I had only put in DNS entries for the two IPs used for the HTTP and the Remote Console access….. I forgot to put an entry into DNS that resolved the hostname of the Linux VM to the HTTP IP address.
A quick edit to DNS and then a restart of the vCD services fixed the problem I experienced.

4. vcloud-container-debug.log

This log file shows the debugging information. The detail in this log file will be dependant upon the level of debugging set. I didn’t actually end up looking at this log as the error was discovered in the -info.log…. However, it’s another port of call if you can’t work out what’s causing your vCD services to fail.

Rights….. blog entry over…… I’m off to eat my dinner! =)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s