Known bug with upgrading vCSA via VAMI

So there’s a known bug where upgrading vCSA via the VAMI freezes at 70%…. I was doing a mass upgrade of all my vCSAs in the demo environment at work, and all of them got stuck at 70%.

vcsa

After reading the Release Notes for 6.0U1b, it turns out it’s a known issue: http://pubs.vmware.com/Release_Notes/en/vsphere/60/vsphere-vcenter-server-60u1b-release-notes.html

New In the vCenter Server Appliance Management Interface, the vCenter Server Appliance update status might be stuck at 70%
In the vCenter Server Appliance Management Interface, the vCenter Server Appliance update status might be stuck at 70%, although the update is successful in the back end. You can check the update status in the /var/log/vmware/applmgmt/software-packages.log file. After a successful update, a message similar to the following is seen in the log file:
Packages upgraded successfully, Reboot is required to complete the installation

Workaround: None.

Anyways, after checking the software-packages.log, I could see the packages upgraded successfully entry so just rebooted the vCSA. All up and working again!

vcsa2

If you want steps on how to upgrade your vCSA, then have a look at my previous blog entry: Upgrading vCenter Server Appliance to 6.0 update 1

Advertisements

Upgrading vRealize Operations to 6.2

Now that vRealize Ops 6.2 has been released, it’s time to upgrade your Ops Manager virtual appliance. So how do you do that? Well, it’s pretty simple actually!

Nearly all of VMware’s virtual appliances have a simple upgrade process where you download an upgrade PAK file and upload it to the admin page of the appliance – and once uploaded it’s just a simple “click and install”….!

  1. First up, download the 6.2 upgrade PAK file from the My VMware Portal. You will required TWO upgrade PAK files, one to upgrade the vApps OS, the other to upgrade the vROps product.
    vrop01
    For an OS upgrade, the file is: vRealize_Operations_Manager-VA-OS-xxx.pak
    For the product upgrade of virtual appliance clusters, the file is: vRealize_Operations_Manager-VA-xxx.pak
  2. Before starting the upgrade it’s probably best to either take a backup or a snapshot of your entire vRealize Operations cluster as a precaution.
    Note: The cluster can be online or offline when running the upgrade.
    Log into the master node administrator interface via your web browser:
    https://<master-node-FQDN-or-IP-address>/admin
  3. On the left navigation menu, click Software Update. Note the version that vROps is currently at (for me it was 6.1). Click Install a Software Update.
    vrop02
  4. Firstly perform the OS upgrade. This updates the OS on the virtual appliance and restarts each virtual machine. Follow the wizard to locate and install the OS PAK file.
    vrop04
    Note: If you have customised the content that vROps provides – such as alerts, symptoms, recommendations, and policies – and you want to install content updates, a best practice is to clone the content before performing the upgrade. You can then select the option to reset out-of-the-box content when you install the software update, and the update will provide new content without overwriting any customised content.
    vrop03
  5. Click Upload to stage the upgrade files.
    vrop05
  6. Once upload has completed, a summary of what the PAK file contains is listed. Click Next and accept the EULA, then click Finish to start the upgrade process.
    vrop06
  7. Once the upgrade is complete, vROps will restart and you need to log back into the admin page. Navigate to Software Update and you will see a message stating what previous software update was installed.
    vrop07
  8. Now repeat the upload and installation process for the Product upgrade PAK file.
    vrop08
  9. Once again, vROps will reboot after the Product upgrade PAK file has been installed. Log back in and navigate to Software Update, you should now see that vROps has been upgraded.
    vrop09

 

There you go… nice and simple!

If you encounter any issues, then head over to the vROps 6.2 Release Notes: http://pubs.vmware.com/Release_Notes/en/vrops/62/vrops-62-release-notes.html

Upgrading vCenter Server from 5.5 to 6.0u1

Now that VMworld Europe is over I’ve had more time to sit down and look at MTI‘s Solution Centre and decided that I’d take the opportunity to upgrade my company’s primary demo environment to vSphere 6.0. Previously I had held off doing an upgrade because we run a PernixData demo environment on our main ESXi cluster and were waiting for the new FVP to be released. Now that it has (FVP 3.0), there was no reason to stick to an outdated environment!

So like most guys who don’t RTFM….. I delved straight in and mounted the vCenter ISO to kick off the upgrade – the first thing it does is run a pre-upgrade check.
vc1

Unfortunately for my environment, the pre-upgrade check flagged up an unsupported database version…..
vc2

Turns out the lowest version of Microsoft SQL Server supported is 2008 R2 SP1 and the version I deployed years ago was 2008 R2 RTM (no SPs).

To verify the SQL Server version, compatibility level, and edition you can execute a simple SQL query:

  1. Open the SQL Server Management Studio and connect to the SQL Server that vCenter Server database resides on.
  2. Run the this query on the vCenter Server database to verify the version, level and edition:
    SELECT SERVERPROPERTY(‘productversion’), SERVERPROPERTY (‘productlevel’), SERVERPROPERTY (‘edition’)
    vc2a

To find out what SQL server build you have, pop along to this great website: http://www.sqlsecurity.com/faqs-1/sql-server-versions/2008-r2

The Database Interoperability Matrix for VMware can be found here: http://www.vmware.com/resources/compatibility/sim/interop_matrix.php

So if you’re in the same position as me, you pretty much have one of two options:

  1. Do a fresh install and lose all your historical data and other configurations from vCenter.
  2. Do a database migration to a supported DB.

Fortunately for me, you can easily migrate from SQL Server 2008 to 2012 – and again you have two options on how to do this:

  1. Do an in-place upgrade where the SQL Server is upgraded where it’s currently installed
  2. Do a database migration where the old SQL DB is migrated onto a new SQL Server environment.

In my case I decided the second option would be the best option as I also wanted to upgrade the OS to Windows Server 2012. There are a number of migration options available to you, but for me the easiest option was to do a backup of the old database and restore it onto the new database!

I won’t go into how to deploy SQL Server 2012, as there are loads of tutorials online so here’s the process I did to backup and restore my DB:

Note: In order to transfer the backed up database file from the old SQL Server 2008 R2 VM to the new SQL Server 2012 R2 VM I simply added a new vDisk to the 2008 VM, backed up the DB onto that vDisk, then attached it to the 2012 VM.
You will also need to know the user account assigned to the VCDB.

  1. Before backing up the vCenter Database, ensure the vCenter Server Services are stopped.
  2. Backup the vCenter Database from within SQL Server Management Studios: Right-click the DB, select Tasks and Back Up.
    vc3
  3. Create a Full Backup and choose the destination (in my case a new disk which I will disconnect and add to the new SQL VM).
    vc4
  4. Once backup is complete, remove the vDisk from the VM, ensuring you choose the “Remove from virtual machine” option, DO NOT CHOOSE THE “… and delete files from disk”.
    vc5
  5. On the new SQL VM, create a new vDisk and select “Use an existing virtual disk”.
    vc5a
  6. Browse to the datastore containing the old SQL VM and select the vmdk file relating to the vDisk with the database backups.
    vc6
  7. Once mounted, open a console to the new SQL VM and check the DB backup files are there. Open up SQL Server Management Studio and right-click Database and select Restore Database.
    vc6a
  8. Verify options are correct and restore.
    Restoring a database automatically creates the database files that are needed by the restoring database. By default, the files that are created by SQL Server during the restoration process use the same names and paths as the backup files from the original database on the source computer.
    Optionally, when restoring the database, you can specify the device mapping, file names, or path for the restoring database.
    vc7 vc8
    vc9
  9. When a database is restored on another system, the SQL Server login or Microsoft Windows user who initiates the restore operation becomes the owner of the new database automatically.
    Once the DB has been restored, there are a number of additional configurations required, one of which is to recreate the DB security users and SQL Agent Jobs.
  10. Create a new Login to SQL Server 2012 making sure the new login matches the old one from SQL Server 2008. Assign the VCDB as the default DB and ensure the new user is the VCDB owner.
    vc10vc12vc11
  11. Finally change the DB compatibility level from 2008 to 2012. This allows the usage of the new SQL Server 2012 features. The following script can be used to automate the change (rather than going into each database property):
    USE [master]
    GO
    ALTER DATABASE [mydatabase] SET COMPATIBILITY_LEVEL = 110
    where [mydatabase] is the database to change the compatibility level
  12. Re-create all the SQL Server Agent jobs, for a complete list of the jobs that should be present, see:
    http://kb.vmware.com/kb/2033096
  13. Configure Microsoft SQL Server TCP/IP for JDBC and create a 64bit ODBC DSN.
    vc13vc14
  14. Once the DB has been restored, you can remove the vDisk that was attached with the backup files.
  15. Complete the vCenter Server 6.0 installation (I won’t go through the process here). For the demo environment, we used an Embedded PSC Deployment and when prompted we chose the DSN to the migrated VCDB and chose to use the data on this DB rather than re-initialising the DB.
    vc15

Upgrading to SRM 5.8 and vSphere Replication Appliance 5.8

There were new versions released earlier this month for a number of VMware products, and one of the things I like to do is keep my demo environment up to date! =)

Upon upgrading SRM from 5.5 to 5.8 and vRA from 5.5 to 5.8 (I’m using vSphere replication with SRM in my demo environment), I noticed that the usual process of upgrading the vRA from the admin page doesn’t work if you point it at the online repository!
srm2

From the Release Notes, it turns out that the downloadable ISO image is the only means of upgrading from vRA 5.5.x or 5.6 to 5.8!
You’re not able to use VUM or the VAMI (Virtual Appliance Management Interface – the vRA admin page) to do the upgrade! =(

So what you have to do is download the full vRA ISO image, mount the ISO to the vRA and then change the update repository to use CD-ROM updates:
srm3
srm4

 

NOTE: Ensure you follow the VMware Upgrade method to the DOT…….!!
The first update went wrong as I mounted the ISO file from my desktop to the vRA via the vSphere Client, and when it updated and rebooted it lost the connection to vCenter and I didn’t have the option to re-register the vRA…. >_<”
I pretty much had to delete the vRA and install a brand new one and I’m still none the wiser as to what went wrong during the upgrade!!

vRA Upgrade Procedure

  1. Download the vRA 5.8 ISO image and copy the ISO file to a datastore that is accessible from the vCenter Server where your vRA is currently deployed.
  2. In Web Client, right-click the vSphere Replication virtual machine and select Edit Settings.
  3. In Virtual Hardware, select CD/DVD Drive > Datastore ISO File.
  4. Navigate to the ISO image in the datastore.
  5. For File Type, select ISO Image and click OK.
  6. Check the box to connect at power on and follow the prompts to add the CD/DVD Drive to the vSphere Replication virtual machine.
  7. Restart the vSphere Replication virtual machine.
  8. In a Web browser, log in to the virtual appliance management interface (VAMI).
    If you are updating vSphere Replication 5.1+, go to https://vr_appliance_address:5480.
  9. Click the Update tab.
  10. Click Settings and select Use CDROM Updates, then click Save Settings.
  11. Click Status and click Check Updates.
    The appliance version appears in the list of available updates.
  12. Click Install Updates and click OK.
  13. After the updates install, click the System tab and click Reboot to complete the upgrade.
  14. Log out of the vSphere Web Client, clear the browser cache, and log in again to see the upgraded appliance.

 

Once you’ve upgraded the vRA to 5.8, you can now go ahead and upgrade your SRM server to 5.8…… simply run the SRM installer and follow the installation wizard!

Note: if you didn’t upgrade the vRA before trying to upgrade the SRM server, when the installer prompts to connect to the vCenter Server, you will get the following error:
srm1

 

BIG WARNING HERE: As soon as you upgrade your SRM to 5.8, you can only manage it via the Web Client…..!!!
And it is completely re-written……. took me a while to work out where all the settings were located!
Same with vRA 5.8, you can’t set replication within the old vSphere client.

Note: I still have no idea where the SRM alarms are set……. ¬_¬”

 

PS: One of the cool things about vRA 5.8 is the ability to replicated to the cloud! =)

 

Edit (15/10/14) – The alarms for SRM can be found under the alarms of the vCenter Server that you connected the SRM server to:
In web-client, select vCenter the vCenter Server and under the “Manage” tab, you will find a tab called “Alarm Definitions”. This is where all the SRM alarms are located. =)

srm alarms

Upgrading vCenter Server

Just a quick post today as I’ve had numerous clients call me recently regarding the same issue with upgrading their vCenter Server…..

If you’re upgrading from vCenter Server 5.5 to a later addition (like 5.5 update 1c), then if you mount the ISO and try to update vCenter using the “Simple Install”, the installer will probably display the following message:

vcerror

 

This basically means that some of the components of the update may already be installed. To continue with the upgrade you’ll need to update the individual components under the “Custom Install” option.

… simples….. =)

vCenter Server 5.5.0a Released

Hmm…. an update to 5.5 already!

Release notes here:
https://www.vmware.com/support/vsphere5/doc/vsphere-vcenter-server-550a-release-notes.html

If you’re using the vCenter Server Appliance (vCSA) and have already upgraded from 5.1 to 5.5, then updating the appliance is a very simple process if your vCSA is able to connect out to the internet:
Note: this update process will reboot your vCSA so you will lose connectivity to your vCenter Server! Be warned!

1. Log into the admin page of the vCSA – https://ip-address:5480 using the root account (default password is ‘vmware’)
Image

2. Select the Update tab and click Check Updates under the Status section
Image

3. Select Install Updates and click OK to start the update process
Image
Image

5. Once update is complete (for me it was about 10mins – dependent on size of DB), reboot the vCSA to initialise the new update.
Image

6. Click the System tab and click the Reboot button.
Image
Image

Once rebooted, the vCenter Server should be assessible again.

If you are not connected to the internet, then you can download the update ISO file from your ‘My VMware’ portal.
Once downloaded (about 1.6GB), mount the ISO file to the vCSA VM and navigate to the Settings section under Update. Select Use CDROM Updates as the Update Repository, click Save Settings and then follow the instructions outlined above from Step 2.
Image

The Joys and Pains of upgrades…..

So with the latest update releases across all the VMware products, I set about upgrading my demo kit last friday……

Upgrading the vCenter Server Appliance was straight forward enough….

  • mount the update iso to the VM
  • browse to the management webpage
  • click Update Repository selecting Use CD-ROM Updates
  • follow the wizard and kick back and relax as it goes off and does its job…..

After 2hrs and a quick reboot – voila…. sorted!

Next up was an update of my demo DR vCenter Server installed on a Windows VM with SQL Express…. again, simple and quick……

  • mount the ISO file, run the SSO installer (separate one, not the simple as that doesn’t work), run through the wizard making sure you use the SSO admin password! Reboot (as it changes the window services).
  • run the Web Client installer. Once complete check SSO hasn’t blown up or lost it’s AD domains! =)
  • run the Inventory Service installer.
  • run the vCenter Server installer.
  • run the vSphere client installer.
  • an hour later and it’s all done (actually I was multi-tasking with other work, so I guess you’re talking 20mins in total).

At some point I’ll have to update the production vCenter Server which I’m hoping will be the same pain-free and quick process – only difference is we’re using SQL Server 2008 R2…. if I’m daring enough maybe I’ll patch the SQL server to SP2…. =)

The only issue I encountered was after upgrading the VDP Appliance, it now doesn’t run backup jobs!

There seems to be a general consensus in the community that VDP either works or doesn’t! It either backs up VMs flawlessly, or it errors out and you spend hours trying to work out what caused the error because the logging functionality is pretty pants!

The reason I jumped straight into the VDP upgrade to 5.1.10 was because of the known issue with backing up Windows 2008 R2 VMs – mentioned here: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2035736

However, after a long and quite a tedious upgrade process, the VDP appliance now fails to run any backup jobs and errors out with a ‘no proxy available to service backup jobs’:

vdp: Failed to initiate a backup or restore for a virtual machine because no proxy was found to service the virtual machine

Pretty much stuck now as I can’t find any mention of how to clear out the error or re-attach the proxies…… *sigh*

Still early days for this update and so far I can’t find anyone else online who has encountered the same problem!

May have to give VMware tech support a call……

 

Edit: Well looks like there are problems with the upgrade process already – mainly SSO (yet again) not working well with multiple identity domains and users associated with a large number of groups: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2050941