So during a recent deployment of vCenter Operations Manager (5.8.2) at a customer site I encountered the following error whilst trying to pair the vCOPs vApp to their vCenter Server.
“Unable to get vCenter Server certificate chain”
This was the first time I had encountered this issue deploying vCOPs, fortunately given how much exposure I got to SSL certifications during a previous project I knew it could be down to one of 2 things….. either the SSL certificate had expired, or that it was not generated with the correct parameters.
Note: Quickest way to look at a vCenter Server’s SSL certificate is to just open a browser and point it at the vCenter’s IP address, then view the certificate…..
(Left – IE, Right – Chrome)
or if it’s a Windows deployment of vCenter 4.1 or later, you can find the certificate here: C:\ProgramData\VMware\VMware VirtualCenter\SSL\rui.crt (Note that c:\ProgramData is a hidden folder!).
It seemed that the SSL certificate was valid (expiry date was 2022), however I noticed that the public key certificate was weak as the key length was only 512 bits!!
What had happened was a previous partner had upgraded them from VI3.5 to vSphere 4.0 to vSphere 5.0 and had forgotten to re-generate the SSL certificates!
Prior to vCenter Server 4.1, by default VMware self-signed their SSL certificates with a public key length of 512 bits! So when they upgraded they kept the same SSL certificates.
Post vCenter Server 4.1, if you installed from scratch the public key length is set to RSA 2048 bits by default.
So because the public key length was only 512 bits, vCOPs could not authenticate the vCenter Servers’ certificate (I believe it has to be a minimum of 1024 bits)!
More info from VMware’s KB here: http://kb.vmware.com/kb/2037082 and Microsoft’s KB here: http://support.microsoft.com/kb/2661254
As it was a production environment and they couldn’t afford to regenerate their SSL certificates, I had to ‘inject’ the vCenter Server certificate into the vCOPs VMs keystores as follows:
- Copy the rui.crt file (the SSL certificate) on the vCenter Server into the tmp drive of the vCOPs UI VM. (This can be easily achieved using WinSCP).
- Login to the console of the UI VM as root.
- Change to the directory where the certificate keystore is located: /usr/lib/vmware-vcops/user/conf
- Issue this command to add the vCenter Server certificate to the certificate store: keytool -importcert -file /tmp/rui.crt -alias https://<VC FQDN or IP>/sdk -keystore truststore -storepass oxygen
- Issue this command to verify that the certificate is in the certificate store: keytool -list -keystore truststore -storepass oxygen
- Issue this command to copy the truststore file from the UI virtual machine and paste it to the Analytics virtual machine: scp truststore secondvm-external:/usr/lib/vmware-vcops/user/conf/
- Restart all services with the su – admin -c “vcops-admin restart” command, or reboot the vApp from the vCOPs admin page.
Once the SSL certificate was injected into the vCOPs VMs keystore it was plain sailing and we could continue with the setup wizard.
Ideally if you still have weak certificates in your environment, you should really be replacing them by generating new SSL certs! =)