Getting a valid certificate on your vmware vSphere vCenter 6.7

I thought I should share my finding of this rather simple process (when you finally get it working) after spending a few hours troubleshooting and researching.

By default vCenter holds its own CA that caters to all moving parts within vSphere. From version 6.0 it’s possible to just replace the machine SSL certificate on the VCSA and the external PSC, a model also know as the hybrid model.
Hybrid because it keeps the internal CA for all other functions that don’t relate to the machine certificate.

As mentioned, the process is well documented and there are articles all over the place, so why follow this one?

For one, it’s one of the few tested on version 6.7 – and it also describes how to get the certificate when your certsrv website of your Enterprise PKI is not working for some reason; all hail the command line. I also found some of the current documentation a bit confusing so I wanted to make it a little more clear on what’s going where.

In this example we had the following environment:

• A running Enterprise PKI based on ADCS installed on a Windows 2008 server with a broken certsrv website (meaning that newly created templates are not published to the certsrv website)
• I used a fresh install of VCSA 6.7 with embedded PSC

Tools required:
SSH, SCP, OpenSSL

For a complete machine SSL certificate replacement on your vSphere environment, please note the following:

• If you have a vCenter Server with an embedded Platform Services Controller (PSC), there will be one Machine SSL certificate.
• If you have a vCenter Server with an external Platform Services Controller, each machine will have its own Machine SSL certificate. Therefore, you must perform this task on each machine.

Let’s get cooking

The first thing we need to do is create the certificate request (CSR) on the VCSA. This process generates the CSR with an accompanying private key.

SSH to the VCSA with the root user and execute the following command to temporary switch from the appliance shell to bash.

> shell

Since we are going to use the shell remotely when copying out the files from the VCSA, we’ll start off by changing root’s default appliance shell to the bash shell. So execute the following command to change the shell. You can always change it back if necessary.

# chsh -s /bin/bash root

Now we are ready to start the certificate manager with the following command:

# /usr/lib/vmware-vmca/bin/certificate-manager

Choose option 1 and proceed to enter a valid SSO and VC privileged user credential in order to proceed with the certificate request.

Choose option 1 and fill out the certificate details:

If you entered the details in the correct format, you will now have generated the CSR and private key.

Copy the CSR file of the VCSA using this command from a remote computer with scp installed.

scp [email protected]:/tmp/vmca_issued_csr.csr ./

Completing the request

If you copied the CSR onto a windows machine you may already have certutil.exe and certreq.exe on there. You may need to execute as an administrative user if you have not assigned permissions to the certificate template to your own user. Otherwise move the CSR file retrieved from the VCSA directly onto the CA server.

Before we can complete the certificate request we need to create a certificate template that will determine some of the properties of the certificate. You you already have an appropriate template in place you can skip this step.

Follow this article from VMarena on how to configure the template:

If you followed the article, you should end up with a template named “vSphere 6.7”. We’ll use that when requesting the certificate.

As I mentioned, in this scenario our CA certsrv website is not showing newly published templates so we will complete the request using the next best thing after sliced bread, the command line interface; AKA the CLI!

Open an administrative Command Prompt on the server and execute the following line taking care to either change to the directory where you have transferred the CSR file or changing the command to reflect the full path to the file.

certreq -attrib "CertificateTemplate:vSphere6.7" -submit vmca_issued_csr.csr

NOTE: As you can see, the space was removed from the certificate template name “vSphere 6.7” – If you don’t remove it, you will get this error “Denied by Policy Module 0x80094800”.

You will receive a dialog asking you to choose what CA you want to use for issuing the certificate – make your choice and click OK.

Next you’ll receive a “Save As” dialog. Choose a location and name the file:

vmca_issued_cer.cer

Let’s get the CA certificate chain while we are at it. The following command will export the chain in a binary format containing the root CA certificate and subordinate CA certificate(s) in the binary container format, DER.

certutil -ca.chain chain.der

With the chain exported we need to convert it to base64, extract the certificated from container format and construct a file with the certificate chain, one certificate at a time. We can do that with a one-liner using openssl.

So move the chain.der to a computer where you have openssl installed.

openssl pkcs7 -inform DER -outform PEM -in ./chain.der -print_certs > vmca_issued_chain.cer

Now copy the vmca_issued_cer.cer and vmca_issued_chain.cer back to the VCSA. If you have stored them in the same directory, you can just use the following command.

scp vmca_issued_* [email protected]:/tmp

We now have the three files we need to install the machine SSL certificate.

• vmca_issued_cer.cer – This is the certificate itself
• vmca_issued_key.key – This is the matching private key
• vmca_issued_chain.cer – This contains the certificate chain of trust

Meanwhile, back on the VCSA…

So we got all the files we need in the /tmp folder, so lets install the certificate.

Access the VMCA Certificate Manager and choose option 1 and then option 2.

Please provide valid SSO and VC privileged user credential to perform certificate operations.
 Enter username [[email protected]]: 
 Enter password:

     1. Generate Certificate Signing Request(s) and Key(s) for Machine SSL certificate

     2. Import custom certificate(s) and key(s) to replace existing Machine SSL certificate

 Option [1 or 2]: 2

 Please provide valid custom certificate for Machine SSL.
 File : /tmp/vmca_issued_cer.cer

 Please provide valid custom key for Machine SSL.
 File : /tmp/vmca_issued_key.key

 Please provide the signing certificate of the Machine SSL certificate
 File : /tmp/vmca_issued_chain.cer

 You are going to replace Machine SSL cert using custom cert
 Continue operation : Option[Y/N] ? : y
 Command Output: /tmp/vmca_issued_cer.cer: OK

 Get site nameCompleted [Replacing Machine SSL Cert…]

If everything went according to plan, you should the the “Completed” statement near the bottom. Note that services will restart without warning, so make sure you are planning a short service window to complete the task.

Should an error occur the system will automatically rollback to the original certificate but also do a complete service restart, so make sure you get it right the first time or you are in for a coffee break 🙂

Resources

While researching information in order to complete this task, I mainly got information from these sites:

https://vmarena.com/replace-vcsa-6-7-certificate-vmca-by-an-adcs-signed-certificate/

https://kb.vmware.com/s/article/2112277

https://blogs.vmware.com/vsphere/2015/07/custom-certificate-on-the-outside-vmware-ca-vmca-on-the-inside-replacing-vcenter-6-0s-ssl-certificate.html