Server Certificate could not be updated: Private key does not match

In this article, we will discuss how to troubleshoot “Server Certificate could not be updated: Private key does not match”, and and determine the root case of this issue. This concern arose during the process of uploading a new SSL/TLS certificate to Trellix ePolicy Orchestrator (ePO) for securing web-based access to the console over the internet.” Please, see how to Fix Error 0x87E10BC6 on a PC running Windows non-core Edition, and What to know about “Failed to perform Veeam Worker Upgrade”.

We will be employing OpenSSL as it provides a powerful way to convert SSL certificates into different file formats to meet the requirements of various servers and control panels. Since each system may demand a specific format, administrators and developers rely on OpenSSL to seamlessly transform certificates for compatibility.

By using OpenSSL’s built-in commands, we can quickly generate the necessary certificate types, ensuring secure and efficient deployment. In fact the certificates we provide are issued in x.509 format which is the version of ASN.1

Note: OpenSSL package comes pre-installed with macOS and most Linux distributions but not for Windows. Therefore, you will need to install it for Windows OSes.

Here are some related articles; how to Export and Convert Private Keys to .PEM Format in Windows, and How to Install OpenSSL on Windows Computers. Also, see How to export a certificate in PFX format in Windows.

Error “Server Certificate could not be updated: Provided Private key does not match”

Having troubleshooted this issue, I can confirm that the problem is related to the certificate issued by the Harica root CA, which Trellix does not accept. Here is a similar issue and as such, I will recommend escalating this to Trellix.

Note: A Certificate Authority (CA) root certificate is the highest-level certificate issued by a Certificate Authority. These authorities are entities that issue digital certificates, which confirm the identity of users, systems, and organizations online. A root certificate serves as the anchor of trust in a public key infrastructure (PKI).

Every certificate chain is anchored by a root certificate that ultimately authenticates the end-entity certificate used in secure communications. When a root certificate is not trusted, it raises risks to security, privacy, and vulnerability to cyberattacks. I can assure you that, non of the reasons mentioned here is the cause for distrust.

Please, see how to “Create a self-signed certificate and export it in PFX format via PowerShell [Part 1], and how to Generate self-signed certificate and export in PFX format via PowerShell [Part 2].

Verify Product Certificate

To verify the certificate on Trellix ePO. Log in to the ePO console as an administrator, and navigate to Menu, Configuration, and Certificate Manager.

Tip Only: The Certificate Manager is another part of the basic server configuration. The latest certificates are loaded by default with a fresh installation of ePO. But if you have upgraded from an older version of ePO. Ensure you migrate the ePO certificates to the latest hash algorithm.

Reproducing the error: Upload/Install new certificate on Trellix ePO

All the necessary settings specific to your ePO server are in Server Settings. Please, see ePO Server Settings: Trellix ePO AD integration and ENS Agents Installation. The available entries here allow you to fine-tune you ePO server based on the needs of your organization. Customizations made here affect all your ePO users.

Log in to the ePO console as an administrator, and navigate to Menu, Configuration, Server Settings

From the Setting Categories list, select Server Certificate, then click Edit. Browse to the server certificate file, then click Open. Browse to the private key file, then click Open. If needed, type the private key password, then click Save.

Replace the default or expired ePO server certificate with a valid certificate that has been signed by a certificate authority (CA). This is the best option, because the certificate is signed by a trusted CA.

To do this, choose Actions and Update Certificate. Next, browse and upload your files:

• Server Certificate (.cer/.crt/.pem)
• Private Key (.pem)
• Enter the password if the key is protected.

The error will be prompted as as shown below

Note: If the server certificate or private key are not in these formats mentioned above or specified in Trellix. They must be converted to one of the supported formats before they can be used to replace the server certificate.

Because of the error prompted, we will need to verify the mismatch. Therefore, follow through to learn these steps.

Please, see how to change Active Directory Domain name from dot local to dot com, and how to setup a Domain Controller as Recommended by Microsoft.

Convert a PFX file into a CER file and extract the private key in PEM format

Since HARICA provides certificates in various formats and most times not included with the private key. You may need to convert them to match your specific requirements. This process involves using OpenSSL commands. Before conversion, however, you must first export the certificate and its corresponding private key from the certificate store as a PFX file.

To do this, launch the MMC (Microsoft Management Console) by typing mmc in the Run dialog (Win + R). Go to File > Add/Remove Snap-in and select Certificates. Choose My User Account, Computer Account, or Service Account depending on where your certificate resides. Please, see How to export a certificate in PFX format in Windows.

Since we have discussed these steps previously in the above and below articles. We will not be sharing the steps (images) in this guide. Also, see how to “Export and Convert Private Keys to .PEM Format in Windows“. Also, see how to generate CSR and Request a Certificate from Herica CA.

Note: It is possible for a .pfx file to be exported without a private key. This happens when the private key is not included during the export process. The private key is only included if it is specifically selected during the export. When exporting a certificate to a .pfx file (e.g., through Windows Certificate Store or another tool).

You will be prompted with an option like “Export the private key.” If this option is not selected, the resulting .pfx file will only contain the public certificate and not the private key. If the original private key is password-protected and the correct password isn’t provided during the export process. The private key won’t be included in the exported .pfx file.

Here are some guides on Install Let’s Encrypt Wildcard SSL Certificate in cPanel using Certify The Web, and how to Import certificates into Trusted Root and Personal certificate store. Also, see Tomcat Keystore Update: How to rebind Tomcat Keystore with a new Certificate.

Managing certificates and key with OpenSSL

After installing OpenSSL, we converted the PFX file into a CER certificate and extracted the private key in PEM format as outlined in the referenced guide above. With OpenSSL now available, we can efficiently manage and administer certificates and keys as needed.

Despite discussing what OpenSSL is above. One of OpenSSL’s key functionalities is handling Transport Layer Security (TLS) certificates. TLS certificates play a crucial role in encrypting and securing online communications as discussed above previously. This ensures that data exchanged between servers and clients remains private and authenticated.

Depending on the system or application requirements, TLS certificates often need to be converted into different formats. OpenSSL offers the flexibility to transform certificates to match specific needs, including converting between PEM, DER, PKCS#7, and PKCS#12 formats.

When you type openssl in the command-line interface (CLI). You are dropped into the OpenSSL interactive shell. From there, OpenSSL will display a list of all supported standard command as shown belows.

Common Certificate Format Conversions Using OpenSSL

When working with TLS certificates, you may need to convert them into a compatible format for different servers, devices, or applications as discussed above.

Inspect the contents of an exported .pfx file

Inspect the contents of a .pfx file (also known as a PKCS#12 file) and display detailed information about its components, such as certificates, private keys, and additional metadata.

openssl pkcs12 -info -nodes -in exported.pfx

The -nodes option means “no DES encryption,” which prevents OpenSSL from encrypting the private key when extracting it. As a result, this command displays the private key in plaintext along with other details.

Please, see how to generate CSR and Request a Certificate from Herica CA, how to encrypt Microsoft SQL Server Traffic, and “Connecting to the RDP host: Fix the Certificate could not be verified back to the root certificate“.

Verify the PFX File Contains the Private Key for Trellix Validation

The following command can be used to check if the .pfx file actually contains a private key

openssl pkcs12 -in input.pfx -info -noout

Note: The -noout option tells OpenSSL not to output certain information, like the private key or certificates. This command only prints metadata about the .pfx file, without exposing the private key.

The -noout flag prevents OpenSSL from dumping the actual private key or certificate content into the output. If you need to specifically extract the private key, you must omit -noout and use -nodes or explicitly extract it using the command below. This ensures that only the private key is extracted.

openssl pkcs12 -in exported.pfx -nocerts -out private_key.pem -nodes

Here are some interesting blogposts: Setup VirtualHost with SSL on WAMP Server, how to Join Bulk Devices using a Provisioning Package to Azure, and how to upgrade Veeam ONE to v12.1.

Option 1: Extract the certificate (.cer) file

The PFX (PKCS#12) file is a bundled file that contains the certificate (publick key), the private key and optionally the CA chain most times. You extract the .cer file to get the public certificate only, which is safe to share and often required by servers, clients, or third-party systems while keeping the private key secure and separate which is required by many applications for installation, validation, or interoperability while keeping the private key secure.

To do this, replace “input.pfx” with the name of your PFX file, and certificate.cer will be the output file containing the certificate.

openssl pkcs12 -in input.pfx -clcerts -nokeys -out certificate.cer

Note: If you need both the certificate in .crt format and the private key in .pem format. The command below will extract the certificate from the PFX file and save it in .crt format.

openssl pkcs12 -in input.pfx -clcerts -nokeys -out certificate.crt

Option 2: Extract the private key in PEM format (.pem)

Also, to extract the private key in .pem format, run this command to extract the private key. The following command will extract the private key from the .pfx file. A new file priv-key.pem will be generated in the current directory. This command will prompt a password set on the pfx file.

To do this, extract the private key from the PFX file and save it in PEM format.

openssl pkcs12 -in input.pfx -nocerts -out private_key.pem -nodes

Please, see How to Create a User and Custom Domain in Entra ID, how to Delete Azure Tenant: Remove Custom Domain from Entra ID, and how to Fix missing path and delete a Veeam Backup Repository.

Validate the Private Key

If the key is valid, you should see “RSA key ok” in the output. If there’s an issue, OpenSSL will report an error.

openssl rsa -in private_key.pem -check

PowerShell to list All Certificates with Private Keys

This will list every certificate with a private key and show you exactly where it is stored

Get-ChildItem -Recurse Cert:\ | Where-Object { $_.HasPrivateKey -eq $true } | Select-Object Subject, Issuer, PSPath

Open the Certificate Store (“certlm.msc for Local Machine” and “certmgr.msc for launching Current User”) to view more details of a certificate.

Both certlm.msc and certmgr.msc are Microsoft Management Console (MMC) snap-ins for managing certificates; the difference is that certlm.msc manages certificates for the Local Computer (system-wide), while certmgr.msc manages certificates for the Current User.

Verify Mismatch Between Certificate and Private Key

Due to the error we are having, we will indeed verify if the uploaded private key file indeed contains a valid private key as shown in the error message below.

If the certificate and private key don’t match (e.g., the private key is from a different certificate). The export process won’t include the private key. Verify that the certificate and private key are part of the same pair. You can do this by comparing their modulus values using openssl:

As you can see, they match. If the MD5 hashes do not match. The private key is not correctly paired with the certificate

Note: By default, the conversion steps above with openssl (extracting .crt, .cer, and .pem files) produce PEM-encoded certificates and keys. If you wish to generate a private key and csr and to convert a pem file to p12 which can be renamed to .pfx

As you can see, you need to use a different certificate authority as there are known problems with Harica CA. It will interest you to know that this CA is listed in the CCADB list for Microsoft. yet, it is not trusted by some enterprises even when it is listed in the trusted root certificate store. From my experience, this works for some applications and not all for example Trellix ePO and SAP Hana etc at the time of writing this article.

I am not sure of the reason for the Cross-Certificate Transition Confusion. HARICA transitioned from its 2015 Root CA to the 2021 Root CA, issuing cross-certificates to maintain compatibility. this transition created confusion and trust issues for systems relying on older root hierarchies, especially if the cross-certificates weren’t properly included or recognized.

Finally, since HARICA is one of the few CAs that issues certificates for .onion domains, which are used in privacy-focused or Tor-based services. This niche focus may not align with the needs of mainstream enterprise environments, leading to lower adoption.

Therefore, my final verdict would be to contact your application support or employ a new CA such as Sectigo etc.

FAQs

I hope you fund this article very useful on how to troubleshoot Server Certificate could not be updated: Private key does not match. Please, feel free to leave a comment below.