OpenSSL problem with Let's Encrypt

On September 30th, 2021 the SSL certificate DST Root CA X3, with which the certificates issued by Let’s Encrypt could be validated so far, expired. Let’s Encrypt, in turn, introduced its own root certificate (ISRG Root X1) a long time ago, which is now accepted on all reasonably up-to-date servers and devices.

For older end devices (especially old Android smartphones) can also cope with the new Let’s Encrypt certificates, a special approach was taken: the presented certificates are accompanied by a further intermediate certificate, which was signed by the DST Root CA X3, which has actually expired. Old Android devices have no problem with expired root certificates, and newer devices should find a valid (direct) validation path and thus also accept the certificate.

The idea is that every “normal” client trusts the issuer of intermediate certificate #1 (ISRG Root X1) and thus finds a valid certification path. Devices that don’t know ISRG Root X1 choose the route via the additional intermediate certificate #2 and accept it based on the signature of DST Root CA X3.

That’s the theory.

In practice, however, there are unfortunately problems in some places.

Debian/Ubuntu and c_rehash

LiveConfig stores all TLS certificates in the usual directory for the respective distribution - under Debian/Ubuntu this is /etc/ssl/certs/. Any intermediate certificates are also saved there - these end with the file name -ca.crt.

There is now a special mechanism that creates symbolic links to the certificates that are based on the fingerprint of the respective certificate. This is done under Debian/Ubuntu for all files in /etc/ssl/certs/ that end with .crt - in this case also with the chain certificates ( -ca.crt).

This leads to the creation of a symlink with the name /etc/ssl/certs/8d33f237.0 (rarely can also be .1 or .2) pointing to the Let’s Encrypt R3 intermediate certificate. If you now establish a connection with OpenSSL (as a client) to a server that presents a certificate list as shown in the table above, the following happens: instead of accepting the issuer ISRG Root X1 and thus ending the validation path as “valid”, the certificate #2 gets also checked - which was issued by an invalid (because expired) CA and is therefore invalid. Validation failed.

workaround

It is actually sufficient to remove this one symlink:

rm /etc/ssl/certs/8d33f237.?

From this point on, the establishment of outgoing SSL connections works again without any problems.

After all, this symlink is only generated when c_rehash is executed (usually via update-ca-certificates, and then apparently only if there are local changes to the CA store, e.g. by manually installed CA certificates).

LiveConfig 2.13.0

With the update to version 2.13.0, LiveConfig under Debian/Ubuntu moves all intermediate certificates from /etc/ssl/certs/*-ca.crt to /etc/ssl/chains/ and then calls c_rehash to update the hash symlinks. That’s it.