Lesson 7 of 7 in Security basics for DevOps

TLS certificates: what they prove and how they expire

Core12 min

TLS encrypts traffic between a client and a server, and a certificate is how the server proves it owns the name you connected to. A certificate names the host, is signed by an issuer your client already trusts, and is only valid between two dates.

Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.

Read these first: Shift left and think like a defender

The problem this solves

Encryption alone is not enough. If you encrypt a conversation with whoever answers, you may be talking privately to an impostor. A client needs to be sure that the server it reached is the one that owns the name in the address bar or the URL in its config.

For DevOps engineers the practical pain is expiry and mismatch. Certificates have an end date, so one that nobody renewed turns into an outage at an awkward hour, and a certificate issued for the wrong name makes every client refuse the connection. You need to read a certificate quickly and know which field to look at.

How it works

How a client decides to trust a server
How a client decides to trust a serverThe client connects, the server presents its certificate, the client checks it against its list of trusted issuers, and only then sends encrypted traffic.1 connect2 presents3 checked against4 trusted: encryptClientbrowser, curl, appServername: example.comCertificatename, issuer, datesTrusted issuersclient's CA list
  • People and clients
  • Runs code
  • Security

The client connects, the server presents its certificate, the client checks it against its list of trusted issuers, and only then sends encrypted traffic.

  1. Client to Server: 1 connect
  2. Server to Certificate: 2 presents
  3. Certificate to Trusted issuers: 3 checked against
  4. Trusted issuers to Client: 4 trusted: encrypt

A certificate is a small signed document with three things you will inspect most. The subject names, which on modern certificates live in the Subject Alternative Name (SAN) field, say which hostnames it is valid for. The issuer says who signed it. The validity dates say when it starts and stops being accepted.

Trust works by chain. Your operating system or browser ships with a list of certificate authorities (CAs) it trusts. The server's certificate is signed by an intermediate CA, which is signed by one of the trusted roots. The client walks that chain and checks each signature, the names and the dates.

A certificate you generate yourself is self-signed, so nobody's trust list contains it. That is fine for local experiments, and it is why clients refuse it until you tell them explicitly to trust that one certificate. The steps below do exactly that on your own machine.

Try it yourself

You need openssl (version 1.1.1 or newer) and curl. Everything stays on your machine except the last step, which only reads a public site's certificate. On Windows use WSL or Git Bash. In Git Bash, set MSYS_NO_PATHCONV=1 before the -subj command.

  1. 1
    Create a throwaway self-signed certificate
    mkdir -p ~/tls-demo && cd ~/tls-demo
    openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 1 \
      -subj "/CN=localhost" -addext "subjectAltName=DNS:localhost"

    This makes a private key and a certificate valid for one day for the name localhost. They are for this demo only.

  2. 2
    Read the fields that matter
    openssl x509 -in cert.pem -noout -subject -issuer -dates -ext subjectAltName

    Subject and issuer are the same, which is what self-signed means. notAfter is the expiry. The SAN list is the set of names this certificate is valid for.

  3. 3
    Check expiry the way a monitor would
    openssl x509 -in cert.pem -noout -checkend 3600; echo "exit code: $?"
    openssl x509 -in cert.pem -noout -checkend 172800; echo "exit code: $?"

    -checkend N asks whether the certificate will still be valid N seconds from now. The first call says it will not expire within an hour and exits 0. The second asks about two days, which is past this certificate's one day, so it says it will expire and exits 1. Monitoring scripts use that exit code.

  4. 4
    Serve it and watch trust fail, then succeed
    openssl s_server -accept 8443 -cert cert.pem -key key.pem -www &
    sleep 1
    curl -sS https://localhost:8443/ | head -n 3
    curl -sS --cacert cert.pem https://localhost:8443/ | head -n 3
    kill %1

    The first curl is refused because no trusted issuer signed the certificate. The second tells curl to trust exactly this file and receives a response. The error wording for the first one depends on your curl and TLS library, but it will mention the certificate or issuer not being trusted.

  5. 5
    Inspect a real site's certificate
    openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null \
      | openssl x509 -noout -subject -issuer -dates

    This only reads the public certificate. -servername sends the hostname so the server returns the right certificate. Notice that the issuer differs from the subject, and note the notAfter date. Remove the demo files with rm -r ~/tls-demo.

What breaks, and where to look

  • curl: (60) SSL certificate problem: certificate has expired or a browser warning about an expired certificate.

    WHY IT HAPPENS
    The certificate's notAfter date has passed and nobody renewed it, or an automatic renewal job failed quietly.
    CHECK FIRST
    Run openssl s_client -connect <host>:443 -servername <host> </dev/null 2>/dev/null | openssl x509 -noout -dates. Then check the renewal job's logs and add an expiry alert well before the date.
  • curl: (60) SSL certificate problem: unable to get local issuer certificate

    WHY IT HAPPENS
    The client cannot build a chain to a trusted root. The server often forgot to send its intermediate certificate, or the client's trust list is missing or old.
    CHECK FIRST
    Run openssl s_client -connect <host>:443 -servername <host> -showcerts </dev/null and count how many certificates the server sends. A leaf alone without its intermediate is the usual culprit.
  • no alternative certificate subject name matches target host name when you connect.

    WHY IT HAPPENS
    The name you used is not in the certificate's SAN list. This happens when you connect by IP address or by a different alias than the certificate was issued for.
    CHECK FIRST
    Print the names with openssl x509 -in cert.pem -noout -ext subjectAltName or inspect the live one as above, then connect using a name that is listed.

When not to use this

  • Never turn off verification (for example curl -k or verify=False) to make an error go away outside a throwaway test. It removes the one check that proves who you are talking to.
  • Self-signed certificates are fine for local learning. For anything others use, get certificates from a public CA or your organisation's internal CA, and automate renewal.

Check yourself

What three things on a certificate do you check first?

The names it is valid for (SAN), the issuer that signed it, and the start and end dates.

Why does a client reject a self-signed certificate?

No certificate authority in the client's trust list signed it, so the client cannot confirm who issued it.

What does openssl x509 -checkend 3600 tell you?

Whether the certificate will still be valid in 3600 seconds. Exit code 0 means yes and 1 means it will have expired.

Lesson 92 of 92 in the DevOps path

Search

Search cheat sheets and projects