Certificate Basics#

A TLS certificate binds a public key to a domain name. The certificate is signed by a Certificate Authority (CA) that browsers and operating systems trust. The chain goes: your certificate, signed by an intermediate CA, signed by a root CA. All three must be present and valid for a client to trust the connection.

Self-Signed Certificates for Development#

For local development and testing, generate a self-signed certificate. Clients will not trust it by default, but you can add it to your local trust store.

PKI Fundamentals#

A Public Key Infrastructure (PKI) is a hierarchy of trust. At the top sits the Root CA, a certificate authority that signs its own certificate and is explicitly trusted by all participants. Below it are Intermediate CAs, signed by the root, which handle day-to-day certificate issuance. At the bottom are leaf certificates, the actual certificates used by servers, clients, and workloads.

Root CA (self-signed, offline, 10-20 year validity)
  |
  +-- Intermediate CA (signed by root, online, 3-5 year validity)
        |
        +-- Leaf Certificate (signed by intermediate, 90 days or less)
        +-- Leaf Certificate
        +-- Leaf Certificate

Never use the root CA directly to sign leaf certificates. If the root CA’s private key is compromised, the entire PKI must be rebuilt from scratch. Keeping it offline and behind an intermediate CA limits the blast radius. If an intermediate CA is compromised, you revoke it and issue a new one from the root – leaf certificates from other intermediates remain valid.

Kubernetes Disaster Recovery Runbooks#

These runbooks cover the incidents you will encounter in production Kubernetes environments. Each follows the same structure: detection, diagnosis, recovery, and prevention. Print these out, bookmark them, put them in your on-call wiki. When the alert fires at 2 AM, you want a checklist, not a tutorial.

Incident Response Framework#

Every incident follows the same cycle:

1. Detect – monitoring alert, user report, or kubectl showing unhealthy state
2. Assess – determine scope and severity. Is it one pod, one node, or the entire cluster?
3. Contain – stop the bleeding. Prevent the issue from spreading
4. Recover – restore normal operation
5. Post-mortem – document what happened, why, and how to prevent it

Runbook 1: Node Goes NotReady#

Detection: Node condition changes to Ready=False. Pods on the node are rescheduled (if using Deployments). Monitoring alerts on node status.

The TLS Handshake#

Every HTTPS connection starts with a TLS handshake that establishes encryption parameters and verifies the server’s identity. The simplified flow for TLS 1.2:

Client                              Server
  |── ClientHello ──────────────────>|   (supported versions, cipher suites, random)
  |<────────────────── ServerHello ──|   (chosen version, cipher suite, random)
  |<──────────────── Certificate  ──|   (server's certificate chain)
  |<───────────── ServerKeyExchange ─|   (key exchange parameters)
  |<───────────── ServerHelloDone  ──|
  |── ClientKeyExchange ───────────>|   (client's key exchange contribution)
  |── ChangeCipherSpec ────────────>|   (switching to encrypted communication)
  |── Finished ────────────────────>|   (encrypted verification)
  |<──────────── ChangeCipherSpec ──|
  |<──────────────────── Finished ──|
  |<═══════ Encrypted traffic ═════>|

TLS 1.3 simplifies this significantly. The client sends its key share in the ClientHello, allowing the handshake to complete in a single round trip. TLS 1.3 also removed insecure cipher suites and compression, eliminating entire classes of vulnerabilities (BEAST, CRIME, POODLE).