DNS Failover Patterns: TTL Tradeoffs, Health Check Design, and Real-World Failover Timing

DNS Is Not a Load Balancer#

This needs to be said upfront: DNS was designed for name resolution, not traffic management. Using DNS for failover is a pragmatic hack that works well enough for most use cases, but it has fundamental limitations.

DNS responses are cached at multiple levels (recursive resolvers, OS caches, application caches, browser caches). You cannot force a client to re-resolve. You can set a TTL, but clients and resolvers are free to ignore it (and some do). Java applications, for example, cache DNS indefinitely by default in some JVM versions unless you explicitly set networkaddress.cache.ttl.

DNS Deep Dive: Record Types, Resolution, Troubleshooting, and Cloud DNS Management

How DNS Resolution Works#

When a client requests api.example.com, the resolution follows a chain of queries. The client asks its configured recursive resolver (often the ISP’s, or a public one like 8.8.8.8). The recursive resolver does the heavy lifting: it asks a root name server for .com, the .com TLD server for example.com, and the authoritative name server for example.com returns the answer for api.example.com. Each level caches the result according to the record’s TTL, so subsequent requests short-circuit the chain.

Jobs and CronJobs: Batch Workloads, Retry Logic, and Scheduling

Jobs and CronJobs#

Deployments manage long-running processes. Jobs manage work that finishes. A Job creates one or more pods, runs them to completion, and tracks whether they succeeded. CronJobs run Jobs on a schedule. Both are essential for database migrations, report generation, data pipelines, and any workload that is not a continuously running server.

Job Basics#

A Job runs a pod until it exits successfully (exit code 0). The simplest case is a single pod that runs once: