Helm Values and Overrides#

Every Helm chart has a values.yaml file that defines defaults. When you install or upgrade a release, you override those defaults through values files (-f) and inline flags (--set). Getting the precedence wrong leads to silent misconfigurations where you think you set something but the chart used a different value.

Inspecting Chart Defaults#

Before overriding anything, look at what the chart provides. helm show values dumps the full default values.yaml for any chart:

Image Patching and Lifecycle#

Building a container image and deploying it is the easy part. Keeping it patched over weeks, months, and years is where most teams fail. A container image deployed today with zero known vulnerabilities will accumulate CVEs as new vulnerabilities are disclosed against its OS packages, language runtime, and dependencies. You need an automated system that detects stale base images, triggers rebuilds, and rolls out updates safely.

Ingress Controllers and Routing Patterns#

An Ingress resource defines HTTP routing rules – which hostnames and paths map to which backend Services. But an Ingress resource does nothing on its own. You need an Ingress controller running in the cluster to watch for Ingress resources and configure the actual reverse proxy.

Ingress Controllers#

The two most common controllers are nginx-ingress and Traefik.

nginx-ingress (ingress-nginx):

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace

Note: there are two different nginx ingress projects. kubernetes/ingress-nginx (community) and nginxinc/kubernetes-ingress (NGINX Inc). The community version is far more common. Make sure you install from https://kubernetes.github.io/ingress-nginx, not the NGINX Inc chart.

Istio Service Mesh#

Istio adds a proxy sidecar (Envoy) to every pod in the mesh. These proxies handle traffic routing, mutual TLS, retries, circuit breaking, and telemetry without changing application code. The control plane (istiod) pushes configuration to all sidecars.

When You Actually Need a Service Mesh#

You need Istio when you have multiple services requiring mTLS, fine-grained traffic control (canary releases, fault injection), or consistent observability across service-to-service communication. If you have fewer than five services, standard Kubernetes Services and NetworkPolicies are sufficient. A service mesh adds operational complexity – more moving parts, higher memory usage per sidecar, and a learning curve for proxy-level debugging.

kind Validation Templates#

kind (Kubernetes IN Docker) runs Kubernetes clusters using Docker containers as nodes. It was designed for testing Kubernetes itself, which makes it an excellent tool for validating infrastructure changes. It starts fast, uses fewer resources than minikube, and is disposable by design.

This article provides copy-paste cluster configurations and complete lifecycle scripts for common validation scenarios.

Cluster Configuration Templates#

Basic Single-Node#

The simplest configuration. One container acts as both control plane and worker. Sufficient for validating that deployments, services, ConfigMaps, and Secrets work correctly.

kubectl Debugging#

When something breaks in Kubernetes, you need to move through a specific sequence of commands. Here is every debugging command you will reach for, plus a step-by-step workflow for a pod that will not start.

Logs#

kubectl logs <pod-name> -n <namespace>                           # basic
kubectl logs <pod-name> -c <container-name> -n <namespace>       # specific container
kubectl logs <pod-name> --previous -n <namespace>                # previous crash (essential for CrashLoopBackOff)
kubectl logs -f <pod-name> -n <namespace>                        # stream in real-time
kubectl logs --since=5m <pod-name> -n <namespace>                # last 5 minutes
kubectl logs -l app=payments-api -n payments-prod --all-containers  # all pods matching label

The --previous flag is critical for crash-looping pods where the current container has no logs yet. The --all-containers flag captures init containers and sidecars.

Kubernetes API Deprecation Guide#

Kubernetes deprecates and removes API versions on a predictable schedule. When an API version is removed, any manifests or Helm charts using the old version will fail to apply on the upgraded cluster. Workloads already running are not affected – they continue to run – but you cannot create, update, or redeploy them until the manifests are updated. This guide walks through the complete workflow for detecting and fixing deprecated APIs before an upgrade.

Kubernetes API Server: Architecture, Authentication, Authorization, and Debugging#

The API server (kube-apiserver) is the front door to your Kubernetes cluster. Every interaction – kubectl commands, controller reconciliation loops, kubelet status updates, admission webhooks – goes through the API server. It is the only component that reads from and writes to etcd. If the API server is down, the cluster is unmanageable. Everything else (scheduler, controllers, kubelets) can tolerate brief API server outages because they cache state locally, but no mutations happen until the API server is back.

Kubernetes Audit Logging: Tracking API Activity for Security and Compliance#

Audit logging records every request to the Kubernetes API server. Every kubectl command, every controller reconciliation, every kubelet heartbeat, every admission webhook call – all of it can be captured with the requester’s identity, the target resource, the timestamp, and optionally the full request and response bodies. Without audit logging, you have no record of who did what in your cluster. With it, you can trace security incidents, satisfy compliance requirements, and debug access control issues.

Kubernetes Controllers: Reconciliation Loops, the Controller Manager, and Custom Controllers#

Kubernetes is a declarative system. You tell it what you want (a Deployment with 3 replicas), and controllers make it happen. Controllers are the engines that continuously reconcile desired state with actual state. Without controllers, your YAML manifests would be inert data in etcd.

The Controller Pattern#

Every controller follows the same loop:

1. Watch the API server for changes to a specific resource type
2. For each change, compare desired state (spec) to actual state (status)
3. Take action to bring actual state closer to desired state
4. Update status to reflect current actual state
5. Repeat

This is a level-triggered model, not edge-triggered. A controller does not just react to changes – it reconciles the entire state on each pass. If a controller crashes and restarts, it re-reads all objects and converges to the correct state without needing to replay missed events. This makes controllers resilient to transient failures.