Choosing a Secret Management Strategy#

Secrets – database credentials, API keys, TLS certificates, encryption keys – must be available to pods at runtime. At the same time, they must not be stored in plain text in git, should be rotatable without downtime, and should produce an audit trail showing who accessed what and when. No single tool satisfies every requirement, and the right choice depends on your security maturity, operational capacity, and compliance obligations.

The Problem with Environment Variables#

Environment variables are the most common way to pass secrets to applications. Every framework supports them and they require zero dependencies. They are also the least secure option. Any process running as the same user can read them via /proc/<pid>/environ on Linux. Crash dumps include the full environment. Child processes inherit all variables by default.

# Anyone with host access can read another process's environment
cat /proc/$(pgrep myapp)/environ | tr '\0' '\n' | grep DB_PASSWORD

Environment variables are acceptable for local development. For production secrets, use one of the patterns below.

GitOps for Kubernetes#

GitOps is a deployment model where git is the source of truth for your cluster’s desired state. A controller running inside the cluster watches a git repository and continuously reconciles the live state to match what is declared in git. When you want to change something, you commit to git. The controller detects the change and applies it.

This replaces kubectl apply from laptops and CI pipelines with a pull-based model where the cluster pulls its own configuration. The benefits are an audit trail in git history, easy rollback via git revert, and drift detection when someone makes manual changes.