Why Production Parity Matters Locally#

The most expensive bugs are the ones you find after deploying to production. A minikube cluster with default settings lacks ingress, metrics, and resource enforcement – so your app works locally and breaks in staging. The goal is to configure minikube so that anything that works on it has a high probability of working on a real cluster.

Choosing Your Local Kubernetes Tool#

Before configuring minikube, decide if it is the right tool.

Why Monitor a POC Cluster#

Monitoring on minikube serves two purposes. First, it catches resource problems early – your app might work in tests but OOM-kill under load, and you will not know without metrics. Second, it validates that your monitoring configuration works before you deploy it to production. If your ServiceMonitors, dashboards, and alert rules work on minikube, they will work on EKS or GKE.

The Right Chart: kube-prometheus-stack#

There are multiple Prometheus-related Helm charts. Use the right one:

Why GitOps on a POC Cluster#

Setting up ArgoCD on minikube is not about automating deployments for a local cluster – you could just run kubectl apply. The point is to prove the deployment workflow before production. If your Git repo structure, Helm values, and sync policies work on minikube, they will work on EKS or GKE. If you skip this and bolt on GitOps later, you will spend days restructuring your repo and debugging sync failures under production pressure.

Choosing the Right Workload Type#

Every application fits one of four deployment patterns. Choosing the wrong one creates problems that are hard to fix later – a database deployed as a Deployment loses data on reschedule, a batch job deployed as a Deployment wastes resources running 24/7.

Pattern 1: Stateless Web App#

A web API that can be scaled horizontally with no persistent state. Any pod can handle any request.

The Secret Zero Problem#

Every secrets management system has the same fundamental challenge: you need a secret to access your secrets. Your Vault token is itself a secret. Your AWS credentials for SSM Parameter Store are themselves secrets. This is the “secret zero” problem – there is always one secret that must be bootstrapped outside the system.

Understanding this helps you make pragmatic choices. No tool eliminates all risk. The goal is to reduce the blast radius and make rotation possible.

Docker-in-Docker on Jenkins: Postgres Tests Can’t Reach localhost#

A Jenkins job runs docker run -d -p 5432:5432 postgres:17-alpine and gets back a container ID. The next step is psql -h localhost -p 5432 -U postgres and it returns Connection refused. The retry loop tries 30 times and gives up. The test job fails with “could not connect to server”.

If you’ve added longer waits, switched to --network host, or rewritten the test script to launch its own postgres container, none of that will help. The problem is the network model: Jenkins running in a Kubernetes pod uses the host’s docker socket to launch SIBLING containers. Those siblings live on the host’s docker bridge network, not in Jenkins’s pod network namespace. localhost from inside Jenkins is the pod’s loopback; the published port is on the host’s interface.

CircleCI Pipeline Patterns#

CircleCI pipelines are defined in .circleci/config.yml. The configuration model uses workflows to orchestrate jobs, jobs to define execution units, and steps to define commands within a job. Every job runs inside an executor – a Docker container, Linux VM, macOS VM, or Windows VM.

Config Structure and Executors#

A minimal config defines a job and a workflow:

version: 2.1

executors:
  go-executor:
    docker:
      - image: cimg/go:1.22
    resource_class: medium
    working_directory: ~/project

jobs:
  build:
    executor: go-executor
    steps:
      - checkout
      - run:
          name: Build application
          command: go build -o myapp ./cmd/myapp

workflows:
  main:
    jobs:
      - build

Named executors let you reuse environment definitions across jobs. The resource_class controls CPU and memory – small (1 vCPU/2GB), medium (2 vCPU/4GB), large (4 vCPU/8GB), xlarge (8 vCPU/16GB). Choose the smallest class that avoids OOM kills to keep costs down.

Running Temporal Server on Minikube#

This guide deploys Temporal Server on a local Minikube cluster with PostgreSQL persistence. By the end, you will have the Temporal frontend, Web UI, and CLI all working against a real Kubernetes deployment.

If you need background on what Temporal is, start with Introduction to Temporal.

Prerequisites#

Your machine needs at least 4 CPU cores and 8 GB RAM available to Docker. For minikube driver details, see Minikube Setup and Drivers and Minikube Docker Driver.

Temporal High Availability#

A single-replica Temporal deployment works for development, but any pod going down takes the workflow engine offline. This guide configures a multi-replica cluster with proper resource allocation, Elasticsearch visibility, and health monitoring.

For the single-replica setup this builds on, see Running Temporal Server on Minikube.

Why HA Matters#

With multiple replicas, losing a pod triggers a brief rebalance (seconds), not an outage.

Azure DevOps Pipelines#

Azure DevOps Pipelines uses YAML files stored in your repository to define build and deployment workflows. The pipeline model has three levels: stages contain jobs, jobs contain steps. This hierarchy maps directly to how you think about CI/CD – build stage, test stage, deploy-to-staging stage, deploy-to-production stage – with each stage containing one or more parallel jobs.

Pipeline Structure#

A complete pipeline in azure-pipelines.yml:

trigger:
  branches:
    include:
      - main
      - release/*
  paths:
    exclude:
      - docs/**
      - README.md

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: common-vars
  - name: buildConfiguration
    value: 'Release'

stages:
  - stage: Build
    jobs:
      - job: BuildApp
        steps:
          - task: GoTool@0
            inputs:
              version: '1.22'
          - script: |
              go build -o $(Build.ArtifactStagingDirectory)/myapp ./cmd/myapp
            displayName: 'Build binary'
          - publish: $(Build.ArtifactStagingDirectory)
            artifact: drop

  - stage: Test
    dependsOn: Build
    jobs:
      - job: UnitTests
        steps:
          - task: GoTool@0
            inputs:
              version: '1.22'
          - script: go test ./... -v -coverprofile=coverage.out
            displayName: 'Run tests'
          - task: PublishCodeCoverageResults@2
            inputs:
              summaryFileLocation: coverage.out
              codecoverageTool: 'Cobertura'

  - stage: DeployStaging
    dependsOn: Test
    condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
    jobs:
      - deployment: DeployToStaging
        environment: staging
        strategy:
          runOnce:
            deploy:
              steps:
                - download: current
                  artifact: drop
                - script: echo "Deploying to staging"

trigger controls which branches and paths trigger the pipeline. dependsOn creates stage ordering. condition adds logic – succeeded() checks the previous stage passed, and you can combine it with variable checks to restrict certain stages to specific branches.