Sandbox to Production#

An agent that produces infrastructure deliverables works in a sandbox. It does not touch production. It does not reach into someone else’s cluster, database, or cloud account. It works in an isolated environment, tests its work, captures the results, and hands the human a verified deliverable they can execute on their own infrastructure.

This is not a limitation – it is a design choice. The output is always a deliverable, never a direct action on someone else’s systems. This boundary is what makes the approach safe enough for production infrastructure work and trustworthy enough for enterprise change management.

Securing Docker-Based Validation Templates#

Validation templates define the environment agents use to test infrastructure changes. If a template runs containers as root, mounts the Docker socket, or skips resource limits, every agent that copies it inherits those risks. This reference covers the security patterns every docker-compose validation template must follow.

1. Non-Root Execution#

Containers run as root by default. A vulnerability in a root-running process gives an attacker full control inside the container and a much larger attack surface for container escapes.

What Backstage Provides#

Backstage is an open-source developer portal originally built by Spotify, now a CNCF Incubating project. It serves as the single UI layer for an internal developer platform, unifying the service catalog, documentation, scaffolding templates, and plugin-based integrations behind one interface. It does not replace your tools — it provides a consistent frontend for discovering and interacting with them.

The core components:

• Software Catalog: A registry of all services, libraries, APIs, and infrastructure components, populated from YAML descriptor files in your repositories.
• TechDocs: Documentation-as-code powered by MkDocs, rendered directly in the Backstage UI alongside the service it describes.
• Scaffolder: A template engine that creates new projects from predefined templates — repositories, CI pipelines, Kubernetes manifests, and all.
• Plugins: Backstage’s extension mechanism. The community provides plugins for Kubernetes, ArgoCD, PagerDuty, GitHub Actions, Terraform, and hundreds of other tools.

Installation#

Backstage requires Node.js 18+ and Yarn. Create a new Backstage app:

Tekton Pipelines#

Tekton is a Kubernetes-native CI/CD framework. Every pipeline concept – tasks, runs, triggers – is a Kubernetes Custom Resource. Pipelines execute as pods. There is no central server, no UI-driven configuration, no special runtime. If you know Kubernetes, you know how to operate Tekton.

Core Concepts#

Tekton has four primary resources:

• Task: A sequence of steps that run in a single pod. Each step is a container.
• TaskRun: An instantiation of a Task with specific inputs. Creating a TaskRun executes the Task.
• Pipeline: An ordered collection of Tasks with dependencies, parameter passing, and conditional execution.
• PipelineRun: An instantiation of a Pipeline. Creating a PipelineRun executes the entire pipeline.

The separation between definition (Task/Pipeline) and execution (TaskRun/PipelineRun) means you define your CI/CD process once and trigger it many times with different inputs.

Template Contribution Guide#

Agent Zone validation templates are reusable infrastructure configurations that agents and developers use to validate changes. A Kubernetes devcontainer template, an ephemeral EKS cluster module, a static validation pipeline script – each follows a standard format so that any agent or developer can pick one up, understand its purpose, and use it without reading through implementation details.

This guide defines the standards for contributing templates. It covers directory structure, required files, testing, quality expectations, versioning, and the submission process.

Validation Path Selection#

Not every infrastructure change needs a full Kubernetes cluster to validate. Some changes can be verified with a linter in under a second. Others genuinely need a multi-node cluster with ingress, persistent volumes, and network policies. The cost of choosing wrong is real in both directions: too little validation lets broken configs reach production, while too much wastes minutes or hours on environments you did not need.

Validation Playbook Format#

A validation playbook is a structured procedure that tells an agent exactly how to validate a specific type of infrastructure change. The key problem it solves: the same validation (for example, “verify this Helm chart works”) requires different commands depending on whether the agent has access to kind, minikube, a cloud cluster, or nothing but a linter. A playbook encodes all path variants in one document so the agent picks the right commands for its environment.

ARM64 Kubernetes: The QEMU Problem with Go Binaries#

If you run Kubernetes on Apple Silicon (M1/M2/M3/M4) via minikube with the Docker driver, you will eventually try to run an amd64-only container image. For most software this works through QEMU emulation. For Go binaries, it crashes hard.

The Problem#

Go’s garbage collector uses a lock-free stack (lfstack) that packs pointers with counter bits in the high bits of a 64-bit integer. QEMU’s user-mode address translation changes the effective address space layout, which breaks this packing assumption. The result:

Container Build Optimization#

A container build that takes eight minutes in CI is not just slow – it compounds across every push, every developer, every day. The difference between a naive Dockerfile and an optimized one is often the difference between a two-minute build and a twelve-minute build. The techniques here are not theoretical. They are the specific changes that eliminate wasted time.

BuildKit Over Legacy Builder#

BuildKit is the modern Docker build engine and the default since Docker 23.0. If you are running an older version, enable it explicitly with DOCKER_BUILDKIT=1. BuildKit provides several capabilities the legacy builder lacks.

Minikube Setup, Drivers, and Resource Configuration#

Minikube runs a single-node Kubernetes cluster on your local machine. The difference between a minikube setup that feels like a toy and one that behaves like production comes down to three choices: the driver, the resource allocation, and the Kubernetes version. Get these wrong and you spend more time fighting the tool than using it.

Installation#

On macOS with Homebrew:

brew install minikube

On Linux via direct download: