Agent Evaluation and Testing#

You cannot improve what you cannot measure. Agent evaluation is harder than traditional software testing because agents are non-deterministic, their behavior depends on prompt wording, and the same input can produce multiple valid outputs. But “it is hard” is not an excuse for not doing it. This article provides a step-by-step framework for building an agent evaluation pipeline that catches regressions, compares configurations, and quantifies real-world performance.

Feature Flags: Decoupling Deployment from Release#

Deployment and release are not the same thing. Deployment is shipping code to production. Release is enabling that code for users. Feature flags make this separation explicit. You deploy code that sits behind a conditional check, and you control when and for whom that code activates – independently of when it was deployed.

This distinction changes how teams work. Developers merge unfinished features to main because the code is behind a flag and invisible to users. A broken feature can be disabled in seconds without a rollback deploy. New features roll out to 1% of users, then 10%, then 50%, then 100%, with a kill switch available at every stage.