Choosing a CNI Plugin: Calico vs Cilium vs Flannel vs Cloud-Native CNI

Choosing a CNI Plugin#

The Container Network Interface (CNI) plugin is one of the most consequential infrastructure decisions in a Kubernetes cluster. It determines how pods get IP addresses, how traffic flows between them, whether network policies are enforced, and what observability you get into network behavior. Changing CNI after deployment is painful – it typically requires draining and rebuilding nodes, or rebuilding the cluster entirely. Choose carefully up front.

Network Policies: Namespace Isolation and Pod-to-Pod Rules

Network Policies: Namespace Isolation and Pod-to-Pod Rules#

By default, every pod in a Kubernetes cluster can talk to every other pod. Network policies let you restrict that. They are namespace-scoped resources that select pods by label and define allowed ingress and egress rules.

Critical Prerequisite: CNI Support#

Network policies are only enforced if your CNI plugin supports them. Calico, Cilium, and Weave all support network policies. Flannel does not. If you are running Flannel, you can create NetworkPolicy resources without errors, but they will have absolutely no effect. This is a silent failure that wastes hours of debugging.

Network Security Layers

Defense in Depth#

No single network control stops every attack. Layer controls so that a failure in one does not compromise the system: host firewalls, Kubernetes network policies, service mesh encryption, API gateway authentication, and DNS security, each operating independently.

Host Firewall: iptables and nftables#

Every node should run a host firewall regardless of the orchestrator. Block everything by default:

# iptables: default deny with essential allows
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH from management network only
iptables -A INPUT -p tcp --dport 22 -s 10.0.100.0/24 -j ACCEPT

# Allow kubelet API (for k8s nodes)
iptables -A INPUT -p tcp --dport 10250 -s 10.0.0.0/16 -j ACCEPT

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT

The nftables equivalent is more readable for complex rulesets: