gRPC Security#

gRPC uses HTTP/2 as its transport, which means TLS is not just a security feature — it is a practical necessity. Many load balancers, proxies, and clients expect HTTP/2 over TLS (h2) rather than plaintext HTTP/2 (h2c). Securing gRPC means configuring TLS correctly, authenticating clients, authorizing RPCs, and handling the gRPC-specific gotchas that do not exist with REST APIs.

gRPC Over TLS#

Server-Side TLS in Go#

import (
    "crypto/tls"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials"
)

func main() {
    cert, err := tls.LoadX509KeyPair("server-cert.pem", "server-key.pem")
    if err != nil {
        log.Fatal(err)
    }

    tlsConfig := &tls.Config{
        Certificates: []tls.Certificate{cert},
        MinVersion:   tls.VersionTLS13,
    }

    creds := credentials.NewTLS(tlsConfig)
    server := grpc.NewServer(grpc.Creds(creds))

    pb.RegisterMyServiceServer(server, &myService{})

    lis, _ := net.Listen("tcp", ":50051")
    server.Serve(lis)
}

Client-Side TLS in Go#

import (
    "crypto/x509"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials"
)

func main() {
    // For public CAs (Let's Encrypt, etc.), use system cert pool
    creds := credentials.NewTLS(&tls.Config{
        MinVersion: tls.VersionTLS13,
    })

    // For internal CAs, load the CA cert explicitly
    caCert, _ := os.ReadFile("ca-cert.pem")
    certPool := x509.NewCertPool()
    certPool.AppendCertsFromPEM(caCert)
    creds = credentials.NewTLS(&tls.Config{
        RootCAs:    certPool,
        MinVersion: tls.VersionTLS13,
    })

    conn, err := grpc.NewClient("api.internal:50051",
        grpc.WithTransportCredentials(creds),
    )
    defer conn.Close()

    client := pb.NewMyServiceClient(conn)
}

TLS in Python#

import grpc

# Server
server_credentials = grpc.ssl_server_credentials(
    [(open("server-key.pem", "rb").read(), open("server-cert.pem", "rb").read())]
)
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
pb_grpc.add_MyServiceServicer_to_server(MyService(), server)
server.add_secure_port("[::]:50051", server_credentials)
server.start()

# Client
ca_cert = open("ca-cert.pem", "rb").read()
channel_credentials = grpc.ssl_channel_credentials(root_certificates=ca_cert)
channel = grpc.secure_channel("api.internal:50051", channel_credentials)
client = pb_grpc.MyServiceStub(channel)

Mutual TLS for gRPC#

mTLS is the strongest authentication model for service-to-service gRPC. Each service has a certificate, and both sides verify each other.

Istio Security#

Istio provides three security capabilities that are difficult to implement without a service mesh: automatic mutual TLS between services, fine-grained authorization policies, and egress traffic control. These features work at the infrastructure layer, meaning applications do not need any code changes.

Automatic mTLS with PeerAuthentication#

Istio’s sidecar proxies can automatically encrypt all pod-to-pod traffic with mutual TLS. The key resource is PeerAuthentication. There are three modes:

• PERMISSIVE – Accepts both plaintext and mTLS traffic. This is the default and exists for migration. Do not leave it in production.
• STRICT – Requires mTLS for all traffic. Plaintext connections are rejected.
• DISABLE – Turns off mTLS entirely.

Enable strict mTLS across the entire mesh:

Istio Service Mesh#

Istio adds a proxy sidecar (Envoy) to every pod in the mesh. These proxies handle traffic routing, mutual TLS, retries, circuit breaking, and telemetry without changing application code. The control plane (istiod) pushes configuration to all sidecars.

When You Actually Need a Service Mesh#

You need Istio when you have multiple services requiring mTLS, fine-grained traffic control (canary releases, fault injection), or consistent observability across service-to-service communication. If you have fewer than five services, standard Kubernetes Services and NetworkPolicies are sufficient. A service mesh adds operational complexity – more moving parts, higher memory usage per sidecar, and a learning curve for proxy-level debugging.

Defense in Depth#

No single network control stops every attack. Layer controls so that a failure in one does not compromise the system: host firewalls, Kubernetes network policies, service mesh encryption, API gateway authentication, and DNS security, each operating independently.

Host Firewall: iptables and nftables#

Every node should run a host firewall regardless of the orchestrator. Block everything by default:

# iptables: default deny with essential allows
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH from management network only
iptables -A INPUT -p tcp --dport 22 -s 10.0.100.0/24 -j ACCEPT

# Allow kubelet API (for k8s nodes)
iptables -A INPUT -p tcp --dport 10250 -s 10.0.0.0/16 -j ACCEPT

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT

The nftables equivalent is more readable for complex rulesets:

Service-to-Service Authentication and Authorization#

In a microservice architecture, services communicate over the network. Without authentication, any process that can reach a service can call it. Without authorization, any authenticated caller can do anything. Zero-trust networking assumes the internal network is hostile and requires every service-to-service call to be authenticated, authorized, and encrypted.

Mutual TLS (mTLS)#

Standard TLS has the client verify the server’s identity. Mutual TLS adds the reverse – the server also verifies the client’s identity. Both sides present certificates. This provides three things: encryption in transit, server authentication, and client authentication.

TLS and mTLS Fundamentals#

TLS (Transport Layer Security) encrypts traffic between two endpoints. Mutual TLS (mTLS) adds a second layer: both sides prove their identity with certificates. Understanding these is not optional for anyone building distributed systems — nearly every production failure involving “connection refused” or “certificate verify failed” traces back to a TLS misconfiguration.

How TLS Works#

A TLS handshake establishes an encrypted channel before any application data is sent. The simplified flow:

Choosing a Service Mesh#

A service mesh moves networking concerns – mutual TLS, traffic routing, retries, circuit breaking, observability – out of application code and into the infrastructure layer. Every pod gets a proxy that handles these concerns transparently. The control plane configures all proxies across the cluster.

The decision is not just “which mesh” but “do you need one at all.”

Do You Actually Need a Service Mesh?#

Many teams adopt a mesh prematurely. Before evaluating options, identify which specific problems you are trying to solve:

The TLS Handshake#

Every HTTPS connection starts with a TLS handshake that establishes encryption parameters and verifies the server’s identity. The simplified flow for TLS 1.2:

Client                              Server
  |── ClientHello ──────────────────>|   (supported versions, cipher suites, random)
  |<────────────────── ServerHello ──|   (chosen version, cipher suite, random)
  |<──────────────── Certificate  ──|   (server's certificate chain)
  |<───────────── ServerKeyExchange ─|   (key exchange parameters)
  |<───────────── ServerHelloDone  ──|
  |── ClientKeyExchange ───────────>|   (client's key exchange contribution)
  |── ChangeCipherSpec ────────────>|   (switching to encrypted communication)
  |── Finished ────────────────────>|   (encrypted verification)
  |<──────────── ChangeCipherSpec ──|
  |<──────────────────── Finished ──|
  |<═══════ Encrypted traffic ═════>|

TLS 1.3 simplifies this significantly. The client sends its key share in the ClientHello, allowing the handshake to complete in a single round trip. TLS 1.3 also removed insecure cipher suites and compression, eliminating entire classes of vulnerabilities (BEAST, CRIME, POODLE).

The Core Principle#

Zero trust networking operates on a simple premise: no network location is inherently trusted. Being inside the corporate network, inside a VPC, or inside a Kubernetes cluster does not grant access to anything. Every request must be authenticated, authorized, and encrypted regardless of where it originates.

This is a departure from the traditional castle-and-moat model where a VPN places you “inside” the network and everything inside is implicitly trusted. That model fails because attackers who breach the perimeter have unrestricted lateral movement. Zero trust eliminates the concept of inside versus outside.