Cilium CNI Security Testing: Network Policy Bypass, Hubble Visibility Gaps, and eBPF Exploitation

Cilium Security Architecture

CiliumNetworkPolicy Audit

# List all CiliumNetworkPolicies and CiliumClusterwideNetworkPolicies
kubectl get ciliumnetworkpolicies --all-namespaces -o yaml | \
  python3 -c "
import yaml, sys
docs = list(yaml.safe_load_all(sys.stdin))
for cnp in docs:
    if not cnp or cnp.get('kind') not in ['CiliumNetworkPolicy','CiliumClusterwideNetworkPolicy']:
        continue
    meta = cnp.get('metadata', {})
    spec = cnp.get('spec', {})
    print(f\"{meta.get('namespace','cluster')}/{meta.get('name','?')}\")

    # Check for ingress default allow (no ingress rules = deny all ingress)
    if 'ingress' not in spec and 'ingressDeny' not in spec:
        print('  [?] No ingress rules — falls through to default behavior')

    # Check for wildcard selectors
    ingress_rules = spec.get('ingress', [])
    for rule in ingress_rules:
        sources = rule.get('fromEndpoints', []) + rule.get('fromCIDR', [])
        for src in sources:
            if src == {}:  # Empty selector = all endpoints
                print('  [!] WILDCARD ingress from all endpoints')
"

# Check cluster-wide default deny policy
kubectl get ciliumclusterwidenetworkpolicy -o json | \
  python3 -c "
import json,sys
policies = json.load(sys.stdin).get('items', [])
has_default_deny = any(
    not p.get('spec',{}).get('ingress') and
    p.get('metadata',{}).get('name','').startswith('default-deny')
    for p in policies
)
print(f'Default deny policy exists: {has_default_deny}')
"

Network Policy Bypass Techniques

Host Network Bypass

# Pods with hostNetwork: true share the node's network namespace
# CiliumNetworkPolicy applies to pod identity, not node identity
# hostNetwork pods may bypass pod-level CNP rules

# Test if a hostNetwork pod can reach services that should be blocked
kubectl run bypass-test \
  --image=alpine \
  --overrides='{"spec":{"hostNetwork":true,"containers":[{"name":"test","image":"alpine"}]}}' \
  --rm -it -- sh -c "wget -qO- http://target-service.default.svc.cluster.local"

# Check which pods use hostNetwork
kubectl get pods --all-namespaces -o json | \
  python3 -c "
import json,sys
pods = json.load(sys.stdin)['items']
for p in pods:
    if p['spec'].get('hostNetwork'):
        meta = p['metadata']
        print(f\"hostNetwork pod: {meta['namespace']}/{meta['name']}\")
"

NodePort and ExternalIP Bypass

# CiliumNetworkPolicy may enforce L7 policy on ClusterIP traffic
# but NodePort traffic may bypass L7 filtering depending on configuration

# Test access via NodePort (bypasses service mesh routing)
NODE_IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
NODE_PORT=$(kubectl get svc target-service -o jsonpath='{.spec.ports[0].nodePort}')
kubectl exec -it test-pod -- wget -qO- "http://$NODE_IP:$NODE_PORT/api/admin"

# Test if L7 policy (HTTP methods/paths) is enforced on NodePort traffic
kubectl exec -it test-pod -- curl -X DELETE "http://$NODE_IP:$NODE_PORT/api/resource"
# If DELETE succeeds when CNP only allows GET → bypass

CIDR Policy Gaps

# CiliumNetworkPolicy fromCIDR rules apply to IP ranges
# Pods can sometimes reach cluster-external IPs not covered by CIDR policies

# Check for unrestricted external egress
kubectl exec -it test-pod -- curl -s --max-time 5 "http://169.254.169.254/metadata"
kubectl exec -it test-pod -- curl -s --max-time 5 "http://8.8.8.8"
# If either succeeds → no egress restriction to cloud metadata/internet

# Check if CIDR policy covers cloud metadata service
kubectl get cnp --all-namespaces -o yaml | grep "169.254.169.254\|metadata"
# Empty result → cloud metadata reachable from all pods

Identity-Based Policy Attacks

Cilium assigns identities to endpoints based on Kubernetes labels. Policies reference identities via label selectors. An attacker who can change pod labels may manipulate identity assignment.

# CiliumIdentity shows how Cilium resolves endpoint identities
kubectl get ciliumidentities --all-namespaces | head -20
kubectl get ciliumendpoints --all-namespaces -o json | \
  python3 -c "
import json,sys
eps = json.load(sys.stdin)['items']
for ep in eps:
    meta = ep['metadata']
    identity = ep.get('status',{}).get('identity',{})
    labels = identity.get('labels',[])
    print(f\"{meta['namespace']}/{meta['name']}: ID={identity.get('id','?')}\")
    for lbl in labels[:3]:
        print(f'  {lbl}')
"

# Identity spoofing test: change pod labels to match a higher-trust identity
# If a policy allows traffic from pods with label 'role=frontend':
# An attacker who can patch pod labels can add 'role=frontend' to their pod
kubectl patch pod attacker-pod -p '{"metadata":{"labels":{"role":"frontend"}}}'
# Now attacker pod has frontend identity — can reach backend services

# Verify: check what Cilium identity was assigned after label change
kubectl get ciliumendpoint attacker-pod -o json | \
  python3 -c "import json,sys; ep=json.load(sys.stdin); print(ep['status']['identity'])"

# Prevent: use namespace-scoped policies that also require namespace label
# fromEndpoints:
#   - matchLabels:
#       role: frontend
#       k8s:io.kubernetes.pod.namespace: frontend-namespace

Hubble Visibility Gap Testing

# Hubble captures flow records from eBPF — check what's actually visible

# Access Hubble CLI
kubectl exec -it -n kube-system cilium-xxx -- hubble observe --last 100

# Check if Hubble is capturing all namespaces
kubectl exec -it -n kube-system cilium-xxx -- \
  hubble observe --namespace kube-system --last 50

# Common Hubble blind spots:
# 1. Host network traffic (pods with hostNetwork:true)
kubectl exec -it -n kube-system cilium-xxx -- \
  hubble observe --node-name $(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') \
  --type drop --last 50

# 2. Encrypted traffic (Cilium WireGuard encryption) — flows visible but payload encrypted
# 3. Traffic that bypasses Cilium (raw sockets, iptables rules before Cilium)

# Check Hubble relay authentication (should require mTLS)
kubectl get configmap -n kube-system cilium-config -o yaml | grep -i "hubble-tls\|hubble-disable-tls"

# Test unauthenticated Hubble relay access
kubectl port-forward -n kube-system svc/hubble-relay 4245:443 &
# Try connecting without cert:
hubble observe --server localhost:4245 2>&1 | head -5
# "transport: authentication handshake failed" = secured
# Successful connection = unauthenticated relay access

# Hubble UI access control
kubectl port-forward -n kube-system svc/hubble-ui 12000:80 &
curl -s "http://localhost:12000/" | head -5
# If accessible without auth → all network flows visible to anyone who can port-forward

eBPF Map Inspection and Manipulation

# Cilium uses eBPF maps to store endpoint state, policy, and connection tracking
# CAP_BPF allows reading these maps — reveals network topology

# Check if pods have CAP_BPF capability
kubectl get pods --all-namespaces -o json | \
  python3 -c "
import json,sys
pods = json.load(sys.stdin)['items']
for p in pods:
    for c in p['spec'].get('containers',[]):
        caps = c.get('securityContext',{}).get('capabilities',{})
        add = caps.get('add',[])
        if 'CAP_BPF' in add or 'CAP_NET_ADMIN' in add or 'CAP_SYS_ADMIN' in add:
            meta = p['metadata']
            print(f\"{meta['namespace']}/{meta['name']}/{c['name']}: caps={add}\")
"

# From a privileged pod or node access: inspect Cilium's eBPF maps
# List Cilium BPF maps
bpftool map list | grep -i cilium

# Dump policy map — reveals allowed connections
bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_policy_00000

# Dump endpoint map — reveals all pod IPs and their identities
bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_lxc | head -50

# Dump connection tracking map — reveals active connections
bpftool map dump pinned /sys/fs/bpf/tc/globals/cilium_ct4_global | \
  python3 -c "
import sys
# Parse connection tracking entries to reveal internal service communication patterns
print('CT table entries (first 20):')
for i, line in enumerate(sys.stdin):
    if i >= 20: break
    print(line.strip())
"

Tetragon Runtime Security Testing

# Tetragon provides kernel-level enforcement via eBPF
# TracingPolicy resources define which syscalls/events trigger alerts or enforcement

# List TracingPolicies
kubectl get tracingpolicies --all-namespaces -o yaml | \
  python3 -c "
import yaml, sys
for tp in yaml.safe_load_all(sys.stdin):
    if not tp: continue
    meta = tp.get('metadata', {})
    spec = tp.get('spec', {})
    print(f\"Policy: {meta.get('name','?')}\")
    for kprobe in spec.get('kprobes', []):
        print(f\"  kprobe: {kprobe.get('call','?')} action: {kprobe.get('action','observe')}\")
"

# Test coverage: check which syscalls have enforcement vs observe-only
# Enforce = block; Observe = alert only
# Common gap: SIGKILL action vs process execution enforcement

# Test if Tetragon detects privilege escalation attempts
# In a test pod:
kubectl exec -it test-pod -- sh -c "
# Try to write to /etc/passwd (should be blocked if policy exists)
echo 'pwned:x:0:0:root:/root:/bin/sh' >> /etc/passwd 2>&1 | head -5
"

# Check Tetragon events for the test activity
kubectl exec -it -n kube-system tetragon-xxx -- tetra getevents --namespace default

# Test network enforcement via Tetragon (if network kprobes are configured)
kubectl exec -it test-pod -- curl -s --max-time 2 "http://169.254.169.254" 2>&1
kubectl exec -it -n kube-system tetragon-xxx -- tetra getevents --namespace default \
  --process curl

Cilium Security Hardening

# 1. Deploy cluster-wide default deny policy
apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: "default-deny-all"
spec:
  endpointSelector: {}
  ingress:
  - {}  # This is a deny-all: empty ingress list blocks all ingress
  # Note: Add explicit allow policies for required traffic after this

# 2. Restrict pod label mutation via OPA/Gatekeeper/Kyverno
# Policy: deny pod updates that add trust-related labels
# This prevents identity spoofing via label manipulation

# 3. Enable Hubble with mTLS
# cilium-config:
hubble-tls-cert-file: /var/lib/cilium/tls/hubble/server.crt
hubble-tls-key-file: /var/lib/cilium/tls/hubble/server.key
hubble-tls-client-ca-files: /var/lib/cilium/tls/hubble/client-ca.crt

# 4. Restrict Hubble UI access via NetworkPolicy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: restrict-hubble-ui
  namespace: kube-system
spec:
  podSelector:
    matchLabels:
      k8s-app: hubble-ui
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system

# 5. Enable Cilium WireGuard transparent encryption
# Encrypts all pod-to-pod traffic: no eavesdropping between nodes
# helm values:
# encryption:
#   enabled: true
#   type: wireguard

# 6. Use namespace-scoped policies with namespace label selector
# Prevents cross-namespace identity spoofing
fromEndpoints:
  - matchLabels:
      role: frontend
      "k8s:io.kubernetes.pod.namespace": frontend  # Require specific namespace

# 7. Block access to cloud metadata from all pods unless explicitly needed
apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: "deny-cloud-metadata"
spec:
  endpointSelector: {}
  egressDeny:
  - toCIDR:
    - "169.254.169.254/32"  # AWS/GCP/Azure metadata endpoint