Skip to main content

Kubernetes Calico Crash Course

TL;DR, A picture is worth a thousand words. Kubernetes Calico in a glance

Here is the longer version...​

Kubernetes Calico Crash Course

Table of Contents​

  1. Introduction
  2. Core Components
  3. Installation
  4. Networking Mechanics
  5. Pod-to-Pod Communication
  6. Network Policies
  7. Advanced Features
  8. Troubleshooting
  9. Configuration Discovery

Introduction​

Calico is a cloud-native networking solution providing:

  • High-performance container networking
  • Advanced network security policies
  • Cross-platform compatibility (K8s, OpenStack, bare metal)
  • Choice of data planes (Linux, eBPF, Windows)

Core Components​

ComponentFunctionRuns On
FelixPrograms routes/ACLsEach node
BIRDBGP route distributionEach node
confdConfig generationControl nodes
CNI PluginPod networkingEach node
TyphaAPI server proxy (large clusters)Control nodes

Installation​

# Install operator
kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml

# Apply custom resources
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml

# Verify installation
watch kubectl get pods -n calico-system

Networking Mechanics​

IP Address Management (IPAM)​

apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: default-pool
spec:
  cidr: 192.168.0.0/16
  ipMode: CrossSubnet
  natOutgoing: true

Routing Options​

  1. BGP Direct Routing

    • Uses node IPs for pod-to-pod communication
    • Requires BGP-enabled network infrastructure

  2. Overlay Networks

    • IP-in-IP (protocol 4) or VXLAN (UDP 4789)
    • Works across any L3 network

Pod-to-Pod Communication​

Architecture Overview​

Calico enables direct pod-to-pod communication using these key components:

  • veth pairs for same-node communication
  • tunl0 interface for IP-in-IP encapsulation
  • BGP routing for cross-node traffic
  • iptables/eBPF for policy enforcement
graph TD
    A[Pod A] -->|veth pair| B[Host A]
    B -->|BGP/Overlay| C[Host B]
    C -->|veth pair| D[Pod B]

Communication Flow​

Same node

sequenceDiagram
    Pod1->>veth1: Outgoing packet
    veth1->>Linux Bridge: Forward
    Linux Bridge->>veth2: Direct delivery
    veth2->>Pod2: Incoming packet

Cross-node

sequenceDiagram
    Pod1->>tunl0: Encapsulated packet
    tunl0->>Node1 NIC: Add outer header (src:10.30.1.1, dst:10.30.1.2)
    Node1 NIC->>Node2 NIC: Routed via underlay
    Node2 NIC->>tunl0: Decapsulate
    tunl0->>Pod2: Deliver original packet

Configuration​

  1. IP Pool Setup Define CIDR and encapsulation mode:
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: default-pool
spec:
  cidr: 192.168.0.0/16
  ipipMode: Always  # Options: Always, CrossSubnet, Never
  vxlanMode: Never
  natOutgoing: true
  1. Interface Detection Configure IP autodetection method:
# calico-config ConfigMap
data:
  ip_autodetection_method: "interface=eth.*"
  1. BGP Configuration For direct routing without encapsulation:
calicoctl create -f - <<EOF
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  logSeverityScreen: Info
  nodeToNodeMeshEnabled: true
EOF

Verification​

  1. Check routes on worker node:
ip route show | grep calico
# Example output:
# 192.168.1.0/24 via 10.30.1.1 dev tunl0 proto bird
  1. Test pod connectivity:
kubectl exec pod1 -- ping -c 3 <pod2-ip>
  1. Inspect BGP peers:
calicoctl node status
# Output:
# IPv4 BGP status
# +--------------+-------------------+-------+----------+-------------+
# | PEER ADDRESS |     PEER TYPE     | STATE |  SINCE   |    INFO     |
# +--------------+-------------------+-------+----------+-------------+
# | 10.30.1.2    | node-to-node mesh | up    | 09:32:44 | Established |
# +--------------+-------------------+-------+----------+-------------+

Troubleshooting Common Issues​

  1. Missing Routes Ensure Calico pods are running:
kubectl get pods -n calico-system
  1. Encapsulation Mismatch Verify IP pool configuration matches cluster network:
calicoctl get ippool -o yaml
  1. Firewall Conflicts Required open ports:
  • IP-in-IP: Protocol 4
  • VXLAN: UDP 4789
  • BGP: TCP 179
  1. Interface Detection Check detected host IP:
calicoctl get node <node-name> -o yaml | grep 'ipv4Address'

This implementation enables near line-rate performance (< 1ms latency) while maintaining Kubernetes network model compliance1. For hybrid cloud deployments, consider VXLAN encapsulation when BGP isn't feasible2.

Configuration Matrix​

ScenarioConfigurationPerformanceRequirements
Same NodeDirect veth routing10 Ξs latencyDefault setup
Cross-Node/BGPBGP routing table
# Inspect interfaces
ip link show | grep calico

# View BGP peers
birdcl -s /var/run/calico/bird.ctl show protocols

Network Policies​

Example: Multi-Tier App Security​

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: frontend-policy
spec:
  selector: role == 'frontend'
  ingress:
    - action: Allow
      protocol: TCP
      source:
        selector: app == 'ingress-controller'
      destination:
        ports:[443]
  egress:
    - action: Allow
      protocol: TCP
      destination:
        selector: role == 'backend'
        ports:[8080]

Advanced Features​

  1. eBPF Data Plane

    calicoctl patch kubecontrollersconfiguration default \
      --patch='{"spec": {"controllers": {"node": {"hostEndpoint": {"autoCreate": "Enabled"}}}}}'

  2. WireGuard Encryption

    apiVersion: projectcalico.org/v3
    kind: FelixConfiguration
    metadata:
      name: default
    spec:
      wireguardEnabled: true

  3. Service Graph

    calicoctl get serviceGraph -o wide

Troubleshooting​

Common Issues​

  1. Pods can't communicate across nodes

    • Check BGP peer status
    • Verify IP pool CIDR matches cluster network
    • Ensure firewall allows Calico protocols

  2. Network Policy not enforced

    • Check policy order (calicoctl get policy -o wide)
    • Verify selector matches pod labels

  3. IPAM exhaustion

    calicoctl ipam release --ip= --from=

Configuration Discovery​

  1. Cluster-wide Settings

    calicoctl get clusterinformation -o yaml

  2. Node-specific Config

    calicoctl get node  -o yaml

  3. Current State Analysis

    calicoctl diags --include=all

  4. Policy Audit

    calicoctl get networkpolicy -A -o yaml

This comprehensive guide covers Calico's fundamentals through advanced features. For production deployments, always reference the official documentation for version-specific details.

Footnotes​

  1. Reference for network model compliance statistics â†Đ

  2. Reference for VXLAN encapsulation in hybrid deployments â†Đ