| Tested on | Rocky Linux 10.2 (Red Quartz) workstation |
|---|---|
| Package | kubectl 1.36.3Calico CNI ( calico-node DaemonSet)kube-proxy (iptables mode) |
| Applies to | Any host with kubectl configured; any Kubernetes cluster |
| Cert prep | CKA |
| Lab environment | Multi-node kubeadm cluster with containerd — install Kubernetes with kubeadm. This rewrite uses Calico as the installed CNI example; route and interface names are Calico-specific where noted. |
| Privilege | Normal user for kubectl when kubeconfig is available; [sudo](/sudo-command-in-linux/) or root on nodes to inspect CNI config and routes |
| Scope | Kubernetes networking requirements, Pod/Service/node address ranges, same-Pod networking, Pod-to-Pod on one node and across nodes, CNI data-plane role, Service ClusterIP path, kube-proxy modes, EndpointSlices, where DNS and NetworkPolicy fit, and how to inspect the active stack. Does not cover CNI install, vendor comparisons, packet captures, Service YAML tutorials, CoreDNS troubleshooting, NetworkPolicy examples, or Ingress/Gateway setup. |
Kubernetes networking is easier to reason about as three traffic paths: containers inside one Pod, Pod IP to Pod IP, and Pod to Service ClusterIP. This rewrite walks those paths on a kubeadm cluster running Calico and kube-proxy, and keeps plugin-specific details labeled as examples rather than universal rules.
Kubernetes networking requirements
The platform model expects these properties:
- Every Pod receives its own cluster-routable IP.
- Containers in one Pod share a network namespace and can use
localhost. - Pods should communicate directly across nodes without application-visible NAT under the standard model.
- Nodes must be able to reach Pod IPs.
- Services provide stable virtual access while backend Pods change.
Your CNI product implements the Pod IP and node reachability parts. kube-proxy (or a replacement) implements most Service virtual IP behavior on Linux nodes.
Identify cluster address ranges
Separate three ranges before debugging routes:
| Range | Used for |
|---|---|
| Node network | Host connectivity between machines |
| Pod CIDR | Pod addresses |
| Service CIDR | Virtual Service ClusterIPs |
Those ranges must not overlap. If a Service CIDR collides with a Pod or node network, packets can take the wrong path and ClusterIP proxying becomes unreliable.
On this kubeadm lab, read the cluster networking stanza from the kubeadm ConfigMap:
kubectl -n kube-system get cm kubeadm-config -o yaml | grep -A5 'networking:'networking:
dnsDomain: cluster.local
podSubnet: 192.168.0.0/16
serviceSubnet: 10.96.0.0/12Confirm the controller-manager arguments match:
kubectl -n kube-system get pod kube-controller-manager-k8s-cp -o yaml | grep -E 'cluster-cidr|service-cluster-ip-range'- --cluster-cidr=192.168.0.0/16
- --service-cluster-ip-range=10.96.0.0/12Service ClusterIPs in this cluster fall under 10.96.0.0/12 (for example 10.96.0.10 for kube-dns). Pod IPs fall under the Pod subnet; Calico IPAM may allocate smaller blocks inside that range on each node.
Networking inside one Pod
Containers in the same Pod share one network namespace:
- They share the Pod IP and the port space.
- They reach each other through
localhost. - Two containers cannot both bind the same port.
- Volumes and process namespaces are separate concerns from networking.
Create a two-container Pod so the client talks to nginx on loopback:
kubectl create ns net-model-demonamespace/net-model-demo createdkubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
name: shared-netns
namespace: net-model-demo
spec:
containers:
- name: web
image: nginx:1.27-alpine
ports:
- containerPort: 80
- name: client
image: busybox:1.36
command:
- sh
- -c
- |
until wget -qO- http://127.0.0.1/; do
sleep 1
done
sleep 3600
EOFpod/shared-netns createdkubectl -n net-model-demo wait --for=condition=Ready pod/shared-netns --timeout=90spod/shared-netns condition metkubectl -n net-model-demo logs shared-netns -c client --tail=5<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>The client reached the web container without using a Service or a second Pod IP. Multi-container patterns beyond this networking detail are covered in sidecar and multi-container Pods.
Pod-to-Pod traffic on the same node
When two Pods share a node, traffic stays on that host’s CNI data plane:
Source Pod interface
↓
Pod network namespace
↓
CNI-created host networking
↓
Destination Pod interfaceDo not assume every plugin uses the same bridge, eBPF map, or veth naming. The portable idea is that the source uses the destination Pod IP, and the node-local CNI plumbing delivers the packet.
On this Calico lab, local Pods appear as cali… interfaces and host routes for those Pod IPs. Same-node Pod-to-Pod looks identical at the application layer: wget or curl to the peer Pod IP. The next section creates a cross-node lab to show how remote Pod CIDR reachability works.
Manifest and API checks with curl are covered in the curl command guide.
Pod-to-Pod traffic across nodes
Across nodes, the source node must know how to reach the destination Pod CIDR. The CNI implementation supplies routes, overlays, tunnels, or another data plane. The destination Pod IP remains the application-visible destination.
Create two nginx Pods on different nodes and a Service for later sections. Both Pods are pinned with nodeName: net-a on the worker and net-b on the control plane. nodeName bypasses the scheduler, so a control-plane NoSchedule taint does not block net-b:
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
name: net-a
namespace: net-model-demo
labels:
app: net-demo
role: a
spec:
nodeName: worker01
containers:
- name: web
image: nginx:1.27-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Pod
metadata:
name: net-b
namespace: net-model-demo
labels:
app: net-demo
role: b
spec:
nodeName: k8s-cp
containers:
- name: web
image: nginx:1.27-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: net-demo
namespace: net-model-demo
spec:
selector:
app: net-demo
ports:
- port: 80
targetPort: 80
EOFpod/net-a created
pod/net-b created
service/net-demo createdWait until both Pods are Ready:
kubectl -n net-model-demo wait \
--for=condition=Ready \
pod/net-a pod/net-b \
--timeout=120spod/net-a condition met
pod/net-b condition metList the Pods and confirm they landed on different nodes:
kubectl -n net-model-demo get pods -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
net-a 1/1 Running 0 25s 192.168.5.7 worker01 <none> <none>
net-b 1/1 Running 0 3s 192.168.62.164 k8s-cp <none> <none>From net-a on worker01, fetch nginx on net-b by Pod IP:
NET_B_IP=$(
kubectl -n net-model-demo get pod net-b \
-o jsonpath='{.status.podIP}'
)
kubectl -n net-model-demo exec net-a -- \
wget -qO- --timeout=3 "http://${NET_B_IP}/" | head -3<!DOCTYPE html>
<html>
<head>The client used the remote Pod IP directly. On this Calico example, worker01 has a route toward the control-plane Pod block via the node network:
ip route | grep -E '192.168.62|192.168.5'192.168.5.0 dev cali8c36ee9f3dd scope link metric 1024
blackhole 192.168.5.0/26 proto 80
192.168.5.7 dev calif29d8e222be scope link metric 1024
192.168.62.128/26 via 192.168.56.108 dev enp0s8 proto 80 onlinkThat via 192.168.56.108 line is Calico programming reachability for a remote Pod CIDR block. Other CNIs may show VXLAN devices, WireGuard interfaces, or pure L3 routes instead — treat the mechanism as plugin-specific, and treat “Pod IP reaches Pod IP” as the model.
CNI’s role in the data plane
When the runtime creates a Pod sandbox, it invokes the configured CNI plugins. On each node you typically find:
- Plugin config under
/etc/cni/net.d/ - Plugin binaries under
/opt/cni/bin/ - Optional node agents and controllers from the networking product
List the CNI config on a worker:
ls /etc/cni/net.d/10-calico.conflist
calico-kubeconfigThe Calico conflist is JSON. The following abridged interpretation highlights the relevant plugins and is not literal file syntax:
name: k8s-pod-network
plugins:
- type: calico
ipam:
type: calico-ipam
policy:
type: k8s
- type: portmapCluster networking products often add DaemonSets such as calico-node. Some also implement NetworkPolicy and, in certain modes, Service proxy replacement. Interface-level CNI versus CSI versus CRI boundaries are covered in CNI, CSI, and CRI interfaces.
How Service ClusterIP works
A ClusterIP is usually a virtual address. It is not assigned to a normal Pod interface. Traffic takes this path:
Client Pod
↓
Service ClusterIP:port
↓
Service proxy data plane
↓
Ready endpoint from EndpointSlice
↓
Backend PodIP:targetPortInspect the demo Service:
kubectl -n net-model-demo get svc net-demo -o wideNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
net-demo ClusterIP 10.109.219.198 <none> 80/TCP 3s app=net-demoFrom net-a, call the ClusterIP:
SERVICE_IP=$(
kubectl -n net-model-demo get service net-demo \
-o jsonpath='{.spec.clusterIP}'
)
kubectl -n net-model-demo exec net-a -- \
wget -qO- --timeout=3 "http://${SERVICE_IP}/" | head -3<!DOCTYPE html>
<html>
<head>The client addressed 10.109.219.198, which sits in the Service CIDR, and received a backend nginx response. Service object fields and types are covered in Services, Endpoints, and EndpointSlices.
kube-proxy and its modes
On Linux, kube-proxy can program Service forwarding with:
- iptables — current default when mode is empty
- nftables — stable modern mode and the recommended replacement for IPVS
- IPVS — still available, but deprecated since Kubernetes 1.35
Windows uses a kernelspace mode. kube-proxy watches Services and EndpointSlices, then updates node-level rules. It does not create Pod veth pairs or assign Pod CIDRs. Some CNI implementations replace kube-proxy with their own Service data plane.
This lab’s ConfigMap leaves mode empty, which selects the default iptables path:
kubectl -n kube-system get cm kube-proxy -o jsonpath='{.data.config\.conf}' | grep -E '^mode:|^clusterCIDR:'clusterCIDR: 192.168.0.0/16
mode: ""Confirm from kube-proxy logs:
kubectl -n kube-system logs ds/kube-proxy --tail=20 | grep -iE 'Using|iptables|ipvs|nftables'I0727 12:47:08.728300 1 server_linux.go:49] "Using iptables proxy"
I0727 12:47:14.844109 1 server_linux.go:137] "Using iptables Proxier"Avoid treating one mode as universally preferred. An empty mode currently selects iptables on Linux. nftables is the recommended modern alternative on supported systems. IPVS remains available but is deprecated. Change mode only after reading your Kubernetes version docs and validating the CNI compatibility matrix.
EndpointSlices in Service routing
EndpointSlices hold backend addresses and ports for a Service. Readiness affects whether an endpoint is eligible. Large Services may span multiple slices. kube-proxy or its replacement consumes those objects.
kubectl -n net-model-demo get endpointslices -l kubernetes.io/service-name=net-demo -o wideNAME ADDRESSTYPE PORTS ENDPOINTS AGE
net-demo-cf469 IPv4 80 192.168.5.7,192.168.62.164 78skubectl get ... -o wide lists endpoint addresses but does not show the ready condition. Confirm readiness explicitly:
kubectl -n net-model-demo get endpointslices \
-l kubernetes.io/service-name=net-demo \
-o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]}{"\tready="}{.conditions.ready}{"\n"}{end}'192.168.5.7 ready=true
192.168.62.164 ready=trueEndpointSlice readiness determines whether an endpoint is normally eligible for Service traffic.
Where DNS fits
CoreDNS resolves Service DNS names to Service ClusterIPs for normal Services, or to endpoint records for headless Services. After resolution, traffic still follows the ClusterIP or Pod-IP paths above. CoreDNS health and name-resolution failures belong in DNS and CoreDNS troubleshooting.
Where NetworkPolicy fits
NetworkPolicy selects Pods and declares allowed ingress and egress. Enforcement belongs to a supporting network implementation (this Calico install advertises Kubernetes policy in the CNI conflist). Policy can apply before or after address translation depending on the implementation. It never invents a route the CNI data plane cannot provide.
This lab currently has no NetworkPolicy objects:
kubectl get networkpolicies -ANo resources foundPolicy YAML and examples live in NetworkPolicy with examples.
Inspect the active networking stack
Use API objects first, then node-local CNI and routes when you need the data plane.
List Pods with IPs and nodes:
kubectl get pods -A -o wide | head -8NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
calico-system calico-apiserver-59bcc9778f-2gsfr 1/1 Running 0 77m 192.168.62.146 k8s-cp <none> <none>
calico-system calico-apiserver-59bcc9778f-xtqp6 1/1 Running 0 79m 192.168.62.141 k8s-cp <none> <none>
calico-system calico-kube-controllers-65898666fc-9nng8 1/1 Running 0 79m 192.168.62.140 k8s-cp <none> <none>
calico-system calico-node-94m77 1/1 Running 0 90m 192.168.56.108 k8s-cp <none> <none>Networking DaemonSets on this cluster:
kubectl get daemonsets -A | grep -iE 'NAME|calico-node|kube-proxy'NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
calico-system calico-node 2 2 2 2 2 kubernetes.io/os=linux 89m
kube-system kube-proxy 2 2 2 2 2 kubernetes.io/os=linux 90mServices and EndpointSlices:
kubectl get services -A | head -10NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
calico-system calico-api ClusterIP 10.103.228.134 <none> 443/TCP 89m
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 90m
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 90mkubectl get endpointslices -A | head -8NAMESPACE NAME ADDRESSTYPE PORTS ENDPOINTS AGE
calico-system calico-api-n76qx IPv4 5443 192.168.62.141,192.168.62.146 92m
calico-system calico-kube-controllers-metrics-6vwlr IPv4 9094 192.168.62.140 91m
calico-system calico-typha-v8zfp IPv4 5473 192.168.56.108 92m
calico-system goldmane-f2f72 IPv4 7443 192.168.62.143 92m
calico-system whisker-hmr79 IPv4 8081 192.168.62.142 92m
default kubernetes IPv4 6443 192.168.56.108 93m
kube-system kube-dns-vll48 IPv4 53,53,9153 192.168.62.144,192.168.62.145 93mOn a node, confirm CNI config and Pod routes as shown earlier (ls /etc/cni/net.d/, ip route). Together, those views answer which product owns Pod networking and which process owns Service virtual IPs.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Pods cannot reach Pod IPs across nodes | CNI not Ready, missing routes/overlay | Check CNI DaemonSet; inspect ip route on both nodes |
| ClusterIP times out but Pod IP works | kube-proxy or Service proxy replacement issue | Check kube-proxy Pods/logs and EndpointSlices |
| ClusterIP works, DNS name fails | CoreDNS or client DNS config | Use the DNS troubleshooting guide |
| Same-Pod localhost fails | Wrong port or container not listening | Confirm shared netns and the listening port |
| Overlapping CIDRs after install | podSubnet/serviceSubnet/node network collision | Redesign non-overlapping ranges before production traffic |
What's Next
- Kubernetes Services, Endpoints and EndpointSlices
- Troubleshoot a Kubernetes Service That Is Not Working
- Services, CoreDNS and Name Resolution
References
Summary
Kubernetes networking is a layered contract: Pods get IPs, containers in one Pod share localhost, Pods talk by Pod IP across the cluster, and Services publish stable virtual addresses in a separate CIDR. CNI implements the Pod and node data plane; kube-proxy or a replacement turns ClusterIPs into forwarding toward ready endpoints.
On this Calico lab, cross-node Pod traffic used the peer Pod IP while Calico installed routes for remote Pod CIDR blocks, and kube-proxy in iptables mode handled ClusterIP access using EndpointSlice backends. Plugin interface names and route protocols are examples, not universal constants.
When something fails, decide which path broke — same-Pod, Pod-to-Pod, or Pod-to-Service — then inspect CNI health, routes, kube-proxy mode, and EndpointSlices before changing YAML elsewhere. Next, practice Service object details or NetworkPolicy once the data plane paths above are clear.

