How do Kubernetes control-plane components coordinate during a node failure? Explain the sequence involving the API server, etcd, scheduler, controller manager, and kubelet in a concise technical answer.
A Kubernetes node failure is handled as a reconciliation sequence rather than a direct component-to-component command chain:
The kubelet reports health. Each node’s kubelet periodically updates the node’s
statusand renews aLeasein thekube-node-leasenamespace; the default Lease update interval is 10 seconds. If the kubelet or node fails, those heartbeats stop. [citation_1utuxd]The API server exposes and persists the state. The kubelet writes its updates to the API server. The API server is the control plane’s coordination point: it validates requests, reads and writes cluster state, and persists that state in etcd. Other components observe changes through the API server rather than calling one another directly. [citation_1utuw6]
The controller manager detects the failure. The node controller, running inside
kube-controller-manager, watches node heartbeats. After the configured grace period it changes the node’sReadycondition toUnknown/NotReadyand applies anunreachableornot-readytaint. The taint prevents new scheduling there; after the pod-eviction timeout—five minutes by default in the documented behaviour—the node controller initiates eviction of affected Pods. [citation_1utuxd]Controllers restore the desired workload. For Pods managed by a Deployment, ReplicaSet, or similar controller, eviction or loss of the old Pod creates a gap between desired and current state. The relevant controller writes replacement Pod objects through the API server. The scheduler watches for Pods without
spec.nodeName, selects suitable healthy nodes using resource and placement constraints, and writes the chosennodeNameback through the API server. [citation_1utuw1]The new kubelet starts the replacement. The kubelet on the selected node watches for Pods assigned to it, asks the container runtime to create them, and reports their status back through the API server, which persists the updates in etcd. The failed node’s kubelet cannot acknowledge deletion while it is unreachable, so a network partition can leave old Pod objects in
TerminatingorUnknownuntil communication is restored or an operator intervenes. [citation_1utuw4][citation_1utuxd]
Thus, kubelet supplies observations; API server provides the coordination interface; etcd records durable cluster state; controller manager detects drift and requests correction; scheduler chooses placement; and another kubelet makes the replacement real. Timing depends on heartbeat, grace-period, eviction, toleration, and workload-controller settings, so “rescheduled” does not necessarily mean instantaneous. [citation_1utuxd][citation_1utuw6]