proomt

Search

Search posts, papers, and topics

All posts

Kubernetes8 min readintermediate

Spotlight on SIG Apps

Read on kubernetes.io

Summary

Interview with SIG Apps chairs Janet Kuo and Maciej Szulik about the evolution of Kubernetes workload controllers, current focus on node‑lifecycle resilience, new sub‑projects like JobSet and LeaderWorkerSet for AI workloads, and upcoming KEP‑4443 to improve Job failure reporting.

  • Node lifecycle issues are being tackled via a dedicated Working Group to reduce manual recovery of DaemonSets/Jobs on flaky nodes.
  • New workload patterns for AI/ML (JobSet, LeaderWorkerSet) add coordinated failure handling and checkpoint‑based restarts.
  • KEP‑4443 proposes named PodFailurePolicy rules to surface richer failure reasons, aiding higher‑level controllers.
  • SIG Apps balances adding features (e.g., Agent Sandbox) with backward‑compatible, opt‑in designs, often preferring CRDs for experimental APIs.

Reliability of core workload controllers directly impacts production platform stability, especially for large‑scale AI and batch jobs where node failures can cause costly downtime. The discussed improvements aim to automate recovery, provide clearer failure signals, and introduce scalable patterns…

5/10
aikubernetesworkloadsnode lifecyclejobsetleaderworkerset

Related reading

  1. Microsoft Open-Sources TauGrid to Simplify AI Workload Management on Kubernetes

    Microsoft open‑sourced TauGrid, a Helm‑installable platform that bundles Kueue, KubeRay and other components to schedule, monitor and checkpoint AI workloads on GPU‑enabled Kubernetes clusters. Engineers define jobs in a simple `tau.yaml`, and TauGrid handles queuing, health checks, and observability, simplifying AI workload management.

    InfoQinfoq.com2 min

  2. Monitor TAS and gang scheduling for AI training in Kubernetes

    Kubernetes’ default scheduler can’t satisfy AI training’s need for low‑latency GPU interconnects and simultaneous pod start‑up. The blog explains how the open‑source Kueue job queue adds topology‑aware placement (using node labels like `topology.kubernetes.io/rack`) and how the Coscheduling plugin adds a permit phase that only binds a gang of pods when the full set is ready, preventing idle GPU r…

    Datadogdatadoghq.com19 min

  3. Article: Beyond Relevance: A Governance-First Architecture for Enterprise Personalization

    The article proposes a governance‑first architecture for enterprise personalization, where policy‑driven steps (memory, journey graph, AI routing, scoring, trust checks, outcome simulation) shape the recommendation before it is returned. A reference FastAPI implementation demonstrates the pattern with external YAML policies and optional LLM assistance.

    InfoQinfoq.com19 min

  4. Kubernetes Multi-Cluster Project Karmada Reaches CNCF Graduation

    Karmada, a multi-cluster and multi-cloud Kubernetes orchestration project, has graduated to the highest maturity tier within the CNCF. It extends the standard Kubernetes API to manage workloads across diverse environments, enhancing multi-component scheduling for AI training jobs and promoting priority-based scheduling.

    InfoQinfoq.com3 minrelease

  5. Kubernetes v1.37: Pod-Level Resource Managers graduated to Beta

    Kubernetes v1.37 adds Pod‑Level Resource Managers to beta (off by default). The feature lets Kubelet’s Topology, CPU, and Memory managers consume pod‑level `.spec.resources` to reserve exclusive NUMA‑aligned CPUs/memory for primary containers while sidecars share a pod‑isolated pool. A new PodResources gRPC API now reports `cpu_ids` and `memory` per pod. Enable via the `PodLevelResourceManagers`…

    Kuberneteskubernetes.io2 min

  6. Kubernetes 1.37 Released: Stable Metrics API and Rootless Kubelet in Beta

    Kubernetes 1.37 (Garhwal) ships GA Metrics API, beta rootless kubelet, GA resilient watchcache, GA pod certificates, and a suite of alpha/beta features (workload‑aware scheduling, pod checkpoint/restore, StatefulSet Recreate strategy). The release focuses on stability, security, and AI/ML cost‑optimisation.

    InfoQinfoq.com2 minrelease