Search posts, papers, and topics
Pine59 migrated its large data pipelines to Google Cloud's Managed Airflow (Gen 3) with Airflow 3, achieving significant performance gains and improved MLOps capabilities. This modernization reduced DAG run times by up to 32% and enhanced developer workflows with custom plugins.
Karmada, a multi-cluster and multi-cloud Kubernetes orchestration project, has graduated to the highest maturity tier within the CNCF. It extends the standard Kubernetes API to manage workloads across diverse environments, enhancing multi-component scheduling for AI training jobs and promoting priority-based scheduling.
The post announces the official ClickHouse provider for Apache Airflow, showing how to install it, configure a ClickHouse connection, and run queries using standard SQL operators or a ClickHouseHook. It includes examples for both self‑hosted Airflow and the Astronomer managed platform.
