proomt

Search

Search posts, papers, and topics

All posts

Google Cloud BlogWill Ochandarena3 min readrelease notesintermediate

Accelerating the borderless Lakehouse: Announcing preview of cross-cloud caching

Read on google.com

Summary

Google Cloud previewed cross‑cloud caching for its Borderless Lakehouse. The feature caches sub‑file Parquet blocks in Google Cloud, encrypts them with GMEK, isolates cache per tenant/region, and validates freshness via metadata checks. In tests it can reduce cross‑cloud data transfer to <5% of the original size, lowering query latency and cost for Iceberg tables stored in other clouds. BigQuery…

  • Cache operates at column‑chunk granularity, avoiding full‑file transfers for columnar queries.
  • Cached blocks are encrypted at rest with Google‑managed keys and isolated by project/region.
  • Freshness is enforced by checking remote object metadata; stale blocks expire automatically.
  • Google claims typical cross‑cloud queries now transfer under 5% of the source data.

Cross‑cloud analytics often bottleneck on egress costs and latency. By caching only the needed column chunks locally, organizations can run federated queries without replicating entire datasets, making multi‑cloud data lakes more cost‑effective and responsive for BI and AI workloads.

6/10
storagebigqueryicebergcross cloudcaching

Related reading

  1. Elastic announces GA of cross-project search on Serverless, enabling teams to query across all linked projects without moving a byte

    Elastic Cloud Serverless now offers General Availability of Cross‑Project Search (CPS), letting you run a single query across up to 100 linked projects (any region, cloud, or solution type) without moving data. Setup is UI‑driven, permissions are evaluated per‑project, and costs are $0.009 / GB retained plus $0.05 / GB egress. CPS works for Observability, Security, Search, vector DB, and ML jobs,…

    Elasticelastic.co6 minrelease

  2. Introducing Filestore agent volumes: fully managed storage for agent workspaces

    Google Cloud adds Filestore agent volumes, a fully‑managed, elastic file‑system that automatically provisions isolated POSIX workspaces for GKE‑based AI agent sandboxes. Volumes attach in milliseconds, support RWX with file‑level locking, and charge only for used capacity with automatic tiering, aiming to cut cold‑start latency and storage waste for large‑scale agent fleets.

    Google Cloud Bloggoogle.com4 min

  3. Replica-aware routing public beta

    Replica‑aware routing (public beta) lets ClickHouse Cloud users pin a query stream to a specific replica by sending a custom header (HTTP) or overriding the TLS SNI (native). The proxy (Envoy) hashes the tag and consistently forwards all requests with the same tag to that replica, giving read‑after‑write consistency for temporary tables, session objects, and warm replica caches. Stickiness is bes…

    ClickHouseclickhouse.com6 min

  4. CloudBees vs Harness: Why Migration Isn't the Fix

    The article argues that Harness’s “free migration” offer hides significant downstream costs (training, pipeline rebuilds, compliance recertification) and that even after migration you still lack unified governance across heterogeneous CI/CD tools. CloudBees positions its Unify control plane as a tool‑agnostic layer that adds visibility, continuous governance, AI‑driven test selection, and hybrid…

    Codeshipcloudbees.com5 min
  5. 1 points

    Saving another 100TB of RAM with math (and Rust)

    Cloudflare reduced the memory footprint of its Pingora Backend Router by re‑examining the consistent‑hashing implementation in the pingora‑ketama library. By increasing the number of virtual hash points per server from the default 1 to the standard 160 (and applying weighted hashing based on disk capacity), they cut the per‑node overhead enough to reclaim >100 TB of RAM across the fleet. The post…

    Hacker News front pagecloudflare.com13 minHN478120lobste.rs33

  6. M4N VM family, now GA: Highest per-core IOPS and throughput for I/O and memory-bound workloads

    Google Cloud’s GA‑available M4N VM family pairs 5th‑gen Intel Xeon CPUs with Google’s custom Titanium offload to deliver up to 1 M IOPS, 25 GiB/s block‑storage throughput, 400 Gbps VM‑to‑VM bandwidth, and a 26 GB/vCPU memory ratio (up to 5.9 TiB RAM). The design targets memory‑bound, I/O‑intensive workloads (Oracle, SAP HANA, vector search, real‑time analytics) and claims >20 % TCO reduction for…

    Google Cloud Bloggoogle.com5 min