proomt

Search

Search posts, papers, and topics

All posts

SitePoint16 min readtutorialintermediate

Cutting Node.js Memory Footprint with HyperLogLog and Count-Min Sketch in TypeScript

Read on sitepoint.com

Summary

A step‑by‑step tutorial showing how to implement HyperLogLog and Count‑Min Sketch in pure TypeScript for Node.js, with a fast MurmurHash3‑x86‑32, memory‑bounded registers, merge support, and a Vitest benchmark suite that runs under `--expose-gc`.

  • A 32‑bit MurmurHash3 implementation with reusable TextEncoder and `Math.imul` for speed.
  • HyperLogLog class using configurable precision (p = 14 → 16 KB, ~0.8 % error) with correct small‑/large‑range corrections and register‑wise merge.
  • Count‑Min Sketch (not shown in excerpt) built on the same hash pair via Kirsch‑Mitzenmacher technique.
  • Benchmarks demonstrate sub‑megabyte memory usage versus hundreds of megabytes for native `Set`/`Map` at millions of keys.

Node.js services that ingest millions of events per hour quickly run out of heap when using exact containers. Probabilistic sketches give bounded memory and predictable GC, enabling high‑throughput streaming analytics without scaling containers.

6/10
typescriptperformancenodejsprobabilistic data structureshyperloglogcount min sketch

Related reading

  1. Migrating the GitHub Copilot runtime to Rust, using Copilot

    The Copilot agent runtime was rewritten from a 130‑k‑line TypeScript/Node.js codebase into a native Rust library (~830 k lines of Rust) to reduce startup latency, memory use, and improve reliability. The migration was done incrementally (in‑place) across 128 PRs, with AI‑generated code handling most of the work. The new runtime exposes a C ABI for in‑process embedding by all six Copilot SDK langu…

    GitHub Oldgithub.blog65 minHN188

  2. Eliminating AI Code Hallucinations with TypeScript Compiler Diagnostics and TDD Loops

    The article shows how to replace raw TypeScript compiler output with a programmatic JSON feedback loop that merges diagnostics and Vitest test failures, feeding the structured data back to an LLM coding agent for self‑correction. It provides concrete code for extracting, enriching, validating, and de‑duplicating errors, and demonstrates token savings and deterministic iteration until compilation…

    SitePointsitepoint.com15 min
  3. 1 points

    Saving another 100TB of RAM with math (and Rust)

    Cloudflare reduced the memory footprint of its Pingora Backend Router by re‑examining the consistent‑hashing implementation in the pingora‑ketama library. By increasing the number of virtual hash points per server from the default 1 to the standard 160 (and applying weighted hashing based on disk capacity), they cut the per‑node overhead enough to reclaim >100 TB of RAM across the fleet. The post…

    Hacker News front pagecloudflare.com13 minHN478120lobste.rs33

  6. TanStack Charts Introduced with a Framework Agnostic Grammar of Graphics for TypeScript

    TanStack Charts α is a framework‑agnostic, TypeScript‑first charting library that implements a grammar‑of‑graphics API (marks, scales, channels, transforms, layers). It ships as a single npm package with adapters for React, Preact, Vue, Solid, Svelte, Angular, Lit, and vanilla DOM, and can run on server or client. The runtime is headless; type inference stays tied to the source row, so each mark…

    InfoQinfoq.com2 minrelease