1
Frontier AI on Your Own Hardware
Tim Dettmers’ lab releases an open‑source stack that lets a single‑GPU desktop run 125‑B‑parameter LLMs (Qwen 3.8 Flash Next) and even a 550‑B model (DeepSeek V4.1) with 1.5‑bit quantisation, 450 t/s token throughput, and an auto‑compaction system (CliffCompaction) that halves inference cost and enables multi‑million‑token agent sessions. The stack combines an agent harness, a fast inference fram…
Hacker News front pagetimdettmers.com17 minHN11660