Search posts, papers, and topics
Numberwang is a tiny character‑level CNN packaged as a 1.8 MB JSON model and a ~100‑line pure‑Python inference script that classifies numeric strings in eleven languages without any external dependencies. It reaches 88.9 % macro‑F1 on a held‑out set, but struggles with unseen arithmetic expressions.
Uber developed a context-aware mechanism to prevent retry storms in deep microservice dependency chains. It introduces "error ownership" where services claim errors they originate and unclaim errors they propagate, allowing upstream callers to make informed retry decisions and avoid amplifying load on already struggling services.
The authors reverse‑engineer AMD’s CDNA matrix cores, build per‑architecture MATLAB models that match hardware bit‑for‑bit on 10 M random tests, and use them to compare AMD vs NVIDIA tensor‑core accuracy.
The default GitHub actions/setup-go action degrades parallel Go CI performance due to incomplete cache keys and stale writes. A new open-source action, cloudx-io/setup-go, resolves this with job-specific and run-ID-based keys, cutting test job runtimes by 69%.
Stars!VM is a 32‑bit Windows executable that embeds a custom 80286 emulator and a Win16‑to‑Win32 bridge to run the 1995 4X game Stars! without any external emulators. The author adds differential fuzzing, hot‑routine patching, I/O buffering and asset compression, achieving about a 2× speedup in turn generation.
xAI announced Grok 4.7, a new LLM for coding and knowledge work. The post claims it’s twice as fast and half the price of comparable models, and shows a CursorBench 4.0 chart with a 46.3 % “Extra High” score at $6.01 average cost per task. No architecture details, training data, or engineering rationale are provided.
The author built a Git server on object storage (objgit) and found standard Git packfiles inefficient due to random‑access and round‑trip latency. By designing a columnar, object‑storage‑native packfile format that can be accessed via HTTP range requests, they achieved acceptable performance for large repositories without client‑side changes.
The post shares a live dashboard that displays overview metrics about reconnecting. No further details are provided.
Video DeltaNet (VDN) replaces full‑softmax attention in video diffusion models with a hybrid: per‑frame local Softmax for fine detail and a bidirectional linear memory (Video Delta Attention) for long‑range context. A teacher‑alignment schedule injects the linear branch into a pretrained MiniMax H3 model, preserving Softmax for text/audio streams. On eight NVIDIA B200 GPUs VDN‑H3 denoises a 14.3‑…
Fathom introduces a per-query read depth mechanism for sparse decoding over offloaded KV caches, allowing each query to adaptively decide how many bits of each key channel to read. This method significantly speeds up decoding for large language models with long contexts by reducing host memory traffic, achieving 1.67x faster GPU decoding on Qwen3-8B at one million tokens.
A Show HN project that presents a flight‑simulator UI where the user is only a passenger. No technical details, design rationale, or code are provided in the article.
Go 1.27 adds a set of span‑class‑specific malloc functions for allocations ≤ 80 bytes. By generating a tiny, constant‑size allocator per span class the runtime can inline size‑dependent work (e.g. zero‑clear) and skip span‑class lookup, yielding 20‑30 % faster small allocations and ~1 % overall speed‑up for allocation‑heavy programs. The implementation is generated automatically via an AST inline…
The Copilot agent runtime was rewritten from a 130‑k‑line TypeScript/Node.js codebase into a native Rust library (~830 k lines of Rust) to reduce startup latency, memory use, and improve reliability. The migration was done incrementally (in‑place) across 128 PRs, with AI‑generated code handling most of the work. The new runtime exposes a C ABI for in‑process embedding by all six Copilot SDK langu…
When2Think introduces a post‑training framework that lets a large reasoning model decide per‑instance how much reasoning depth to allocate, using difficulty‑aware reward shaping (IDAC) and verifier rewards. It cuts token usage by ~28% while boosting Pass@3 by 10% on AIME24 and reaches 40% Pass@3 on AIME25, outperforming compression and routing baselines.
The paper presents Edge0, a streaming MoE inference engine that predicts the next layer's routing one token ahead, allowing expert weights to be fetched from SSD while compute proceeds. This enables a 35 B‑parameter MoE to run on a single 24 GB machine at ~20 tokens/s using only ~3 GiB of active memory and with near‑teacher accuracy.
Increasing the number of LLM candidates (N) improves reasoning accuracy, but the way those candidates are generated (batch size vs sequential calls) dramatically affects latency, GPU‑hours, and energy. On A100 GPUs, eight serial 1‑candidate calls consume ~5× more energy and take ~6× longer than a single 8‑candidate batched call, even though total candidate count is identical. The authors recommen…
VC-Attention introduces a training‑free low‑bit attention pipeline for diffusion transformers. It smooths value tensors via lightweight online clustering (V‑Smooth) and quantizes only the residual after subtracting block means, restoring the mean from the softmax row sum. It also replaces the FP32 softmax exponential with a fused FP8 cast (ExpCast‑FP8) that maps log‑scores directly to E4M3 probab…
A detailed, personal review of Paul McKenney’s free online textbook on parallel programming. The author, coming from a TLA⁺/distributed‑systems background, finds the early chapters excellent for building intuition about CPU caches, memory ordering, and false‑sharing, but notes gaps (e.g., shallow coverage of C++11 atomics and MESI). The review is concrete, cites specific chapters, and offers prac…
SiliconBench benchmarks nine Apple‑Silicon LLM serving engines on unified‑memory desktops, measuring speed, memory usage, and output fidelity across Qwen3, Qwen3.5, and Gemma‑4 models. It finds vllm‑metal doubles throughput at modest concurrency, memory budgets often fail to preserve headroom, and only three stacks satisfy all fidelity and model‑coverage requirements, with tensor‑parallel scaling…
This paper investigates the "lossless" claim of Orthrus, a hybrid architecture for accelerating LLM inference. It finds that under BF16 precision, Orthrus diverges from the exact autoregressive output trajectory in over 50% of cases, though FP32 maintains exact matching. Despite BF16 divergence, downstream task performance was not systematically degraded.
Vercel reports that the Jev decision model was adopted by 13% of paid teams within its first day, outpacing prior model launches. Jev claims to be up to 194× faster and 445× cheaper than general‑purpose LLMs while returning structured, probabilistic decisions.
Vercel AI Gateway now offers Jev, a probabilistic decision model that returns typed choices, scores, and booleans instead of raw text. TypeSafe AI reports it runs up to 193× faster and 445× cheaper than standard LLMs, exposed via the experimental evaluate API in AI SDK 7.
Spotify’s AI‑assisted development doubled change volume, exposing gaps in alerting, capacity planning, fleet‑update safety, and mobile quality signals. The team added end‑to‑end monitoring, priority‑based tiering, stronger rollback/observability, and expanded edge capacity. Data shows AI‑generated code isn’t a direct incident cause, but verification pipelines must scale with velocity.
Retrieve-for-Train addresses LLM inference bottlenecks in complex AI search by using offline reinforcement learning to train a lightweight diffusion model. This enables efficient, single-pass generation of diverse, property-aligned sub-queries, bypassing slow autoregressive reasoning.
Atlassian replaced its decade‑old gostatsd‑based metrics pipeline with a fully OpenTelemetry‑based stack by keeping the StatsD‑UDP contract on the client side and swapping in purpose‑built OTel Collector distributions for collection, ingest, aggregation, and forwarding. The migration was done incrementally, saved ~3.9% CPU per service, cut sidecar cost ~30% fleet‑wide, halved aggregation CPU, and…
ModularRSI proposes a contrastive, modular approach to evolve agent harnesses by independently updating five functional modules and aggregating evidence across 2,000 benchmark‑disjoint tasks. Experiments on TB2.0 and SWE‑Bench Verified show consistent gains on unseen tasks and cross‑model transfer.
Hugging Face has released `tokenizers` v1, a major performance update that achieves 3-30x faster encoding than v0.23 while maintaining identical output and API compatibility. Key optimizations include a SIMD-accelerated splitter, a thread-local word cache, and an allocation-free BPE merge loop, ensuring tokenization doesn't bottleneck ML workflows.
Bun, the JavaScript runtime, rewrote its 535K lines of Zig code into Rust in four months using an AI-orchestrated process. This eliminated numerous memory leaks and improved performance, achieving a task estimated to take human engineers a year.
Vercel’s September AI Gateway Production Index shows open‑weight models processing 56% of token volume (up from 7% in Dec 2025) while accounting for only 14% of spend. Token price fell 23.2% month‑over‑month. Anthropic’s Opus 5 captured 22.5% of spend, overtaking Fable 5 which dropped to 4.9%. OpenAI’s new GPT‑6 Astra grabbed ~7.7% of total gateway spend in its first 12 days, more than double Ant…
Apple proposes a memory architecture for agentic LLMs that selectively persists reusable context (specs, schemas, configs, constraints) across sessions and users. Shared workspaces with role‑based access and a zero‑token data‑refresh mechanism cut token usage by 97×, reduce task time by 14×, and raise task‑completion rates to 96% versus 71%‑79% for baselines.
