Search posts, papers, and topics
Hugging Face has released `tokenizers` v1, a major performance update that achieves 3-30x faster encoding than v0.23 while maintaining identical output and API compatibility. Key optimizations include a SIMD-accelerated splitter, a thread-local word cache, and an allocation-free BPE merge loop, ensuring tokenization doesn't bottleneck ML workflows.
MiniCPM5-2B, a 2.6 B‑parameter dense Llama‑style model, tops the Artificial Analysis GDPval‑AA v2 benchmark (831 Elo) and sits on the Pareto frontier of the Intelligence Index v4.2 despite being far smaller than competing models. The post breaks down its capability density, token‑cost efficiency, architecture, and three‑stage training (SFT, RL, on‑policy distillation) to explain why it outperform…
Vercel now supports running Harbor benchmarks, like Terminal‑Bench, inside its Sandbox using Firecracker microVMs. The integration with AI Gateway lets you benchmark multiple models by swapping the `--model` flag, scaling concurrency with `--n-concurrent`.
