Related reading
Android Bench 2.0: Pushing the frontier with challenging long-horizon tasks
Android Bench 2.0 adds a set of long‑horizon tasks (multi‑day Android development problems) and introduces agent‑based evaluation. Scoring is now continuous, with the best model achieving a 28 % pass rate on these tasks, far lower than the ~91 % on earlier short tasks. The post lists new models on the leaderboard and points to updated methodology and GitHub repo.
Benchmarking Wild vs Mold
Reproduces Mold’s linker benchmarks on a 16‑core Ryzen, shows that configuration (filesystem, delete‑output, fork) explains most of the Wild vs Mold speed gap, and notes recent Mold releases and upcoming Wild tweaks that close the gap.
CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design
CADWorld is a new benchmark suite of 200 long‑horizon mechanical CAD tasks in FreeCAD, covering sketching, part modeling, assembly, CAM, FEM, and more. Agents interact via screenshots and GUI actions; success is checked by executable validation of the saved CAD artifacts. Seven existing agents achieve at most 17.5 % success versus an 87 % expert baseline, highlighting the gap between GUI competen…
Hugging Face Daily Papersarxiv.org1 minpaperRun Terminal-Bench and other Harbor evals on Vercel Sandbox
Vercel now supports running Harbor benchmarks, like Terminal‑Bench, inside its Sandbox using Firecracker microVMs. The integration with AI Gateway lets you benchmark multiple models by swapping the `--model` flag, scaling concurrency with `--n-concurrent`.
Vercelvercel.com1 minreleaseSiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops
SiliconBench benchmarks nine Apple‑Silicon LLM serving engines on unified‑memory desktops, measuring speed, memory usage, and output fidelity across Qwen3, Qwen3.5, and Gemma‑4 models. It finds vllm‑metal doubles throughput at modest concurrency, memory budgets often fail to preserve headroom, and only three stacks satisfy all fidelity and model‑coverage requirements, with tensor‑parallel scaling…
Hugging Face Daily Papersarxiv.org1 minpaperSome things Veloren does differently
Veloren’s engine uses a custom ECS, a novel ‘chonks’ voxel storage, full‑world pre‑generation, physically‑based erosion, and a scalable real‑time simulation (rtsim) that keeps tens of thousands of NPCs active, achieving ~50% CPU usage on a 48‑core server with 500+ players.
