proomt

Search

Search posts, papers, and topics

All posts

VercelAmelia Charles6 min readintermediate

Open-weight models take 56% of token volume, Astra doubles Fable 5.1 spend

Summary

Vercel’s September AI Gateway Production Index shows open‑weight models processing 56% of token volume (up from 7% in Dec 2025) while accounting for only 14% of spend. Token price fell 23.2% month‑over‑month. Anthropic’s Opus 5 captured 22.5% of spend, overtaking Fable 5 which dropped to 4.9%. OpenAI’s new GPT‑6 Astra grabbed ~7.7% of total gateway spend in its first 12 days, more than double Ant…

  • Open‑weight (open‑source) models now dominate token volume but remain a small fraction of spend, indicating lower per‑token cost.
  • Average token price across the gateway fell 23.2% YoY, giving teams more inference for the same budget.
  • Anthropic’s mid‑tier Opus 5 gained market share by offering roughly half the price of Fable 5 with comparable capability.
  • Google’s Gemini 3 Flash saw massive churn, with >75% of its lost volume moving to other labs, highlighting the importance of price‑performance balance.

The shift toward open‑weight models and aggressive price competition reshapes how enterprises allocate inference budgets, favoring cost‑effective models for bulk workloads while reserving frontier models for high‑value tasks. Rapid adoption of new models (e.g., Astra) demonstrates that pricing pari…

6/10

Related reading

  1. GLM 5.3 FlashX now available on AI Gateway

    Vercel AI Gateway now offers the GLM‑5.3‑FlashX model, a fast (~200 tps) multimodal coding LLM. The post includes a TypeScript streaming example, notes the model’s fit for coding agents and interactive tools, and lists AI Gateway features (unified API, usage tracking, retries/failover, custom reporting, key budgets, routing rules) with no platform fee.

    Vercelvercel.com1 minrelease
  2. From Megawatts to Tokens: How NVIDIA Maximizes AI Factory Production

    NVIDIA’s DSX platform lets AI data‑centers shift workloads in response to grid signals, squeezing ~24% more token throughput (4 M→5 M tps) and ~23% better performance‑per‑watt on a fixed megawatt budget. The first production demo used Emerald AI’s Conductor to drop a 4 MW load to 3 MW in under a minute without interrupting high‑priority jobs. DSX MaxLPS reallocates headroom across HGX B200 server…

    Nvidianvidia.com5 min
  3. M5 Ultra Mac Studio Review

    The M5 Ultra Mac Studio (256 GB RAM) uses a quad‑die M5 Max architecture with an 80‑core GPU and 1.2 TB/s memory bandwidth, delivering ~70 % faster prompt‑to‑first‑token and token‑generation rates than the M3 Ultra. In the author’s tests Qwen3.8‑Flash‑Next hits 100 tokens/s on short prompts and 60‑85 tokens/s with 64‑256 KB context, making local AI agents (Open Minis, Hermes, Codex) feel snappy a…

    Hacker News front pagemacstories.net39 minHN191174
  4. Jev is the fastest-adopted model in AI Gateway history

    Vercel reports that the Jev decision model was adopted by 13% of paid teams within its first day, outpacing prior model launches. Jev claims to be up to 194× faster and 445× cheaper than general‑purpose LLMs while returning structured, probabilistic decisions.

    Vercelvercel.com1 minHN21
  5. Reimagining advertising with AI

    OpenAI announced new AI‑driven features for ChatGPT Ads: Sponsored Agents that let users chat with business‑run agents after clicking an ad, AI‑assisted copy and image generation in Ads Manager, AI‑powered text customization and translation, plus integrations with HubSpot and Shopify for campaign creation and lead tracking. The features are in limited US testing and aim to streamline ad creation…

    OpenAIopenai.com3 minreleaseHN156180