proomt

Search

Search posts, papers, and topics

All posts

Hacker News front pageBy Federico Viticci39 min readintermediate

M5 Ultra Mac Studio Review

Read on macstories.netHN 234·235 comments

Summary

The M5 Ultra Mac Studio (256 GB RAM) uses a quad‑die M5 Max architecture with an 80‑core GPU and 1.2 TB/s memory bandwidth, delivering ~70 % faster prompt‑to‑first‑token and token‑generation rates than the M3 Ultra. In the author’s tests Qwen3.8‑Flash‑Next hits 100 tokens/s on short prompts and 60‑85 tokens/s with 64‑256 KB context, making local AI agents (Open Minis, Hermes, Codex) feel snappy a…

  • Quad‑die M5 Max + UltraFusion gives 4.5× AI GPU compute vs M3 Ultra.
  • Memory bandwidth up from 819 GB/s to 1.2 TB/s (≈50 % increase).
  • Benchmark: ~70 % faster overall response time vs M3 Ultra; 100 tokens/s on short prompts, 60‑85 tokens/s with large context.
  • Enables always‑on local agents for research, note‑taking, and code assistance without cloud fees.

Local AI workloads are becoming practical on consumer hardware, and the M5 Ultra shows that a Mac can replace expensive GPU rigs for continuous, privacy‑preserving assistant tasks. The performance jump directly translates to smoother multi‑turn interactions and larger context windows, which are cri…

6/10
performancehardwaremacosapple siliconlocal ai

Related reading

  1. SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops

    SiliconBench benchmarks nine Apple‑Silicon LLM serving engines on unified‑memory desktops, measuring speed, memory usage, and output fidelity across Qwen3, Qwen3.5, and Gemma‑4 models. It finds vllm‑metal doubles throughput at modest concurrency, memory budgets often fail to preserve headroom, and only three stacks satisfy all fidelity and model‑coverage requirements, with tensor‑parallel scaling…

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. Shared Selective Persistent Memory for Agentic LLM Systems

    Apple proposes a memory architecture for agentic LLMs that selectively persists reusable context (specs, schemas, configs, constraints) across sessions and users. Shared workspaces with role‑based access and a zero‑token data‑refresh mechanism cut token usage by 97×, reduce task time by 14×, and raise task‑completion rates to 96% versus 71%‑79% for baselines.

    Apple Machine Learning Researchapple.com1 minpaper

  3. M4N VM family, now GA: Highest per-core IOPS and throughput for I/O and memory-bound workloads

    Google Cloud’s GA‑available M4N VM family pairs 5th‑gen Intel Xeon CPUs with Google’s custom Titanium offload to deliver up to 1 M IOPS, 25 GiB/s block‑storage throughput, 400 Gbps VM‑to‑VM bandwidth, and a 26 GB/vCPU memory ratio (up to 5.9 TiB RAM). The design targets memory‑bound, I/O‑intensive workloads (Oracle, SAP HANA, vector search, real‑time analytics) and claims >20 % TCO reduction for…

    Google Cloud Bloggoogle.com5 min

  5. tokenizers v1: encode, decode and scaling, measured

    Hugging Face has released `tokenizers` v1, a major performance update that achieves 3-30x faster encoding than v0.23 while maintaining identical output and API compatibility. Key optimizations include a SIMD-accelerated splitter, a thread-local word cache, and an allocation-free BPE merge loop, ensuring tokenization doesn't bottleneck ML workflows.

    Hugging Facehuggingface.co10 min

  6. From Megawatts to Tokens: How NVIDIA Maximizes AI Factory Production

    NVIDIA’s DSX platform lets AI data‑centers shift workloads in response to grid signals, squeezing ~24% more token throughput (4 M→5 M tps) and ~23% better performance‑per‑watt on a fixed megawatt budget. The first production demo used Emerald AI’s Conductor to drop a 4 MW load to 3 MW in under a minute without interrupting high‑priority jobs. DSX MaxLPS reallocates headroom across HGX B200 server…

    Nvidianvidia.com5 min