proomt

Search

Search posts, papers, and topics

All posts

CodeName OneShai Almog20 min readintermediate

Lies, Damn Lies and Benchmarks

Summary

Codename One engineers dissect why benchmark numbers can be misleading, then share concrete work on GC tuning, proper weak/soft references, and a new probing sequence for their open‑addressed HashMap that cuts miss‑probe counts from >16 k to ~1.5 per lookup.

  • Benchmarks that only measure happy‑path cases (e.g., existing‑key lookups) can hide pathological behavior such as massive probe counts for missing keys.
  • Lowering the GC trigger floor from 24 MB to 38 MB RSS reduced resident memory without hurting throughput; the default live‑set estimate was incomplete, so a simple compile‑time constant now lets deployments pick a lower…
  • Implemented true weak and soft references in ParparVM; soft references now age‑out based on last successful get(), improving cache hit‑rate to 97.44 % at 82 MB RSS versus 87.99 % at 91 MB with the previous pressure‑base…
  • Replaced the default linear‑probing sequence in Codename One’s HashMap with CPython‑style perturbation probing, preserving the fast first‑probe for dense keys while reducing miss probes from ~16 k to ~1.5 and cutting mi…

Real‑world mobile apps care about startup latency, memory footprint, and smooth UI rendering. Mis‑leading benchmarks can lead developers to ship apps that perform well in synthetic tests but stall on edge cases. The article’s concrete GC and hashmap improvements directly translate to lower RAM usag…

7/10

Related reading

  1. Faster Maps: Chasing Swiss Speed

    ParparVM’s HashMap suffered catastrophic miss latency due to linear probing on dense integer keys. By adopting CPython‑style perturbed probing (Swiss‑table style) and extending tagged immediate values to more primitives, miss latency dropped from 32 s to ~45 ms, allocation pressure fell dramatically, and overall performance stayed roughly flat despite a modest hit‑time slowdown.

    CodeName Onecodenameone.com8 min
  2. What Go Taught Us About Java Garbage Collection

    ParparVM’s GC was tuned by lowering the allocation‑trigger floor, adding configurable thresholds, parallel marking, mutator assistance, and proper weak/soft reference handling. These changes cut RSS from 98 MB to 38 MB, reduced worst‑case GC pauses from seconds to sub‑second, and improved cache hit rates with a recency‑based eviction policy.

    CodeName Onecodenameone.com7 min
  3. Five models pass the easy code reviews. Then the diffs get hard.

    A hands‑on benchmark of five LLM code‑review models (Claude Opus 5, GPT 5‑6 Sol, Grok 4.6, Kimi 3, GLM 5.3, Qwen 3.8 Max, DeepSeek v4 Pro) shows that all models ace tiny 15‑line diffs but diverge sharply on 100‑300‑line, cross‑file changes. The author built a deterministic harness (single HTTP call, fixed system prompt, JSON output) and measured detection rate, false‑positives, latency, token usa…

    Bjørn Johansenbjornjohansen.com16 min
  4. I expected better from Google

    The authors of the open‑source project mobile‑use discovered that Google’s Artemis repository contains large blocks of identical Python code, examples, and even the same agent name without any attribution. They document the exact file diffs, the removal of their names via a force‑push, and the omission of their benchmark results from a public leaderboard. The post argues that this violates Apache…

    Lobstersminitap.ai6 minHN18032lobste.rs87