proomt

Search

Search posts, papers, and topics

All posts

AntithesisZosia Collins4 min readintermediate

6.3x Faster Testing with Retroactive Logging

Summary

Antithesis developed "retroactive logging" to address the performance overhead of verbose debugging logs during deterministic fuzz testing. By initially sinking most logs within the hypervisor and replaying runs with full logging only on failure, they achieved 6.3x faster testing.

  • Verbose logging significantly slows down deterministic fuzz testing due to the overhead of moving and storing data.
  • Dynamically changing logging levels during a test run can break determinism, making bug reproduction impossible.
  • Retroactive logging uses an in-hypervisor "black hole" to sink most verbose logs, avoiding performance impact during initial runs.
  • Upon detecting a failure, the exact test run is deterministically replayed with the black hole disabled to retrieve full trace logs.

Engineers working with deterministic testing or debugging complex systems will find this approach valuable for efficiently capturing detailed logs without sacrificing test performance.

7/10

Related reading

  1. Stop restarting your tests

    This article proposes using Temporal to orchestrate large, flaky end-to-end test suites, treating each test execution as a durable unit of work. This approach allows retrying only failed individual tests or browser configurations, significantly reducing wasted CI time from transient failures.

    Temporaltemporal.io11 minHN1
  2. Your Agent Aced the Task. Will It Do It Again?

    The post introduces the Consistency Analyzer, a cheap black‑box diagnostic that flags flip‑prone decision steps in LLM agent traces, and shows how feeding the resulting consistency guidelines back into ALTK‑Evolve halves the gap between mean success and all‑run success (Pass⁵) on the AppWorld benchmark without hurting average accuracy.

    Hugging Facehuggingface.co8 minHN21
  3. Reversing Factorio's RNG

    The author reverse‑engineers Factorio’s RNG, confirms it’s the Boost taus88 generator (a 3‑LFSR XOR combo), proves equivalence with the game’s compiled code via decompilation and symbolic Python checks, and explains how the weak LFSR structure lets you predict future “random” quality outcomes.

    Hacker News front pagegithub.io61 minHN25248lobste.rs26