Search posts, papers, and topics
A year‑long, donation‑funded effort by Josh Bowman‑Matthews on the Servo project resulted in 8 new maintainers, 1150 PR reviews, 114 contributor‑focused issues (92 % resolved), extensive new documentation, and concrete stability work on the JS engine integration and flaky tests, plus support for a community grant.
Swift 6.4 adds Swift Build as the default SPM build system, stabilizes Subprocess (1.0), expands C++/Java interop, boosts WebAssembly performance (up to 40×), introduces new language features (optional some/any, @diagnose, module selectors, async defer, cancellation shields), adds non‑copyable array types, SBOM generation, and broader IDE support.
A systematic evaluation of LLM agents generating multi‑file backend code shows a sharp drop in correctness when structural constraints (framework conventions, ORM usage, API contracts) are added. Across 100 tasks in 8 Python web frameworks, assertion pass rates fall ~27 points, with data‑layer bugs (bad queries, ORM violations) driving most failures. Mid‑size models cope with minimal frameworks (…
A Rust veteran recounts porting a JSONPath library to Zig, highlighting differences in IDE support, project layout, testing ergonomics, functional vs imperative idioms, and Zig's explicit allocator model.
CodeMidas builds RL environments directly from open‑source code: agents explore a repo, infer a spec, generate tests from the original implementation, and filter tasks via execution checks. The pipeline yields 5,545 high‑quality coding tasks across 23 languages and 15 domains. Training the MiMo‑V2.5 agent with GRPO on this dataset improves benchmark scores by 8‑18% (e.g., DeepSWE +11.7%, ProgramB…
PACT is a new benchmark designed to measure how well enterprise LLM agents follow compliance rules, especially when under user pressure. It found substantial variability across 22 models, with even the strongest assistants misapplying rules 6-10% of the time, and user pressure increasing violation rates by 65% on average.
ScienceIDE is a framework that turns existing scientific software repositories into programmable environments that agents can use for task generation, execution, and verification. Training on these environments yields LLMs (PhAI‑IDE series) that outperform baselines on scientific code repair and several general code‑reasoning benchmarks.
The post argues that AI agents could eventually handle low‑level engineering tasks—bug fixing, debugging, UI consistency, growth experiments—if the dev toolchain is made “agent‑legible”. It outlines missing primitives (global memory, code‑base rot prevention, better dev environments) and proposes a bootstrapping process to measure and improve a repo’s “agent readiness”. The piece is largely specu…
A user praises Crap4J, an old Java code quality tool, for effectively identifying "dangerous" code without false positives, unlike other coverage tools. They note its age and incompatibility with Java 7, urging for updates.
CodeName One prepared for Android 17 (API 37) by addressing platform changes proactively, including fixing version number parsing and implementing the new system-rendered location button. They also added robust PEM key parsing and explicit task removal to simplify common security operations for app developers.
Evaluating AI coding agents requires a robust sandbox to prevent agents from retrieving answers from the environment, which can invalidate tests of internal knowledge. A correct answer doesn't guarantee a valid measurement if the agent accessed information it shouldn't have, highlighting the need to define sandboxes by information boundaries rather than just tool restrictions. Always review agent…
Lead is a Postgres full‑text search extension compatible with TIN, intended for CI and testing environments. It offers the same SQL and TINQL features but runs as a full table scan, making it orders of magnitude slower, so it’s only practical for small datasets.
DoorDash built a two‑phase, multi‑agent LLM system (Claude Sonnet orchestrator + Claude Opus cleanup agents) that automatically removes stale feature flags from 623 repos. In a 50‑flag pilot it generated usable PRs for 45 flags at ~14 min and $4.79 each, with 100 % success on simple flags and no regressions.
Spec‑driven development encodes architectural rules as machine‑readable Zod schemas, letting automated agents (e.g., Copilot, Claude Code) be checked for boundary violations, API drift, and runtime invariants via AST analysis and CI tests.
AI-generated code often passes basic CI but contains subtle bugs like vacuous tests or off-by-one errors. This article details a multi-tier CI/CD pipeline using AST analysis, property-based testing, and mutation testing to catch these issues and annotate GitHub PRs with actionable SARIF reports.
Agentic coding lets AI agents write, test, and submit code autonomously, shifting the bottleneck from writing to governing code in production. Enterprises face rising failures, unclear ownership, growing costs, and weakened controls, which require a unified pipeline visibility layer.
Phillip Mortimer argues that AI‑generated code makes all software effectively "write‑only" due to volume, and proposes managing this by treating tests as the sole specification, automating code reviews with LLMs, and decoupling intent from implementation.
Sebastián Passaro (Qubika) demonstrated a live RAG pipeline attack at Testear.la 2026, showing how a single poisoned document can hijack LLM outputs and trigger unsafe actions. He tied the demo to the OWASP LLM Top 10, highlighted open‑source tooling for finding such weaknesses, and advocated a defense‑in‑depth threat model for QA teams. The talk reframed AI from a testing aid to a security surfa…
This article advocates writing end-to-end tests in the backend's language to leverage domain models and persistence tools for efficient test data setup. It details a strategy for structuring E2E tests with inline data creation, small helpers, and parallel execution considerations.
The article shows how to replace raw TypeScript compiler output with a programmatic JSON feedback loop that merges diagnostics and Vitest test failures, feeding the structured data back to an LLM coding agent for self‑correction. It provides concrete code for extracting, enriching, validating, and de‑duplicating errors, and demonstrates token savings and deterministic iteration until compilation…
A step‑by‑step tutorial for building a deterministic AST‑based security scanner that runs in GitHub Actions to catch prototype‑pollution, high‑entropy secrets, and unauthorized network egress in AI‑generated pull requests.
A practical five‑step workflow for reviewing AI‑generated Python code, covering intent clarification, automated quality checks (linters, type checkers, security scanners, tests), risk‑first manual reading, a checklist of common AI mistakes, and fix‑verify loops, plus a ready‑to‑use dev‑environment setup.
Twilio built a two‑phase pipeline that first scores each error‑code doc with a deterministic script, then feeds only low‑scoring pages to an LLM (gpt‑5.4) using a strict prompt. The model’s diff is applied via git, linted, and opened as a PR for a reviewer. In 12 weeks they updated 1,138 pages (≈34.6 s/page) with a human reviewing every change, cutting manual effort from an estimated 18 k hours a…
A hands‑on benchmark of five LLM code‑review models (Claude Opus 5, GPT 5‑6 Sol, Grok 4.6, Kimi 3, GLM 5.3, Qwen 3.8 Max, DeepSeek v4 Pro) shows that all models ace tiny 15‑line diffs but diverge sharply on 100‑300‑line, cross‑file changes. The author built a deterministic harness (single HTTP call, fixed system prompt, JSON output) and measured detection rate, false‑positives, latency, token usa…
Helix is Shopify’s internal LLM‑driven framework for migrating React Native screens to native iOS/Android. It breaks a screen into tiny, reviewable checkpoints and forces each through four strict gates—behavior tests via a CLI, Gemini‑powered visual diff, two adversarial code reviewers, and a final engineer sign‑off—while remembering feedback to become more autonomous over time.
The page offers a 12‑question interactive quiz that checks knowledge of Python’s mutable vs immutable types, covering built‑in type categories, aliasing pitfalls, mutable default arguments, and shallow copy behavior.
Unified (jumbo) builds concatenate multiple C++ source files to reduce header instantiation, often halving cold compile time. However, they hurt incremental rebuilds, cache efficiency, and can change program semantics, making reproducibility and correctness harder.
Firefox DevTools MCP 0.10.3 is released, introducing a new `close_firefox_session` tool, full-page screenshot capability, and configurable download folders. This update also includes bug fixes and clarifies the usage of `restart_firefox` for controlling Firefox via WebDriver BiDi.
Fresh is an opinionated Laravel package skeleton that bundles testing tools, CI pipelines, a Testbench workbench app, and optional Vue 3 frontend and AI Boost skill support. It lets you generate a new package with a single composer command and includes version switchers, static analysis, and GitHub Actions out of the box.
This Jepsen blog post announces a retrospective talk on 13 years of testing distributed systems, linking to slides and a video. The post itself contains no detailed lessons or technical content.
