proomt

Search

Search posts, papers, and topics

All posts

Moove-itMaggie Etulain4 min readintermediate

Poisoned Documents, Real Risks: Sebastián Passaro Puts AI’s Weakest Link to the Test at Testear.la 2026

Read on qubika.com

Summary

Sebastián Passaro (Qubika) demonstrated a live RAG pipeline attack at Testear.la 2026, showing how a single poisoned document can hijack LLM outputs and trigger unsafe actions. He tied the demo to the OWASP LLM Top 10, highlighted open‑source tooling for finding such weaknesses, and advocated a defense‑in‑depth threat model for QA teams. The talk reframed AI from a testing aid to a security surfa…

  • RAG pipelines inherit the same trust‑boundary issues as any external data source – a single malicious document can corrupt model responses.
  • OWASP LLM Top 10 provides a practical checklist for identifying common LLM attack vectors, including prompt injection and data poisoning.
  • Open‑source scanners can automate the discovery of vulnerable documents before attackers do; integrate them into CI/CD for continuous monitoring.
  • Defence‑in‑depth for AI‑augmented QA means layered safeguards: input validation, retrieval filtering, model‑level guardrails, and post‑generation audits.

As RAG becomes the de‑facto method for injecting enterprise knowledge into LLMs, the attack surface expands dramatically. A poisoned internal document can cause misinformation, policy violations, or even trigger downstream automation, exposing organizations to compliance and security risks. Embeddi…

6/10
devopsllmsecuritycloudaitesting

Related reading

  1. Changing the game: How Google uses agentic AI to secure hundreds of millions of lines of code

    Google’s AI & Infrastructure team built an agentic pipeline (Mantis) that runs pre‑submit AI‑driven scans on every code check‑in, validates findings with a fast triage agent (AST + call‑graph analysis) achieving >92% precision in <1 min, then auto‑generates fixes via a bug‑fix agent. Localized threat models and a two‑step scan cut false‑positives to ~3% and prevent hundreds of vulnerabilities eac…

    Google Cloud Bloggoogle.com4 min

  2. Agentic Skill Decay

    Addy Osmani warns that AI agents can short‑circuit the hands‑on practice (“reps”) that builds deep expertise and judgment. He recommends deliberately inserting hypothesis‑forming, “why” questioning, diff inspection, failure prediction, and occasional manual coding into the workflow, especially for junior engineers. A 2026 Anthropic study showed junior developers using AI scored 17 % lower on a fo…

    Addy Osmaniaddyosmani.com16 min

  3. Cloud CISO Perspectives: How Google monitors AI threats and advances AI defenses

    Google’s Threat Intelligence team outlines three AI‑driven shifts—software build changes, expanded attack surface, and enhanced threat capabilities—then describes their multi‑model, graph‑based defense stack (AI Threat Tracker, in‑editor “spellcheck”, Wiz Security Graph, Gemini‑powered AI Threat Defense) and concrete threat examples like supply‑chain poisoning, LLMJacking, and AI‑orchestrated cre…

    Google Cloud Bloggoogle.com11 min

  4. How we AI-updated 1,138 Error-code Docs Pages

    Twilio built a two‑phase pipeline that first scores each error‑code doc with a deterministic script, then feeds only low‑scoring pages to an LLM (gpt‑5.4) using a strict prompt. The model’s diff is applied via git, linted, and opened as a PR for a reviewer. In 12 weeks they updated 1,138 pages (≈34.6 s/page) with a human reviewing every change, cutting manual effort from an estimated 18 k hours a…

    Twiliotwilio.com11 min

  5. Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems

    Emergence World is a continuously running multi‑agent sandbox used to stress‑test frontier LLM‑based agents over weeks. Eight parallel worlds (seven homogeneous, one mixed) generated 850 k LLM calls and ~50 B tokens while agents pursued goals, used tools, and maintained persistent memory. The authors injected three adversarial events—prompt injection, misinformation, and private‑memory exposure—a…

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. Prompts aren’t Real

    The talk argues that prompt engineering is a dead‑end and proposes building large evaluation/optimization pipelines (pass^k testing, adversarial scenario generation, automated prompt optimization) to make LLM agents reliable. It describes a workflow: generate tests, run them with/without a new “skill”, feed results to a genetic optimizer that mutates prompts, validate on hold‑out tests, and itera…

    Hacker News front pageevaluation.club24 mintalkHN11757