proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersSebastián Andrés Cajas Ordóñez, Maximin Lange, Quang Bui1 min readpaperadvanced

ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

Read on arxiv.org

Summary

ModaLens introduces a paired image-swap audit to measure how radiology report availability affects image sensitivity in medical VLMs. It found that MedGemma-27B's answers changed significantly more often when the image was swapped if the report was not available, indicating reports reduce image reliance.

  • ModaLens uses a paired image-swap audit to assess medical VLM image sensitivity.
  • Report availability reduces a medical VLM's sensitivity to image changes by 16.7 percentage points.
  • MedGemma-27B showed 4.26% answer changes with report vs. 20.94% without report on image swaps.
  • The audit involves swapping images while keeping the question and report fixed.

This work is crucial for understanding and improving the reliability of medical VLMs, ensuring they leverage visual information appropriately rather than solely relying on textual reports.

8/10
llmmachine learningevaluationcomputer visionreliabilitymedical ai

Related reading

  1. UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation

    UFO introduces an Atomized Chain‑of‑Evaluation (AEU) framework that breaks omni‑condition alignment in multi‑modal image generation into a sequential set of fine‑grained checks, achieving a 15.25 % boost in correlation with human judgments. The authors also release UFO‑Bench, a benchmark for testing how well models satisfy combined textual and visual conditions.

    Hugging Face Daily Papersarxiv.org1 minpaper

  5. Article: Architecting Secure and Scalable Facial Verification Systems

    A real‑world post‑mortem of a high‑volume face verification service that moved from a naïve synchronous API to an async, layered pipeline (edge validation, preprocessing, decoupled detection/verification, decision engine) to achieve 8.5k rpm, p99 < 1.8 s, 30 % cost savings, and strict privacy controls.

    InfoQinfoq.com15 min

  6. How Value Induction Reshapes LLM Behaviour

    Apple researchers fine‑tune LLMs on curated subsets of value‑oriented preference data and measure cross‑value effects, safety, and anthropomorphic language. They find value induction propagates to related (and sometimes opposing) values, improves safety for positive values, but universally boosts validating, sycophantic language.

    Apple Machine Learning Researchapple.com1 minpaper