Hugging Face Daily PapersHanoona Rasheed, Mohammed Irfan Kurpath, Bin RenSep 28, 20261 min readpaperadvanced

Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision

Summary

This paper evaluates GPT-6 Astra and five other frontier general-purpose AI systems across 34 computer vision capabilities and 55 benchmarks. It finds these systems excel at semantic interpretation and reasoning, but struggle with metric geometric accuracy, faithful reconstruction, and fine-grained specialized knowledge.

GPT-6 Astra demonstrates substantial gains in visual/spatial reasoning and structured prediction over other frontier systems.
General-purpose AI systems now approach or reach reference levels for semantic interpretation, reasoning, and object-centric prediction.
Significant performance gaps remain for tasks requiring metric geometric accuracy and faithful reconstruction.
Temporally consistent dense prediction and specialized fine-grained visual knowledge are still challenging for these models.

Computer vision engineers and researchers should care as it maps the current strengths and weaknesses of frontier general-purpose AI systems, guiding future development and application choices.

8/10

llm machine learning evaluation multimodal computer vision deep learning

Related reading

GPT-6 Astra Is the First Model OpenAI Classifies as Critical for Cybersecurity infoq.com
OpenAI labeled GPT‑6 Astra as “Critical” for cybersecurity under its Preparedness Framework – the first model to meet that bar. In controlled tests the model autonomously discovered zero‑day bugs in a browser and an OS kernel, building working exploit chains in 29 h (browser) and 12 h (kernel). A benchmark of post‑cutoff vulnerabilities confirmed its ability to find unknown flaws. OpenAI reports…
InfoQinfoq.comSteef-Jan Wiggers19d3 min
In-Context Robot Learning with VLM Agents arxiv.org
GPT‑Policy is a framework that lets a large vision‑language model (e.g. GPT‑6 Astra) perform in‑context robot learning: a context compiler extracts visual transitions from demos, the VLM proposes tool actions, and a constrained controller verifies and executes them. Real‑robot experiments show that raw video demos improve success rates even without explicit action labels, and that providing align…
Hugging Face Daily Papersarxiv.orgDongzhou Cheng, Taoran Yi, Ye Fang20d1 minpaper
PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop arxiv.org
This paper introduces PhysVista, a new benchmark to evaluate the physical intelligence of Vision-Language Models (VLMs) using a closed perception-reasoning-assessment loop. Experiments reveal significant limitations in current VLMs' physical reasoning and plausibility assessment capabilities.
Hugging Face Daily Papersarxiv.orgXinge Peng, Yiting Lu, Tianwu Zhi6d1 minpaper
VisionHOPE: Visual Backbones as Self-Modifying Learning Systems arxiv.org
VisionHOPE introduces a visual backbone that updates its own parameters on‑the‑fly using five coupled memories, with a stability‑matched step‑size scheme that keeps updates non‑expansive. The approach attains competitive ImageNet, COCO and ADE20K performance, showing self‑modifying learning systems can serve as practical general‑purpose vision models.
Hugging Face Daily Papersarxiv.orgSiran Peng, Tianshuo Zhang, Tianyu Fu9d1 minpaper
TAPe+ML: A Compact Structured Representation for Multi-Task Computer Vision arxiv.org
TAPe+ML v3 is a compact multi‑task vision system that replaces raw‑pixel processing with a structured TAPe representation. Using <100 k parameters, it achieves 84.7 mAP50 (65.3 mAP50‑95) on COCO detection, 80.7 mask mAP50 (58.4 mask mAP50‑95) on COCO segmentation, 92 % top‑1 on Imagenette and 89.9 % on ImageNet‑Real, while also showing robustness to distribution shift in video scene detection.
Hugging Face Daily Papersarxiv.orgSergey Kurinov, Alexey Upatov21d1 minpaper
GTR: Gated Token Recurrence for Efficient Dense Prediction arxiv.org
GTR replaces quadratic softmax attention with gated linear attention and spatial recurrence, delivering a fast, softmax‑free vision backbone. It reaches 58.9 COCO box AP with ~1.9 ms latency on RTX 4090 and runs efficiently on edge GPUs via TensorRT.
Hugging Face Daily Papersarxiv.orgZhe Feng, Longfei Liu, Wei Liu13d1 minpaper

Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision

Summary

Related reading

GPT-6 Astra Is the First Model OpenAI Classifies as Critical for Cybersecurityinfoq.com

In-Context Robot Learning with VLM Agentsarxiv.org

PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Looparxiv.org

VisionHOPE: Visual Backbones as Self-Modifying Learning Systemsarxiv.org

TAPe+ML: A Compact Structured Representation for Multi-Task Computer Visionarxiv.org

GTR: Gated Token Recurrence for Efficient Dense Predictionarxiv.org

GPT-6 Astra Is the First Model OpenAI Classifies as Critical for Cybersecurity infoq.com

In-Context Robot Learning with VLM Agents arxiv.org

PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop arxiv.org

VisionHOPE: Visual Backbones as Self-Modifying Learning Systems arxiv.org

TAPe+ML: A Compact Structured Representation for Multi-Task Computer Vision arxiv.org

GTR: Gated Token Recurrence for Efficient Dense Prediction arxiv.org