Hugging Face Daily PapersHejia Geng, Zesen Huang, Haoyang Li2 min readpaperadvanced
ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments
Summary
ScienceIDE is a framework that turns existing scientific software repositories into programmable environments that agents can use for task generation, execution, and verification. Training on these environments yields LLMs (PhAI‑IDE series) that outperform baselines on scientific code repair and several general code‑reasoning benchmarks.
- Agents ingest a repository, apply expert‑defined scientific cases and acceptance criteria, and output a self‑contained executable environment.
- The generated environments expose a unified API for task sampling, execution, and automated scientific verification, enabling supervised fine‑tuning and RL.
- Training PhAI‑IDE‑72B/9B/4B on verified interaction trajectories improves held‑out scientific‑code repair by up to X% (paper reports gains) and boosts performance on standard code benchmarks.
- ScienceIDE’s verification pipeline reduces noisy supervision, making large‑scale scientific code data usable for LLM training.
AI researchers and engineering teams building code‑focused LLMs should care because ScienceIDE offers a scalable way to turn the vast corpus of scientific software into high‑quality training data and evaluation environments.
7/10
