Hugging Face Daily PapersHongyang Du, Yunfei Xie, Junjie YeSep 25, 20261 min readpaperadvanced

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

Summary

FuseReg replaces the fixed heuristic of selecting encoder layers for representation autoencoders with a regularization that trains on random subsets of layers, making the downstream decoder robust to any fusion. This yields higher reconstruction quality (PSNR) and lowers unguided generation FID by up to 29% on ImageNet‑256, all without changing the pretrained visual encoder.

FuseReg trains decoders on random subsets of encoder layers, making them robust to any layer fusion without retraining.
A single FuseReg decoder on ImageNet‑256 (DINOv3‑L) achieves higher PSNR than decoders specialized to fixed fusions.
Replacing the decoder with FuseReg reduces unguided gFID by 27% for the RAEv2 DiT‑XL generator.
Joint regularization of encoder and diffusion stages cuts unguided gFID by 29% on DiT‑Base.

Practitioners building image generation or reconstruction systems with pretrained visual encoders should care because FuseReg improves both fidelity and generation quality without extra encoder training.

7/10

diffusion computer vision generative models representation learning autoencoders

Related reading

AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation arxiv.org
AutoRef uses a coding LLM to automatically rewrite the harness that orchestrates multi‑reference image generation, keeping the generator frozen. The resulting harness lifts FLUX.2‑4B scores from 5.72 to 7.37 on MultiBanana and generalizes across models and benchmarks.
Hugging Face Daily Papersarxiv.orgYuta Oshima, Ku Onoda, Yusuke Iwasawa8d2 minpaper
Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation arxiv.org
This paper introduces a training-adaptive Convolutional Sparse Coding (CSC) framework where the sparsity coefficient is learned end-to-end via FISTA unfolding. It uses an information bottleneck perspective to balance representation compression and content preservation, showing improved robustness to input perturbations on CIFAR and ImageNet.
Hugging Face Daily Papersarxiv.orgMeng'en Qin, Yinchen Liu, Mingxuan Cui19d1 minpaper
Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene arxiv.org
Mira-Scene proposes a compositional 3D scene reconstruction pipeline that replaces sparse pose regression with a dense Canonical Coordinate Map (CCM) linking image pixels to bounded object space. Coupled with a diffusion transformer, it achieves up to 40% higher 3D‑IoU than prior methods without scene‑level layout annotations.
Hugging Face Daily Papersarxiv.orgYang-Tian Sun, Tianjia Liu, Zehuan Huang16d1 minpaper
SemanTok: Predictable Semantic Tokens for Efficient Autoregressive Video Generation arxiv.org
The paper presents SemanTok, a flexible video tokenizer that injects frozen DINO features and reconstructs them from any token prefix, achieving strong semantic alignment and fidelity. A 201 M SemanTok AR model matches or exceeds a 3.4× larger VideoFlexTok baseline, with cheaper short‑prefix prediction and better generation quality.
Hugging Face Daily Papersarxiv.orgMikhail Dereviannykh, Vikram Voleti, Simon Donne6d1 minpaper
Deep Residual Learning for Image Recognition arxiv.org
The paper proposes reformulating deep layers as residual functions with identity shortcut connections, making it easy to train networks far deeper than before. Using this design, a 152‑layer ResNet achieved 3.57% top‑5 error on ImageNet, winning ILSVRC 2015.
Hall of Famearxiv.orgKaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun12y42 minpaper
Improving synthesis prediction of small molecules at scale with RetroChimera microsoft.com
RetroChimera is an open‑source retrosynthesis system that ensembles a Transformer‑based de‑novo model and a GNN template selector with a learned ranking layer. In blind expert evaluations it achieved a 90 % route acceptance rate, far above prior models.
Microsoft Researchmicrosoft.comFelix Pultar, John Gardner, Guoqing Liu, Marwin Segler14d5 min

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

Summary

Related reading

AutoRef: Harness Optimization for Agentic Multi-Reference Image Generationarxiv.org

Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representationarxiv.org

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scenearxiv.org

SemanTok: Predictable Semantic Tokens for Efficient Autoregressive Video Generationarxiv.org

Deep Residual Learning for Image Recognitionarxiv.org

Improving synthesis prediction of small molecules at scale with RetroChimeramicrosoft.com

AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation arxiv.org

Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation arxiv.org

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene arxiv.org

SemanTok: Predictable Semantic Tokens for Efficient Autoregressive Video Generation arxiv.org

Deep Residual Learning for Image Recognition arxiv.org

Improving synthesis prediction of small molecules at scale with RetroChimera microsoft.com