proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersMeng'en Qin, Yinchen Liu, Mingxuan Cui1 min readpaperadvanced

Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation

Read on arxiv.org

Summary

This paper introduces a training-adaptive Convolutional Sparse Coding (CSC) framework where the sparsity coefficient is learned end-to-end via FISTA unfolding. It uses an information bottleneck perspective to balance representation compression and content preservation, showing improved robustness to input perturbations on CIFAR and ImageNet.

  • CSC sparsity coefficients can be learned adaptively, not fixed manually.
  • FISTA optimization unfolding enables end-to-end training of sparsity.
  • Information bottleneck theory guides the trade-off in representation learning.
  • A label-free post-training strategy improves robustness to corrupted inputs.

Researchers and engineers building robust computer vision systems can leverage this adaptive sparse coding approach to create more resilient models against various input corruptions.

8/10
machine learningrobustnesscomputer visiondeep learningrepresentation learningsparse coding

Related reading

  1. MoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup

    The paper proposes Mixture of Memory Embeddings (MoME), a context‑aware sparse lookup that replaces each token’s single memory row with a gated mixture of multiple slots. Experiments on Llama‑3, MobileLLM and Qwen3 show MoME outperforms existing memory‑embedding baselines at equal parameter and FLOP budgets and exhibits interpretable routing for polysemous tokens.

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention

    VC-Attention introduces a training‑free low‑bit attention pipeline for diffusion transformers. It smooths value tensors via lightweight online clustering (V‑Smooth) and quantizes only the residual after subtracting block means, restoring the mean from the softmax row sum. It also replaces the FP32 softmax exponential with a fused FP8 cast (ExpCast‑FP8) that maps log‑scores directly to E4M3 probab…

    Hugging Face Daily Papersarxiv.org1 minpaper

  5. Learning Sparse Decision Trees via Transformer Variational Auto-Encoders

    The paper presents TREVIS, a method that encodes decision trees into a continuous latent space using a Tree Transformer VAE, allowing gradient-based optimization of both accuracy and structural sparsity. Experiments claim TREVIS matches the predictive performance of near-optimal algorithms while producing sparser trees.

    Hugging Face Daily Papersarxiv.org1 minpaper