Hugging Face Daily PapersUday Allu, Abhivanth Sivaprakash, Pratik Singh1 min readpaperadvanced
Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion
Summary
D‑RAC extends the W‑RAC pipeline to any document type by first rendering it to PDF, then using a single multimodal LLM pass to turn each page into a retrieval‑optimized Markdown representation (tables become prose, headings are kept). Chunking works on deterministic IDs, avoiding re‑tokenising the source text. On a 236‑doc, 795‑page benchmark D‑RAC processes the whole set in 72 min, creates 1,748…
- Render any input (Word, PPT, scans) to PDF first – PDF is a deterministic, lossless target for visual layout.
- A single multimodal LLM pass converts rendered pages to Markdown, preserving hierarchy and rewriting tables as self‑contained prose.
- Chunk planning operates on identifier metadata, not raw text, keeping token cost low and the process deterministic.
- Empirical results: 72 min for 236 docs, 1,748 chunks, 95.7 % token reduction, 77.8 % cost reduction (GPT‑4.1) and 75 % time reduction vs. agentic chunking.
Enterprise RAG pipelines struggle with heterogeneous formats and expensive, error‑prone OCR or full‑text LLM chunking. D‑RAC offers a reproducible, low‑cost ingestion path that retains structural information crucial for retrieval, making large‑scale knowledge base construction feasible without mass…
8/10
