DuckDBGeertjan Wielenga7 min readintermediate
DuckDB Now Ships inside dbt v2
Summary
dbt v2 (Rust‑based Fusion engine) now bundles a DuckDB adapter, removing the separate Python package. It adds built‑in catalog support (DuckLake, Iceberg), writes dbt’s information schema as Parquet for fast querying, provides native SQL comprehension with column‑level lineage, pins a specific DuckDB version (enabling native extension functions), and offers a migration path from v1.
- No extra install step: dbt v2 auto‑downloads the DuckDB driver via ADBC.
- Catalogs are configured in `catalogs.yml` and attached automatically; Iceberg REST support requires the `use_catalogs_v2` flag.
- Running `dbt parse --generate-info-schema` produces Parquet files that can be queried directly with DuckDB, avoiding large JSON manifests.
- The Fusion engine parses SQL natively, catching column/type errors early and emitting `dbt.column_lineage` Parquet for lineage queries.
Embedding DuckDB in dbt removes a Python dependency, speeds up local development, and lets data teams treat dbt metadata as first‑class data for CI checks, audits, and lineage analysis—all without a warehouse.
6/10
