They all write, store and read data: how to pick the right database ·
SQLite, Postgres, DuckDB and 20 others: they all write, store and read data. The questions you ask every day tell you which one you need.
Duckdb
-
-
Querying the entire internet with DuckDB and Common Crawl ·
Common Crawl publishes petabytes of web crawl data on S3. With DuckDB you can query it directly, no cluster and no download, and measure how fast the vibe-coded web is really growing.
-
Cloudflare ingestion patterns have different cost shapes Short ·
-
Microbatch: how to supercharge dbt-duckdb with the right incremental model ·
Learn when and how to use dbt's microbatch incremental strategy with DuckDB. Covers row groups vs partitions, benchmarks comparing full refresh, merge, delete+insert, and microbatch strategies, plus configuration tips and common pitfalls.
-
DuckDB is my new Excel ·
I'll admit it: as a data engineer I still use Excel. But DuckDB has become my secret weapon for ad-hoc data exploration. No more struggling with CSV imports, no more manual column matching across files, no more giving up on JSON. This guide shows you how to query remote CSVs, handle schema drift across multiple files, unnest nested data, and flatten JSON APIs—all with SQL you can remember if you can remember Excel formulas.
-
Turn Thousands of Messy JSON Files into One Parquet: DuckDB for Fast Data Warehouse Ingestion ·
If you've inherited a bucket full of thousands of tiny JSON files—one per API call, one per event, one per log minute—you know the pain: slow scans, schema anxiety, and rising warehouse bills. This guide shows you how to consolidate them into clean Parquet with DuckDB: handling schema drift, maintaining lineage, optimizing performance, and integrating with dbt. Touch your raw files once, then model against something stable.
-
Setting secrets in dbt for MotherDuck Short ·