Querying the entire internet with DuckDB and Common Crawl ·
Common Crawl publishes petabytes of web crawl data on S3. With DuckDB you can query it directly, no cluster and no download, and measure how fast the vibe-coded web is really growing.
Common Crawl publishes petabytes of web crawl data on S3. With DuckDB you can query it directly, no cluster and no download, and measure how fast the vibe-coded web is really growing.