dbt

• Unit Testing dbt Macros: A workaround for dbt's unit testing limitations Ever wished you could catch that broken SQL logic before it wrecks your dashboards? With dbt 1.8's new unit testing capabilities, you can finally sleep at night! However, support for testing macros is still limited. Let's explore how to test both models and macros with a workaround. Continue reading...
  dbt data-engineering data-quality SQL testing

  6 min read
• Own your web analytics pipeline for €0.02 per day: Snowplow, Terraform, dbt, BigQuery and Docker Running Snowplow for your (web) analytics pipeline to expensive? Here's a €0.02/day minimal, serverless version of Snowplow open source that you can deploy for your blog or website with Terraform (on GCP/BigQuery) in 5 minutes giving you full ownership of a web and app analytics pipeline from data collection to custom data models (👋 goodbye Google Analytics). Continue reading...
  snowplow GA4 gcp BigQuery Terraform dbt Docker

  29 min read
• Mastering Time in dbt: Incremental Merging of Estimates and Actuals for large datasets Managing incrementality (change over time) in a large database is hard. Dbt can help us alleviate some of the pain by making the selection of incremental strategies we have easier to choose from. Lets look at updating an example sales table with actuals and estimates over time. Continue reading...
  dbt BigQuery incrementality

  8 min read
• Dbt In a Box: Using Google Cloud Run and BigQuery to run your dbt SQL models from a Docker container Dbt is a great tool for data transformation. Snowplow is great for collecting web analytics data. What if you could harvest the power of both for just a few cents a day by running dbt in a Docker container on Google Cloud Run Jobs? Continue reading...
  dbt snowplow BigQuery Docker gcp cloudbuild

  13 min read