Data Modelling with dbt (core)

Written by Silja Märdla, Senior Analytics Engineer at Bolt.

In Bolt, we’ve been on a mission to understand the best way to systematically prepare data for our different data users: data analytics, reporting, machine learning, etc.

We’re using dbt (core, not cloud), so naturally, we’ve read their best practice guidelines tens of times. After a while, we realised that it gives good guidance but misses a few crucial distinctions, especially after introducing the Semantic Layer.

Here is our version of what data modelling can look like. This setup allows for a mixed usage of “classical data modelling” and “data modelling with semantic layer”.

The purpose of sources, staging and intermediate models corresponds to that described by dbt. Additionally, we distinguish between core models and data marts.

Core models are the dimension and fact tables that provide business information and act as sources for data usage.

In the case of the classical approach, they will also contain the definitions of business metrics in the form of columns in tables. However, with the semantic approach, the business metrics will be pulled out of the core models and into metrics. Core models will be represented as semantic models, and metrics will be defined on top of semantic models.

Data marts are self-contained tables at certain granularities, ready to be used in reporting, analytics, machine learning and elsewhere downstream. It’s common that the same entities and metrics will appear in multiple data marts.

In the case of the classical approach, data marts are created as aggregations on top of core models. The semantic approach generates data marts from the semantic layer using dimensions and metrics.

We’re not using the dbt cloud, so we don’t have any downstream data users plugged directly into the metrics layer. The option to do that would make the picture complete. 

Join us! 

Bolt is a place where you can grow professionally at lightning speed and create a real impact on a global scale.

If you feel inspired by our approach and want to join us on our journey, check out our job board. We’ve hundreds of open roles, from entry-level to top management — each being an exciting opportunity to contribute to making cities for people, not cars.

If you’re ready to work in an exciting, dynamic, fast-paced industry and are not afraid of challenges, we’re waiting for you!