Written by Hasan Mohammed Tanvir, Data Engineer at Bolt.

A ride completed in Tallinn, a food order delivered in Nairobi, a scooter unlocked in Berlin, a driver paid out in Bangkok — every product event across Bolt's rides, delivery, micromobility, and car-sharing businesses generates a financial entry that needs to be recorded, reconciled, and reported accurately.

At Bolt's scale, operating across 50+ countries, multiple currencies, jurisdictions, and business lines, that adds up to billions of financial ledger entries per month. These entries feed statutory filings, tax reports, management accounts, and FP&A planning — often all at the same time, often requiring different cuts of the same underlying data.

The challenge is not just storing this data reliably. It is making it available to every downstream consumer, at the right grain, at the right time, without each team having to own their own copy of the truth.

The Finance Ledger is the single source of record for all financial activity at Bolt across all legal entities and the foundation every downstream system builds on. Our data layer follows a layered architecture inspired by the medallion pattern — raw staging, clean entries, and a unified finance ledger mart — with a star schema dimensional modelling.

This work is a step toward offering Finance Ledger as a Service (FLaaS) — Finance DWH's core asset made available as a single, always-fresh, always-correct view that any downstream consumer can query at whatever grain and hierarchy level they need. In practice, this means a finance analyst querying ride revenue versus food delivery revenue by country, and an FP&A tool pulling monthly cost centre data for budgeting, both drawing from the same source — without either team owning a separate pipeline.

The ledger_aggr_engine is the backbone of that vision, sitting in the serving layer between the ledger and downstream consumer-specific integration layer, enabling each one to define exactly how they want the data shaped and compressed.

Note: FinLedgerAggr (finance_ledger_aggr) and ledger_aggr_engine are two distinct things. FinLedgerAggr is the pre-aggregated Gold layer — a source table in the data layer produced by a vanilla GROUP BY on a reduced set of columns, with no hierarchical dimension traversal. The ledger_aggr_engine sits above it in the serving layer, applying dimension hierarchy rollups and compressing it further into consumer-specific grains.

The engine can technically aggregate directly from the raw Finance Ledger as well — the decision to consume FinLedgerAggr instead is purely a performance decision. Both tables are partitioned on the same columns and kept in sync, so every pipeline write to the Finance Ledger automatically propagates to FinLedgerAggr. As a result, the engine avoids re-aggregating raw rows that are already summarised, saving compute resources.

The serving layer (green block in the diagram above) feeds all downstream consumers (orange blocks), each consuming the same source but at fundamentally different grains and structures:

FP&A — Bolt's FP&A tool needs data at the agreed planning grain at the monthly level aggregation. For example, deciding whether Bolt Food in a specific market is underperforming against plan.
Reporting tools — Various financial and management reports, including income statements, trial balances, statutory reports, IFRS reporting, and tax reporting, each with its own dimensional requirements and aggregation levels. For example, producing a country-level statutory report for Estonia requires a completely different cut of the same ledger data than an IFRS consolidation across all legal entities.
Accounting journals — Daily accounting journals booking for product needs a different grain again, with specific structural requirements. For example, driver payouts across a market are aggregated into debit and credit journal entries for the day — recording payables to drivers and receivables from customers so that debits equal credits, the books close correctly at month end, and the trial balance holds.
Human User — Technical and non-technical users who need direct access to Finance Ledger data — whether to build their own models on top of it, run ad hoc analysis, or investigate a specific number that doesn't look right.
Technical User — Internal services and tools that need programmatic access to Finance Ledger data through a consistent API or query interface, without owning a separate pipeline or data copy.

The aggregation engine handles these grains through three components:

Aggregation Engine — Shape and Compress

The ledger_aggr_engine is intentionally generic. It knows how to filter a ledger source, traverse dimension hierarchies, and GROUP BY the given keys. Each consumer passes their own group_keys, agg_columns, agg_levels, and measures to get exactly the shape they need. For example, grouping all ride transactions by city, country, and month for FP&A planning uses the exact same engine as aggregating driver payout totals by market for accounting journals — the caller defines the grain, the engine does the rest.

Aggregation Enrichment — Add and Derive

The enrichment layer adds derived attributes to any ledger-compatible source — the raw Finance Ledger or the finance_ledger_aggr (pre-aggregated Finance Ledger). It is purely additive: net amounts from double-entry debit/credit bookings are collapsed into a single value; EUR conversion so all entities are comparable in a common currency regardless of where the transaction occurred; decimal precision; cost type classification; and deterministic, unique ID generation for idempotent downstream loads.

Aggregation Validation — Verify and Report

The validation module is the quality control layer. It generates a reconciliation report comparing source and target: an amount reconciliation, a row count comparison, a null distribution, and dimension coverage, verifying that all expected hierarchy values are present in the output. The validation report is not a pipeline blocker — it runs alongside and produces a structured record of what changed, what was compressed, and whether the numbers balance. In practice, this is what prevents situations where driver payout totals don't match reported revenue.

This post covers the design of the aggregation engine as a generic, consumer-agnostic module and its first production use case — the FP&A integration. The same patterns extend to every downstream consumer. One engine, one source of truth, many tools.

The aggregation engine is the first concrete step toward FLaaS.

The Problem: Materialisation All the Way Down

The original pipeline was deliberately built around a PySpark notebook. Our finance ledger lives in dbt, which cannot create models dynamically at runtime. dbt compiles its entire DAG at build time from static SQL files, so there is no mechanism to read a configuration at runtime and generate a different set of models each day.

PySpark solved this: a notebook could read a Google Sheet at runtime, loop over aggregation rules, and write each result to its own Delta table — giving non-technical users a self-service configuration layer without engineering involvement. The cost was structural. PySpark is imperative and cannot be stored as a SQL view, so every output had to be a materialised table. In practice, this meant data was always a snapshot frozen at the last pipeline run. When the Finance Ledger was updated — a backfill, a correction, a new pipeline writing fresh data — downstream tables wouldn't reflect it until the next scheduled run. At month-end close, teams sometimes reported numbers that were hours behind the actual ledger state. And when a source schema or values changed, the fix had to be manually coordinated across every downstream layer — if one was missed, reports would silently produce wrong numbers.

Note: finance_ledger_aggr is the pre-aggregated Gold source table — not the aggregation engine. See the architecture diagram note for the full distinction.

Each boundary freezes freshness and creates a schema contract risk — a source change requires coordinated propagation across every downstream layer.

Beyond governance, every materialisation is a Spark action: I/O serialisation and deserialisation at each step, growing lineage DAGs that prevent the optimiser from seeing the full plan, and vanilla Spark operators throughout, with no Photon acceleration. However, the pipeline took ~2 minutes to process a month of data, which is not bad at all. Still, the decision to move was architectural, rooted in data governance rather than performance optimisation.

The Architecture Decision: Thin Out to Views

The solution was to eliminate intermediate materialisation and push all computation into views, so every read computes directly against the freshest source. To do that, we needed to replace PySpark with something that generates pure SQL — because a view is just a stored SQL string, and SQL can be stored, a Python function cannot.

The trade-off is moving from runtime dynamism to compile-time parameterisation with dbt — acceptable in practice since integration rules change infrequently. A separate pipeline remains available for ad hoc aggregations by non-technical users until FLaaS is fully live.

The shift to views is deliberate: by pushing computation to query time rather than writing intermediate state, the entire pipeline executes as a single optimised plan. One open question was whether compute could handle the load — the legacy pipeline gave us a ~2-minute benchmark, but the reality turned out better. dbt runs on Photon, Databricks' C++ vectorised engine, and Spark's query optimiser can now see the full plan end-to-end, rather than optimising each materialised step in isolation. Neither was possible when the pipeline ran on vanilla Spark with Python orchestrating discrete write jobs.

Results:

Execution Time

The benchmark we were worried about beating, we halved. As shown in the chart, the new pipeline processes one month of data in 64.96s vs 136.32s previously, and a full year in 375.39s vs 460.78s. The speedup mostly comes from eliminating the I/O serialisation and deserialization cost at every intermediate materialisation step. The gap narrows at larger date ranges simply because there is more data to scan end-to-end, but the architectural win holds at scale — we eliminated materialisation and kept the data always fresh, without the compute time penalty we were initially concerned about. Worth noting: these numbers measure execution time only — cluster spin-up overhead is excluded, so the real-world gap is wider than the chart suggests.

Data Compression

2025 saw the highest data growth rate for the Finance Ledger in four years — a 75% year-over-year increase in raw row volume — driven by the January 2025 launch, which shifted product data from aggregated to fully transactional. Despite this, engine output has stayed stable. Without this compression, the raw data growth would flow directly into downstream tools — reporting systems would slow down, and FP&A tools would struggle to keep up as transaction volume grows. The compression chain absorbs that growth, keeping downstream load predictable regardless of what happens upstream.

Note: Row volume comparison — Finance Ledger vs Finance Ledger Aggr vs FP&A Tool Aggr (output of ledger_aggr_engine). The FP&A tool line began in January 2023, when the consumer went live.

Note: Compression comparison: Finance Ledger vs Finance Ledger Aggr vs FP&A Tool Aggr (ledger_aggr_engine). The FP&A tool line began in January 2023, when the consumer went live.

Conclusion

Performance was a by-product. The primary wins are architectural: zero intermediate materialisation, no freshness lag, no schema contracts between layers, and aggregation rules that are versioned, reviewable, and deployable like any other code. On the business side, the FP&A team achieved less than 3% budget-to-actuals variance this cycle — fresh, correctly grained data is a prerequisite for accurate planning.

The aggregation engine is the first and most critical piece of the FLaaS vision. Without a generic, consumer-agnostic serving layer that can shape the ledger to any grain on demand, everything else — on-demand API access, self-service for finance users, multi-tool rollout — is just an idea. That layer now exists. The rest can be built on top of it.

Join Bolt

Grow fast, make a global impact, and help make cities for people, not cars. Explore hundreds of open roles on our career page.

Finance ledger as a service: Rethinking aggregation engine

We’re hiring!

Recent posts

Bolt and Stellantis to scale autonomous vehicles in Europe