Phase 2 Is in Production, Six Weeks After Phase 1

Phase 2 of the financial data hub we built for a Tier-1 financial institution went into production on September 15. That was six weeks after Phase 1, through the same formal change control: a documented change request, test evidence and independent approval.

Inside a regulated bank, six weeks between production releases isn’t the norm. Here’s what Phase 2 does, what let it move at that pace, and the one testing habit we’d insist on for any migration.

What Phase 2 does

Every user of the application sees a summary of what’s waiting for them: items to approve, review or complete, across more than twenty kinds of action. Today the source system works those counts out at login, which makes it the application’s most expensive screen. Phase 2 computes them in the hub on a refresh cycle and hands them to the application through a single governed view.

That was the “what’s next” when we published our latency measurements in September. Under load testing, a full recompute of every user’s counts runs in under two minutes on the smallest warehouse size. Running the categories in parallel made it slower, because they starved each other on a single node, so the cycle runs them in sequence.

Why it moved at that pace

Nothing about the change process got shorter. What changed is what we brought to it.

  • The evidence was built with the code. The change request’s test evidence came out of the same validation suite that tested the work, not a separate write-up afterward.
  • The deployment was rehearsed. We tore down and rebuilt the test environment in deployment order, so the production run was a repeat, not a first attempt.
  • Performance was measured before approval. A load generator drove sustained change traffic through the new layer in test, so performance was a measured result rather than an open question.
  • Its controls shipped with it. New checks joined the hub’s validation framework on day one. They confirm that every category recomputes on schedule and that every count reaches the user it belongs to.

Test against what runs, not what’s written down

The acceptance standard was parity: the hub’s counts must match the legacy logic. The obvious way to test that is against the documented logic you were handed. We tested against the legacy procedure as it actually runs in production.

That caught a difference five days before cutover. The legacy logic had changed after the handover, and the documentation didn’t reflect it. That’s the most ordinary thing in the world. Every organization has logic that moved after someone wrote it down.

The cost wasn’t ordinary. Built strictly to the documentation, one of the counts ran more than 70% high across the user base. The cutover package was already locked for approval, so the correction went in two nights later as a planned step of the same change, before anything downstream read those numbers.

If your migration’s test plan is built from your documentation, ask what it actually compares against. If the answer isn’t production behavior, you’re testing the copy.

What’s next

Next, working with the client’s team, we switch the application to reading from the hub. That’s when we can measure the number we promised in September: how fast that login-time summary is when it reads pre-counted rows. We’ll publish it once we’ve measured it.

Is your next release six weeks away, or six months?

If every production change in your regulated environment turns into a quarter-long project, the process usually isn’t the whole problem. What arrives at it is. Let’s talk.

“Near Real-Time” Is a Claim. We Turned It Into a Number.

Update, October 2026: the pre-computed counts described at the end of this post are now in production. Here’s what shipped.

The question nobody actually answers

In March we wrote about why the hard part of a financial data hub isn’t the pipeline — it’s the accounting. Since then the hub has grown a ten-prong automated validation framework, native row-level security, an operations dashboard, and a period-balance engine that reconciled against decades of accounting history.

It’s now in production.

But before you take a financial data platform live, there’s a question every vendor waves away with the phrase “near real-time”:

When someone posts a transaction, exactly how long until the data hub knows?

Not roughly. Not “typically.” Exactly — at every hop, under load, with evidence.

Yes, you can load-test a general ledger safely

That sentence stops most conversations, so let’s deal with it first.

We built a reversible load generator. It pushed sustained change traffic, with deliberate bursts, at five to ten times the busiest hour ever observed in production. Every change it made to the financial data was automatically reversed and verified afterward. The test environment ended the campaign byte-for-byte identical to how it started.

Alongside it we built a latency harness that timed each stage of the journey — source commit, change-data-capture ingestion, warehouse consolidation, automated processing, finished query-ready data — without requiring elevated privileges in the warehouse.

Two instruments, one week, a full copy of the production pipeline.

What the numbers said

Ingestion is not the bottleneck. Changes left the source system and landed in the warehouse’s receiving area in 4 to 17 seconds, even at ten times normal load.

Freshness has exactly one dial. Nearly all of the end-to-end wait came from a single consolidation setting in the replication layer — a number we control. Typical commit-to-queryable time: 2 to 4 minutes under controlled load. In production, where traffic is burstier and less predictable than anything we could synthesize, we observe 2 to 8 minutes. Need it faster? That’s a configuration change, not a redesign.

The pipeline is self-healing under pressure. At peak overload, work queued briefly — then the system drained its own backlog and returned to normal speed. Two campaigns, zero failures, zero lost records, zero human intervention.

The smallest warehouse size held. No congestion, no upsizing required. Performance headroom is budget headroom.

Why this should matter to you

If your organization is being sold a “near real-time” data platform, ask for the hop-by-hop numbers.

If nobody can produce them, the SLA conversation you’ll eventually have with your business stakeholders is being deferred to the worst possible moment: after go-live, during an incident.

Our take: measure before you promise. A one-week testing campaign turned three open architecture debates into closed decisions, gave the client’s testers realistic expectations in writing, and documented exactly how the platform behaves on its worst day — before production ever saw it.

What’s next, and what we’ll publish about it

With the hub in production, the next milestone inverts the problem. Instead of asking how fast data can get in, we’re looking at pre-computing answers on the way in — so the application’s most expensive screen, a login-time summary that today fires off a couple of dozen live queries, reads a handful of already-counted rows instead.

We’ll publish the numbers for that one too, once we’ve measured them.

Which is the entire point of this post.

Ready to know your numbers?

You don’t need more dashboards claiming “real-time.” You need architects who can tell you — with evidence — how many seconds sit between a keystroke and an insight, and which dial to turn when the business asks for fewer of them.

If that conversation is overdue at your organization, let’s talk.

Architecting a Financial Data Hub for Multi-Entity Consolidation

Update, October 2026: Phase 2 of this hub is now in production. Read what shipped, six weeks after Phase 1.

Update, August 2026: Phase 1 is now in production. Read the full case study.

The Real Bottleneck in Financial Analytics

We are currently 60 days into a complex enterprise data modernization project, successfully streaming core transactional data into a modern cloud warehouse. But as any seasoned finance professional knows, the data pipeline is just the plumbing. The true bottleneck in modernizing financial analytics is the business logic—specifically, taming the General Ledger.

Most modern data migrations fail because standard data engineers don’t understand the mechanical complexity of accounting. They build tables, but they don’t know how to handle the shifting idiosyncrasies of period balances.

Solving Accounting Complexity at the Database Level

At Datagize, we approach this differently. Drawing on 40+ years of enterprise data engineering and a deep MBA/Accounting foundation, our current milestone involves building out a Financial Data Hub that programmatically solves the most painful aspects of the month-end close.

We do this by flattening the complex account hierarchy directly within the data model. This architectural decision abstracts the heavy logic away from the BI layer and automates the workflows that usually trap finance teams in Excel, including:

  • Seamlessly handling the consolidation of books across multiple entities.
  • Dynamically calculating roll-forwards for Current Year Earnings (CYE) and Retained Earnings (RE).
  • Automating the shift of Net Operating Income (NOI) from the P&L to the Balance Sheet.

Governance: Built-In, Not Bolted On

In financial services, accuracy is only half the battle; security is the other. Our architecture handles strict authentication and native row-level security. Data is restricted strictly on a “need-to-know” basis, ensuring segment leaders only see their approved domains, while secure external data sharing capabilities allow for frictionless, governed output for auditors or board reporting.

The Roadmap: Looking Ahead to Financial Statement Generation (FSG)

With the core Data Hub and consolidation logic deployed, the next question we are working through is what fully integrated Financial Statement Generation (FSG) would look like natively within this architecture. That is a design exercise on the 2026/2027 horizon, and we will write about what we learn from it.

Ready to Transform Your Financial Analytics?

You don’t just need data engineers to move your financial data; you need architects who understand what a balance sheet actually is. If your organization is ready to stop fighting its account hierarchy and modernize its month-end close without compromising financial integrity, let’s talk.