OLTP & OLAP Are Converging Again. But At Which Layer? LTAP vs HTAP
A Data Architects Perspective As Concepts Remain. While Technical Capabilities Evolve
Context
At the Data and AI Summit in June this year, Databricks introduced Lake Transactional/Analytical Processing, or LTAP. My first reaction, I'll admit, was cynicism, just for a change. Every software vendor must invent new names for old things these days. And for those of us who have worked in data architecture long enough to have the grey hair to prove it, like me, this felt very familiar. Hybrid Transactional and Analytical Processing, or HTAP, a term Gartner coined back in 2014, has long advertised bringing operational transactions and analytics "closer together". In my experience it never really became a strong implementation pattern in data platform deliveries.
TL;DR
LTAP is not HTAP with a new badge. Databricks unified the storage layer and deliberately left the engines alone the opposite of what HTAP tried, and I think they are right to.
What changed. One logical copy in object storage. Postgres for transactions, the lakehouse for analytics, no replication pipeline between them.
On Azure, less than the announcement implies. Direct writes are not available. Change Data Feed is preview. Lakehouse//RT is beta.
Fabric is not making the same move. Its SQL database mirrors into OneLake as a separate copy. That is proximity and integration, not convergence.
What has not changed. Modelling, conformed definitions, history, governance. A single copy of data is not the same thing as a single version of the truth.
Then I dug into the detail behind the design thinking this time round and I admit, the cynicism was misplaced. Not because LTAP is a new idea, but because it is a different idea to the one I assumed it was. Databricks are not claiming to have built a single engine that does both jobs. They are explicitly arguing that HTAP failed because it tried to. I happen to agree with them, which is a slightly disorientating way to start reading a vendor announcement. But I do like surprises. It also forces a better question than the one I sat down to write. Not should these workloads converge?, but something harder. If the pipeline between them disappears, what is actually left of the boundary? I think the answer is, most of it. And we should be careful not to confuse the removal of plumbing with the removal of the problem.
Event Announcements If You Missed It - LTAP vs HTAP
Credit where it is due, because the detail matters. LTAP unifies transactional and analytical processing on a single copy of data in the storage layer. Lakebase, the managed PostgreSQL capability introduced last year, now writes into Unity Catalog in open formats Postgres pages transcoded to columnar Parquet in object storage, exposed as Delta and Iceberg. Transactions still run in Postgres and analytics run across the Lakehouse against the columnar copy. Meaning, one logical copy. Two purpose-built engines over the top. No replication pipeline between them. Their argument against HTAP is worth repeating, because it is one most of us have made over a coffee at some point. A single engine serving both workloads ends up with incomplete features, no surrounding ecosystem and no performance isolation all three tracing back to the same decision to unify at the engine layer. Shown visually below.
So, this is not a rebrand of HTAP. It is a fairly deliberate rejection of it in the theme of LTAP vs HTAP.
Considering Azure
I spend most of my time on Azure Databricks, so my next question was the practical one. How much of this can I actually put in a solution design? Less than the announcement implies.
Registering Lakebase in Unity Catalog generally available.
Serving Lakehouse data into Lakebase via synced tables generally available.
Lakebase Change Data Feed public preview.
Lakehouse//RT querying Lakebase beta.
LTAP direct writes not currently available on Azure.
The Blue Route
Microsoft are playing in this space too, and they are not making the same move. At least not yet. There is still time for them to reverse and copy the approach! Too cynical? Sorry. Considering both platforms in the image below.
Drawn side by side they look symmetrical, and that symmetry is the problem. A SQL database in Fabric mirrors automatically into OneLake. Every supported table, no configuration, nothing to opt into. But what lands there is a separate copy in Delta Parquet, queried read-only through the SQL analytics endpoint. That is good engineering and it removes a great deal of pipeline work. It is also, architecturally, proximity rather than convergence. Two copies kept close and kept in step.
Databricks is claiming one copy. Microsoft is claiming two copies you never have to think about. Which is better is a blog for another time, but we should stop calling them the same thing.
Pipelines Were Never The Issue
Here is where I think we should push back, gently, on how this will be received. When a vendor removes the pipeline and integration, it is very easy to hear that as removing the work. It isn't. Latency is reduced, bespoke ingestion is eliminated, data arrives in seconds rather than hours. All genuinely useful. But the parts of analytical engineering that were never about latency are entirely untouched:
Data quality controls.
Historical context and slowly changing state.
Conformed definitions and business rules.
Dimensional or domain-oriented modelling.
Security, governance and semantic models consumers can actually understand.
An operational schema is designed around application behaviour. It is normalised for write efficiency, shaped by how a developer modelled a business process on a particular Tuesday, and it changes when the application changes. None of that stops being true because the same bytes are now readable as Delta. Near-real-time access to an operational table is not automatically an analytical data product. A single copy of data is not the same thing as a single version of the truth. AI may accelerate some of this work. It does not remove the need to understand, model and govern the data and an agent reading an unmodelled operational table will be confidently wrong considerably faster than any human ever managed. They have removed the transport. The translation is still ours.
Relation & Schema Bound Is Not Always Required
My other caution runs in the opposite direction. PostgreSQL and SQL-based transactional engines are mature, reliable and highly capable. But not every application belongs in a relational database. Flexible document structures, globally distributed writes, key-value access at extreme scale, graph relationships, event-sourced models, specialist time-series or search. A relational engine can usually be made to accommodate these. That does not make it the right choice. And there is a specific risk in the LTAP model worth naming. The more attractive the analytical integration becomes, the more gravitational pull the platform's own database exerts on application design. Choosing Postgres because it is right for the workload is good architecture. Choosing it because it comes with a free Delta copy is something else entirely.
Architecture By Platform Catalogue
The risk with any unified platform is assuming that every capability available in it is the preferred capability for every solution. A database should not be selected because it sits next to the Lakehouse, nor an analytical engine because the organisation already uses the transactional database. Nor should a vendor announcement become an architectural principle. But I should argue against myself here, because the case for consolidation is stronger than architects like me usually admit. Deliberate specialisation has a cost. An organisation running Postgres, Kafka, a document store and a Lakehouse needs people who genuinely understand four things, four deep skillsets. Most have people who understand one, and an on-call rota of three. Every additional engine is another failure mode somebody has to reason about at two in the morning. Seen that way, "one platform, slightly compromised" can be the correct decision rather than a lazy one. I have seen more platforms fail from operational overload than from somebody choosing a merely adequate database. And shared platforms mean shared capacity. An analytical query that saturates it is precisely how HTAP used to degrade the transactional applications beside it. LTAP's separation of compute is designed to prevent exactly that, which is the strongest argument in its favour and the first thing I would test rather than assume.
Triangle of Capabilities
In an earlier article I explored whether data fabric and data mesh principles could integrate operational and analytical requirements within a single data product. That ambition still holds, but I would now state it more precisely. A coherent data product does not require a single technical engine. The product boundary can be broader than the database boundary. I find it helpful to think of a modern data product as a triangle:
LTAP's real impact lands on that third point. It does not remove integration. It removes one direction of it, the operational-to-analytical replication that has consumed so much of our engineering effort for so long. Everything else remains. Reverse integration, returning predictions and decisions into operational workflows through governed interfaces. Events between data products that do not share a platform. Contracts and ownership. Operational stores that will never be Lakebase. So, the triangle survives. One of its sides just got considerably shorter, and I would argue that makes the other two more important rather than less. When plumbing stops being the hard part, design becomes the differentiator.
Why Now, and Why Agents
If you want to understand why Databricks built this when they did, look at what keeps appearing in their examples. State and memory for AI agents. I wrote recently about data platforms becoming the operating system for AI, and this feels like a concrete instance of that argument. An agent needs transactional state it can write to, low latency reads of curated analytical data, and a governed path for its decisions to reach operational systems. Serving that from four loosely connected systems is painful. Serving it from one governed copy is a real improvement. That is the most compelling justification for LTAP there is. It is not a justification for consolidating everything else.
Architecture Design Decisions
Going forward, these are the things I'll be thinking about for a solution:
Start with workload characteristics, not the vendor catalogue.
Treat transactions, analytics and integration as separate concerns, even when one platform serves all three.
Do not mistake data availability for analytical readiness.
Keep reverse integration a first-class design concern. It is the half that does not disappear.
Test capacity contention and failure isolation rather than trusting the isolation claims.
Weigh the operational cost of specialisation honestly against the compromise cost of consolidation.
Conclusion
Converge the storage, tot the semantics. Microsoft and Databricks are both making it substantially easier to build applications alongside analytical data, and LTAP is a better answer than HTAP ever was. Unifying at the storage layer while leaving purpose-built engines alone is the right instinct and deserves recognition as more than marketing.
But I would be precise about what has converged. The storage has. The transport has. The semantics have not, and I am not convinced they can. An operational model and an analytical model differ because the questions asked of them differ, and no amount of shared Parquet reconciles that. The pipeline was always the most visible part of the boundary. It was never the hard part. So, my question to the community is not whether transactional and analytical workloads will end up sharing the same engine. Databricks has just conceded that they shouldn't, and I agree with them. It is this. When the pipeline disappears, what happens to the modelling discipline it used to carry? Does removing the friction free us to finally do the harder work properly, or does it quietly convince a generation of delivery teams that the friction was the work? I know which outcome I have seen more often. I would be interested to know whether you have seen different.
Many thanks for reading.
Comments