top of page
Insights on Data, Platforms and AI
Practical insight on data, platforms and AI from people who deliver them.
The Cloud Formations blog focuses on what actually works in modern data programmes. We share lessons from real delivery, platform design decisions, and applied AI use cases, grounded in hands-on experience across strategy and execution.
Expect clear perspectives, technical depth where it matters, and content designed to help you make better decisions, not just follow trends.
Subscribe to to get updates on new posts.
Introducing The Intelligence Factory
Ask an organisation what its data platform delivered this year and you will get a list of things that were built. Pipelines, a Lakehouse migration, a Copilot pilot. Ask which decisions are being made better than they were in January, and the room tends to go quiet. That gap is not a technology failure, it is a framing failure. First we sold the data factory, then the AI factory, both named after the raw material. Here is a different name, and a harder measure of whether the w
@mrpaulandrew
Sep 28
OLTP & OLAP Are Converging Again. But At Which Layer? LTAP vs HTAP
Databricks LTAP and Microsoft Fabric are both bringing transactional and analytical workloads under one roof, but they are not doing the same thing. One converges the storage beneath purpose-built engines. The other keeps two copies you never have to think about. Neither removes the part that was always hardest. A look at what has actually converged, what has not, and why near-real-time access to an operational table is still not an analytical data product.
@mrpaulandrew
Sep 21
Selective Deployments in Microsoft Fabric with fabric_cicd: Why Your CI/CD Validation Might Be Checking the Wrong Thing
This blog showcases an example where we selectively deployed Fabric items to upper environments, capturing silent errors and setting up validation to keep our Fabric Repo clean.
Matt Collins
Aug 24
Seed your database with ease through idempotent data-as-code scripts and SQL DACPACs
Seeding your database made easy through scripting and DACPACs!
Matt Collins
Aug 18
Source Control & Branching - Part 3
In part 1 [https://www.cloudformations.org/post/fabric-source-control-branching-part-1] We looked at how to set up Fabric for a multi-developer team, allocating branches and workspaces and: Dev team each get their own workspace Branches get allocated to workspaces DevOps is the prime source of source control We know what each platform responsibilities are for Fabric & DevOps In Part 2 [https://www.cloudformations.org/post/fabric-source-control-branching-part-2] We looked at t
Jon Lunn
Aug 10
Fabric - Source Control & Branching - Part 2
In part 1 [https://www.cloudformations.org/post/fabric-source-control-branching-part-1] We looked at how to set up Fabric for a multi-developer team, allocating branches and workspaces and: Dev team each get their own workspace Branches get allocated to workspaces DevOps is the prime source of source control We know what each platform responsibilities are for Fabric & DevOps Now we're going to look at deploying Fabric items through DevOps using the Microsoft-supported Fabric
Jon Lunn
Aug 1
Fabric - Source Control & Branching - Part 1
Developing in Microsoft Fabric within a multi-team environment has not always been straightforward. Fabric was initially geared more towards individual users and citizen developers working independently within a workspace, which created challenges when applying traditional source control and DevOps practices. Common issues included: Source control branches being linked at the workspace level, meaning one user switching branches could affect everyone using that workspace. Bran
Jon Lunn
Jul 30
Fabric User Data Functions
Fabric User Data Functions are a bit of a strange hybrid. They can be used like Azure Functions, to call some code outside and act like a serverless compute function, but also can extend notebooks as a source of reusable data logic. I've been using them since preview, and now they support Azure Key Vault (AKV) and Fabric Variable Libraries, I think they are ready (mostly) for productionable use. What are they all about So in my data solutions and processes, there tends to be
Jon Lunn
Jul 20
The Modern Data Engineer
We've been chatting internally about the future of what we do, and Paul outlined in his post, how AI is going to be used across processes and platforms. Check it out here Data Platforms – The New Operating Systems For AI This post is a bit more focused on how to use it in my, (and your) role as a data engineer. What has changed, what's good and what's bad. Back in the day A lot has changed over the years of my career, a long time ago I was just writing SQL queries for Access
Jon Lunn
Jul 13
Fabric - Lakehouse Table Health Check
Microsoft have rolled out a new 'sys' stored procedure called 'sys.sp_get_table_health_metrics' for checking the health of your tables in your Lakehouse. And it outputs a lot of metrics looking at the state of your table and anomaly detection in them. What does it check for? Well it checks to see if your Lakehouse Delta tables are physically well-optimised for SQL query performance. It returns file-level health metrics for a specified Lakehouse table, including file-size dist
Jon Lunn
Jul 8
Turn insight into action
If something you have read resonates, let’s talk about what it could mean for your data platform or roadmap.
bottom of page