top of page
Insights on Data, Platforms and AI
Practical insight on data, platforms and AI from people who deliver them.
The Cloud Formations blog focuses on what actually works in modern data programmes. We share lessons from real delivery, platform design decisions, and applied AI use cases, grounded in hands-on experience across strategy and execution.
Expect clear perspectives, technical depth where it matters, and content designed to help you make better decisions, not just follow trends.
Subscribe to to get updates on new posts.
Selective Deployments in Microsoft Fabric with fabric_cicd: Why Your CI/CD Validation Might Be Checking the Wrong Thing
This blog showcases an example where we selectively deployed Fabric items to upper environments, capturing silent errors and setting up validation to keep our Fabric Repo clean.
Matt Collins
Aug 24
Seed your database with ease through idempotent data-as-code scripts and SQL DACPACs
Seeding your database made easy through scripting and DACPACs!
Matt Collins
Aug 18
Source Control & Branching - Part 3
In part 1 [https://www.cloudformations.org/post/fabric-source-control-branching-part-1] We looked at how to set up Fabric for a multi-developer team, allocating branches and workspaces and: Dev team each get their own workspace Branches get allocated to workspaces DevOps is the prime source of source control We know what each platform responsibilities are for Fabric & DevOps In Part 2 [https://www.cloudformations.org/post/fabric-source-control-branching-part-2] We looked at t
Jon Lunn
Aug 10
Fabric - Source Control & Branching - Part 2
In part 1 [https://www.cloudformations.org/post/fabric-source-control-branching-part-1] We looked at how to set up Fabric for a multi-developer team, allocating branches and workspaces and: Dev team each get their own workspace Branches get allocated to workspaces DevOps is the prime source of source control We know what each platform responsibilities are for Fabric & DevOps Now we're going to look at deploying Fabric items through DevOps using the Microsoft-supported Fabric
Jon Lunn
Aug 1
Fabric - Source Control & Branching - Part 1
Developing in Microsoft Fabric within a multi-team environment has not always been straightforward. Fabric was initially geared more towards individual users and citizen developers working independently within a workspace, which created challenges when applying traditional source control and DevOps practices. Common issues included: Source control branches being linked at the workspace level, meaning one user switching branches could affect everyone using that workspace. Bran
Jon Lunn
Jul 30
Fabric User Data Functions
Fabric User Data Functions are a bit of a strange hybrid. They can be used like Azure Functions, to call some code outside and act like a serverless compute function, but also can extend notebooks as a source of reusable data logic. I've been using them since preview, and now they support Azure Key Vault (AKV) and Fabric Variable Libraries, I think they are ready (mostly) for productionable use. What are they all about So in my data solutions and processes, there tends to be
Jon Lunn
Jul 20
The Modern Data Engineer
We've been chatting internally about the future of what we do, and Paul outlined in his post, how AI is going to be used across processes and platforms. Check it out here Data Platforms – The New Operating Systems For AI This post is a bit more focused on how to use it in my, (and your) role as a data engineer. What has changed, what's good and what's bad. Back in the day A lot has changed over the years of my career, a long time ago I was just writing SQL queries for Access
Jon Lunn
Jul 13
Fabric - Lakehouse Table Health Check
Microsoft have rolled out a new 'sys' stored procedure called 'sys.sp_get_table_health_metrics' for checking the health of your tables in your Lakehouse. And it outputs a lot of metrics looking at the state of your table and anomaly detection in them. What does it check for? Well it checks to see if your Lakehouse Delta tables are physically well-optimised for SQL query performance. It returns file-level health metrics for a specified Lakehouse table, including file-size dist
Jon Lunn
Jul 8
Data Platforms – The New Operating Systems For AI
Instead, we need to think bigger. Think about a conceptual shift to realise this wave of AI powered value from our data solutions. Not just a data platform that delivers AI capabilities and outputs. A data platform that forms the foundations for our AI capabilities. Maybe better described as a complete agentic framework of reasoning engines, plans, skills, and tools. The AI architecture sitting on top of our data architecture. On top, not added at the end.
@mrpaulandrew
Jul 2
Materialized Lake View - Update!
A few months ago, I did a post on Materialized Lake Views, and said that it wasn't quite ready to fit in your normal data pipelines. You could only schedule the refresh with its own in built scheduler or hit the API to kick of a refresh. I had a lovely method to do an API refresh using an Fabric User Data Function, but the API didn't support a Service Principal or Workspace Identity, so I didn't end up using it sadly. However, as part of the June Fabric update, we now have in
Jon Lunn
Jun 16
Turn insight into action
If something you have read resonates, let’s talk about what it could mean for your data platform or roadmap.
bottom of page