top of page
Insights on Data, Platforms and AI
Practical insight on data, platforms and AI from people who deliver them.
The Cloud Formations blog focuses on what actually works in modern data programmes. We share lessons from real delivery, platform design decisions, and applied AI use cases, grounded in hands-on experience across strategy and execution.
Expect clear perspectives, technical depth where it matters, and content designed to help you make better decisions, not just follow trends.
Subscribe to to get updates on new posts.
Metadata Driven Orchestration With Microsoft Fabric + Q&A
Webinar - Free to attend knowledge sharing. Join Cloud Formations, Matt Collins who will be delivering our final instalment for metadata month with a deep dive into Metadata-Driven Orchestration in Fabric, showing how a metadata framework can make your processes smarter, faster, and easier to scale. Reserve your spot today to gain insights from industry experts and take part in a live Q&A with our team. Online via Microsoft Teams Thursday 29th January 2026 - 12:30pm to 1:30p
Christine Andrew
Jan 12
Fabric - How to check if a data pipeline is running?
How do you check if a pipeline is running, not from the monitor, but from your Data Pipelines? Maybe you're like me and you have a Data Pipeline process that needs to check if some other pipeline else is running. In my case I have to check if a process is running due to Delta tables liking you to have one process writing to them, otherwise you can get concurrency issues as two items are trying to update the same delta table metadata file. Those tricky metadata items like th
Jon Lunn
Jan 7
Celebrating Innovation: Our Co-Founder Paul Andrew Joins the Microsoft Imagine Cup 2026 Judging Panel
Join us in celebrating innovation as our co-founder Paul Andrew joins the Microsoft Imagine Cup 2026 judging panel. Discover the impact of the Microsoft Imagine Cup today!
Cloud Formations
Dec 15, 2025
Fabric Variable Libraries
Also on the Cloud Formations You Tube Channel One of the big issues with Deployment Pipelines in Fabric, or as I call them Disappointment Pipelines, has been the lack of being able to parameterise connections. You do have deployment rules in the pipelines, but they are limited in functionality and don't support pipeline parameters (boo!), so if you need to push and change items between workspaces in a typical Development, Test and Production workspaces scenario, you had to co
Jon Lunn
Dec 9, 2025
Mastering MERGE in Microsoft Fabric: Incorporate Deletes into your Incremental Loads with Spark SQL
Including DELETE operations in your MERGE statements is a powerful tool, ensuring you update your data completely within a single operation. See the limitations around what is possible in the Spark SQL flavour of this and how to overcome this.
Matt Collins
Dec 5, 2025
Beyond the Hype: What’s Really Happening in Data & AI in 2025
Cloud Formations have launched the 2025 Data & AI Frontline Survey to uncover what’s working, what’s changing, and what practitioners expect from the year ahead.
Daniel Addy
Dec 1, 2025
Why Metadata is the Foundation for Modern Cloud Data Platforms
Cloud data platforms are becoming more complex every day. With more sources, higher expectations and tighter governance, traditional, code heavy approaches struggle to keep pace, creating systems that are costly to scale and hard to manage. By using metadata effectively, organisations can automate the repetitive, scale efficiently and prepare for AI driven workloads. Metadata driven platforms aren't a new thing, it's about how we apply that experience in more complex cloud da
Daniel Addy
Nov 12, 2025
Using Metadata to Build a Robust Analytics Platform: From Introductory to Advanced Concepts
It is likely you’ve used some of these metadata attributes in data engineering tasks before, but what degree do you need to take it to?
Matt Collins
Nov 6, 2025
Getting Microsoft Fabric into Production - Stories from the Field + Q&A
Webinar - Free to attend knowledge sharing. Join Cloud Formations and industry peers for a practical, story-driven session on what it really takes to move Microsoft Fabric analytics solutions into production. Whether you’re just getting started or already scaling workspaces across your organisation, this session will cut through the hype and share real-world experience of using Microsoft Fabric in anger. Reserve your spot today to gain insights from industry experts and take
Daniel Addy
Oct 29, 2025
The Secret to Accelerated Delivery and ROI: A Cloud Shift Success Story
For many business leaders, driving progress is a constant challenge. Data projects often take too long, cost too much, and fail...
Cloud Formations
Sep 23, 2025
Turn insight into action
If something you have read resonates, let’s talk about what it could mean for your data platform or roadmap.
bottom of page