Accelerate Your Cloud Data Transformation with CF.Cumulus™

An open-source, metadata-driven framework for rapid, scalable, and low-cost data orchestration, trusted across industries.

Get Started with CF.Cumulus

Open-Source on GitHub

Zero license costs, unlimited access to code

Open Source

Supports complex & unique data handling

Extensible

Adopted and used with 100+ repository forks

Proven Framework

Horizontal & vertical scaling for Big Data executions

Scalable

Brings development & operations together

Collaborative

Tried & tested patterns using mature technologies

Transparent & Trustworthy

Faster time-to-insight reduces delivery costs

Cost-Efficient

Cut delivery time from weeks to days

Rapid Deployment

Professional support and guidance, whenever needed

Backed by Experts

Technical Overview

Simplifying Complexity Through Metadata

The core pillars of CF.Cumulus: Control, Ingest, and Transform provide the backbone of a modern cloud data platform. Each module is crafted to simplify your data processes, enabling seamless integration and operation with a metadata-driven approach.

Control

Metadata-driven workflow orchestration
Managed dependency chains
Concurrent batch processing
Multi-threaded execution
Resilient exception handling
Enterprise grade reliability

Ingest

Metadata-driven data ingestion
Adaptable to all scenarios
Extensible data sources
Extensible data formats
Flexible operating modes
Automated code generation

Transform

Metadata-driven data transformation
Rapid engineering of common transformations
Scalable data processing
Baked in support for dimensional models
Supports AI & Machine Learning model inference
Complete flexibility through custom notebooks

Flexible Technology Choices

The CF.Cumulus framework can be applied across a range of Microsoft Azure technology configurations depending on your preference. Each choice of tooling for Orchestration, Compute and Storage with offering different advantages and coupling of resources.

Partially decoupled architecture, in terms of cloud resources used to host CF.Cumulus:

Orchestration: Azure Synapse Analytics
Compute: Azure Synapse Analytics
Storage: Azure Data Lake

Using Azure Synapse Analytics

Coupled architecture, in terms of cloud resources used to host CF.Cumulus:

Orchestration: Microsoft Fabric
Compute: Microsoft Fabric
Storage: Microsoft Fabric

Using Microsoft Fabric

Fully decoupled architecture, in terms of cloud resources used to host CF.Cumulus:

Orchestration: Azure Data Factory
Compute: Azure Databricks
Storage: Azure Data Lake

Using Azure Data Factory & Databricks

How much does CF.Cumulus cost to use?

The CF.Cumulus framework is free for anyone to use on their own projects. We offer optional paid support for deploying, using and running solutions built on the framework.

Please see this document for indicative Azure run cost estimates for using the accelerator.

What is CF.Cumulus and how does it work?

CF.Cumulus is an open-source framework designed for building and managing cloud data platforms. It simplifies the process by providing tools for data ingestion, transformation, and orchestration, allowing you to focus on your data needs rather than infrastructure management.

What are the key features of CF.Cumulus?

Key features include metadata-driven workflows, seamless integration with various cloud services, support for large-scale data operations, and a robust set of tools for data ingestion, transformation, and orchestration.

How does CF.Cumulus simplify cloud data platform management?

CF.Cumulus automates many tasks related to data ingestion, transformation, and orchestration, reducing the need for manual intervention and allowing teams to focus on analysing data rather than managing infrastructure.

What benefits does CF.Cumulus offer for data ingestion and transformation?

CF.Cumulus provides scalable and efficient data ingestion pipelines, supports various data formats, and offers powerful transformation capabilities to prepare data for analysis and reporting.

Is CF.Cumulus suitable for large-scale data operations?

Yes, CF.Cumulus is designed to handle large-scale data operations with its scalable architecture, making it ideal for organizations with significant data processing needs.

How does CF.Cumulus integrate with existing data infrastructure?

CF.Cumulus integrates seamlessly with existing cloud services and data platforms, enabling smooth data flow and interoperability within your current infrastructure.

Is CF.Cumulus cost-efficient and scalable?

Its open-source nature and efficient resource management capabilities reduce costs, while its scalable architecture ensures it can grow with your data needs.

How does CF.Cumulus ensure data security and compliance?

CF.Cumulus includes robust security features, such as encryption and access controls, and adheres to industry standards to ensure compliance with data protection regulations.

Can CF.Cumulus support AI and machine learning workflows?

Yes, CF.Cumulus supports AI and machine learning workflows by providing the necessary infrastructure for data preparation, model training, and deployment.

How does CF.Cumulus handle metadata-driven workflows?

CF.Cumulus uses metadata to automate and manage workflows, ensuring consistency and efficiency in data processing tasks.

Is the complete framework available for use?

Yes, Control, Ingest and Transform are in use across a range of clients and industries today. Our product teams are beavering away to get the open source code released as soon as possible. If you want access to elements of the framework not yet open sourced simply reach out and ask.

What Our Clients Say

"Onboarding new datasets to the platform takes us minutes where it used to take days. CF.Cumulus has been a game changer for our data engineers."

Head of Software Engineering, Housing Sector

"We’ve reduced delivery lead times significantly and saved thousands in run costs."

Data Platform Lead, Retail Tech

"We finally have full visibility and control over our pipelines, without the usual complexity."

Principal Engineer, Public Sector

"CF.Cumulus helped us standardise our ingestion layer in record time. It's changed how we approach engineering."

Head of Data, Financial Services

Getting Started with CF.Cumulus

Ready to dive in and get going? Let's get you started.

There are several ways to get started with CF.Cumulus depending on your needs and level of readiness. Whether you want to deploy instantly with built-in support, require a tailored deployment, or prefer to explore independently, we’ve made it easy to choose the right path. Select the option below that best fits your current situation.

Get Started on Azure Marketplace

Deploy CF.Cumulus into your Azure subscription with just a few clicks. Choose the support plan that fits your needs and start onboarding data quickly with access to our knowledge base and onboarding materials.

Deploy with Azure Marketplace

Custom Deployment Support

Our Assisted Deployment route is designed for teams with complex requirements or enterprise constraints. We work closely with you to design your CF.Cumulus deployment to suit your architecture, processes, and operating model.

Assisted Deployment

Evaluate the Framework

Not ready to deploy just yet? Download our Product Pack to explore the architecture, plugins, deployment options, and use cases. It’s the ideal resource for assessing CF.Cumulus in your environment.

Download Product Pack

Explore the Community Edition

Prefer a DIY approach? Get started with the open-source Community Edition on GitHub. Access the source code, documentation, and templates to deploy CF.Cumulus independently, with the flexibility to scale later.

View on GitHub

Why Cloud Formations

Led by over 30 years experience in the data industry

Having recognised Microsoft Most Valuable Professionals (MVPs) on the team, highlights our deep knowledge and commitment to Microsoft technologies.

We proudly hold the title of Microsoft Partner, showcasing our deep expertise in Microsoft technologies and our commitment to delivering top-tier solutions.

As a trusted Databricks partner, we are at the forefront of data and AI technologies, offering unparalleled expertise in leveraging the power of Databricks for your business.

We are proudly certified with Cyber Essentials, demonstrating our dedication to maintaining robust cybersecurity practices and safeguarding your data.

We are committed to sustainability and environmental responsibility. Every month we plant 100 trees through Evertreen contributing to a greener future.

Strategic Services

Delivery Services

Members Area