Fabric - Source Control & Branching - Part 1

Developing in Microsoft Fabric within a multi-team environment has not always been straightforward. Fabric was initially geared more towards individual users and citizen developers working independently within a workspace, which created challenges when applying traditional source control and DevOps practices.

Common issues included:

• Source control branches being linked at the workspace level, meaning one user switching branches could affect everyone using that workspace.

• Branch changes potentially overwriting or removing workspace items.

• Developers needing permission to create workspaces and assign them to a Fabric capacity before they could branch into their own development environment.

• Uncertainty over which tasks should be managed in Fabric and which should be handled through Azure DevOps.

• Unfamiliarity with DevOps Pipelines

Over the past year, however, Microsoft Fabric has matured significantly. Its source control and deployment capabilities have improved, supported tooling has expanded, and practical development patterns have emerged that reduce friction between Fabric and Azure DevOps.

So I've create three posts that go through the process

These posts covers:

• Deploying Fabric items through Azure DevOps using the Microsoft-supported Fabric CI/CD Python library.

• Creating an Azure DevOps YAML deployment pipeline.

• Using individual development workspaces to support multiple developers and branches.

• Deciding which activities should take place in Fabric and which should be managed through Azure DevOps.

• Using AI Tools to help speed up development of DevOps Pipelines

By the end of these posts, you will have a practical framework for creating a controlled, scalable, multi-user development environment in Microsoft Fabric.

Workspace & Branch Setup

I'm going to assume a few things:

1. You have DevOps or some other source control

2. You are a little familiar with the concepts of branching and pull requests

3. You have a Dev, Test, Prod workspaces to push changes to

Below is an example path of main, dev, and feature branches, in a nice diagram, however don't worry it is mostly there to look nice.

This section will not be an in-depth discussion on CI/CD source control and branching (Lucky you), but here are the main highlights. You'll have 'main' which should match what you have in production, and you should have a 'development' branch which should be items ready to be tested then go into production. That's it. However Fabric doesn't play nice with git on the workspace level.

In the workspace setting you can set up the source control. Here I have a Azure DevOps repo, that is set to the main branch. That's fine until you need to have a multi developer team working on things. You change the branch here, it changes it for everyone. So in your Dev workspace how do you handle this switch between branches. You don't...this should always link to your 'Development' branch.

You can create a feature branch that the dev team can work on, both in Fabric and DevOps, from what ever branch to a 'feature' workspace you want to use, so lets see how you can do it in Fabric.

Fabric Based Branching

So under the source control setting, you can change branches, but also 'Branch out to workspace', this will be the feature we can use. If you click on the reverse 'y' on the right, you'll get the branching options.

You can switch branches, and also the option to branch to a workspace. This will create a branch in the DevOps repo, in Fabric, and the repo will be deployed to the nominated workspace or create a new one. If creating a new one, you must have the right to allocate a capacity to a workspace, otherwise it will not work. If deploying to an existing workspace, it must be backed by a capacity.

You have in the branch to workspace, a few options, existing or new workspace, what branch it is based on, the name of the branch and a few warning/prompts of you can wipe stuff out. So in this case it will branch out and overwrite items to a workspace called Fabric PoC. You don't have the full DevOps experience in Fabric, so none of the lifecycle management of branches is in there, just create.

DevOps Based Branching

In DevOps, you can create a branch in the repo:

You do have the option of linking it to a DevOps work item if you wish, which you don't get to do in the Fabric interface.

In DevOps you can't allocate it to a workspace. You'll have you go into a Fabric workspace and select the branch in the workspace Git options.

User Workspace Setup

Rather than creating new workspaces for features time and time again, I'm going to recommend creating a workspace for each of your developers working on Fabric. They then can adjust the workspace to the branch they are working, then once complete, use a pull request back into the development branch. This way the workspace is a one time setup, you don't have to give any extra permissions to allow them to assign the workspace to a capacity, it's set up and done.

One other recommendation is the use of Fabric Variable Libraires, so notebook and data pipelines parameter and connections etc can be updated quickly. You will have to be careful, not to commit workspace specific variable back to the development branch. See the CF blog post on Variable Libraries and how they can help.

Do I do this in DevOps Or Fabric?

Because Microsoft Fabric and Azure DevOps both support parts of the development workflow, it is important to be clear about which platform should be used for which activity.

Let Fabric handle Fabric workspace integration, and allow DevOps to handle source control stuff like creating branches, pull requests and all that.

Fabric

1. Assigning a workspace to a branch

2. Connecting a workspace to source control

3. Committing Fabric item changes back to the branch

4. Updating the workspace from the branch

For 3, lets use the commit message, I know it sounds like documentation, but it helps keep track of things.

DevOps

Azure DevOps should be used for all the source control stuff

• Creating branches

• Deleting branches

• Managing pull requests

• Reviewing code and Fabric item changes

• Merging changes between branches

• Linking work items, tasks, bugs, and user stories

• Managing approvals

• Running deployment pipelines

• Promoting changes between environments

This approach is great for a multi-team environment because it reduces the risk of developers getting all mixed up each other by switching shared workspace branches. They can work in their own Fabric workspaces, commit changes to their assigned branches, and then use DevOps to manage review, merge, and release items.

The guiding principle is:

Let Fabric do the Fabric workspace work. Let DevOps do the DevOps

Overview

Now we have a good place to start from:

• Dev team get their own workspace

• Branches get allocated to workspaces

• DevOps is the prime source of source control

• We know what each platform responsibilities are

So how to deploy stuff? That's in Part 2