DEV Community

Mintu Ghosh
Mintu Ghosh

Posted on • Originally published at linkedin.com

Deploy only what changed - selective deployment in an Azure DevOps monorepo

#azuredevops #cicd #monorepo #microsoftfabric

Three teams share one repository. A commit touching one team's folder must not
redeploy the other two.

The obvious answer is trigger path filters. They do not work for this, and the
reason is worth understanding before you build anything.

This is Microsoft Fabric, but the mechanism is generic. If you have a monorepo
and an Azure DevOps pipeline, it applies.

Why path filters do not solve it 

trigger:
  branches:
    include: [develop, release, main]
  paths:
    include: [teams/alpha]   # <- decides IF the pipeline runs
Enter fullscreen mode Exit fullscreen mode

A trigger path filter answers one question: should this pipeline run at all. It
cannot tell a later stage which folder changed, because it produces no
variable anything downstream can read.

You could make one pipeline per team. Then every fix is made three times and
they drift.

The actual constraint

Azure DevOps builds its stage graph at compile time. Change detection can
only answer at run time. Stages cannot be created dynamically.

So generate every stage for every team, always, and let run-time detection
decide which execute. Unaffected ones render as Skipped.

That turns out to be better than dynamic generation. A skipped stage is visible
evidence the pipeline considered that team and chose not to deploy it. A graph
that never mentions the team proves nothing.

Choosing the diff baseline

There is no single "previous commit" that is right everywhere. Derive it from
the shape of HEAD:

parts = run_git(["rev-list", "--parents", "-n", "1", head], repo_root).split()
parent_count = len(parts) - 1

if parent_count >= 2:
    return head + "^1", head, "merge commit (first parent)"
if parent_count == 1:
    return head + "~1", head, "single parent"
return None, head, "root commit (no parent)"
Enter fullscreen mode Exit fullscreen mode

The pleasing part: Azure DevOps PR builds check out a merge of source into
target, so the first parent is the target branch before the merge. The
first-parent rule covers PR builds, merge-commit promotions and squash merges
without special-casing any of them. I never needed the System.PullRequest
variables.

Known gap, documented rather than engineered around: a multi-commit push is
evaluated from HEAD~1, so only the last commit is inspected. Promotion goes
through PRs where the full change set is visible, so this only affects direct
pushes to the validate-only branch.

The failure mode that looks like success

A shallow clone has no parent to diff against. Git returns an empty diff,
not an error. Every deployment skips and the run stays green.

That is a false negative that looks like a pass. Fail loudly instead:

if run_git(["rev-parse", "--is-shallow-repository"], repo_root) == "true":
    raise GitError(
        "This is a shallow clone, so there is no history to diff against.\n"
        "Set 'fetchDepth: 0' on the checkout step, or disable shallow fetch."
    )
Enter fullscreen mode Exit fullscreen mode

And on the detection job:

- checkout: self
  fetchDepth: 0
Enter fullscreen mode Exit fullscreen mode

I have a test asserting the pipeline still sets fetchDepth: 0, because it is
the kind of line someone removes while tidying.

Publishing the result 

for solution in registry.solutions:
    value = "true" if result["solutions"][solution.key] else "false"
    print("##vso[task.setvariable variable={};isOutput=true]{}".format(
        solution.key, value))
Enter fullscreen mode Exit fullscreen mode

Consuming it:

- stage: Deploy_UAT_ved
  dependsOn: DetectChanges
  condition: >-
    and(
      succeeded(),
      eq(dependencies.DetectChanges.outputs['Detect.detect.ved'], 'true'),
      eq(variables['Build.SourceBranch'], 'refs/heads/release')
    )
Enter fullscreen mode Exit fullscreen mode

Three things people get wrong here:

  1. Values are strings. eq(..., true) never matches. It must be 'true'.
  2. The reference has three parts — job name, step name, variable name: dependencies.<Stage>.outputs['<Job>.<step>.<var>'].
  3. The consuming stage must dependsOn the producing stage, or the variable cannot be resolved at all.

Two conditions, not one

Every stage carries both:

  • detection decides which solutions changed
  • branch decides where they may deploy

Both must hold. Detection never widens the branch rule and the branch rule never
widens detection.

This matters when reading results. A stage shows "Skipped" for either reason,
and they prove different things. On the production branch, every validate and UAT
stage skips for branch reasons while solutions still deploy. That is promotion
control, not isolation. Isolation is a stage skipping while a sibling at the same
tier succeeds.

I nearly captioned a screenshot wrong on exactly this point.

Resolving config without duplication

Solution plus environment becomes a variable name at compile time:

TARGET_WORKSPACE_ID: ${{ format('$({0}_{1}_WORKSPACE_ID)',
                        upper(parameters.solution),
                        upper(parameters.environment)) }}
Enter fullscreen mode Exit fullscreen mode

ved + UAT expands to $(VED_UAT_WORKSPACE_ID), resolved from the stage's
variable group at run time. The identifier never enters the repository.

Scope the deployment, and refuse an empty source

Two guards worth having, both learned the hard way.

Reject the repository root. A root-scoped deployment sweeps every team's
artifacts into one target, and the cleanup that follows is evaluated against the
wrong set.

Refuse an empty source folder. Orphan removal treats "present in target,
absent from source" as delete. An empty folder therefore means everything in
the target is an orphan. A folder created but not yet populated is a normal
intermediate state, so it has to fail safely rather than proceed.

if item_count == 0 and not read_bool("ALLOW_EMPTY_SOURCE"):
    fail("Solution has no items. Refusing to deploy: orphan removal would "
         "treat every item in the target as an orphan and delete it.")
Enter fullscreen mode Exit fullscreen mode

Test it without touching the cloud

The classifier is a pure function: changed paths in, decision out. That makes
the whole matrix testable offline.

def test_d_ved_and_hr(self):
    self.assertAffected(
        ["fabric-workspaces/Ved/Notebook.Notebook/notebook-content.py",
         "fabric-workspaces/HR/Notebook.Notebook/notebook-content.py"],
        ["ved", "hr"],
    )
Enter fullscreen mode Exit fullscreen mode

70 tests, stdlib unittest, no pytest, no cloud access, under a tenth of a
second. The six scenarios that matter — one team, each other team, two teams,
all three, docs-only — are verified before any pipeline runs.

One test I had to rewrite: it asserted a folder held exactly four items. True
when written, false the moment someone authored a fifth. Do not couple tests to
live content; use fixtures for exact counts.

Verified end to end

Branch Detected Result
develop one solution, 2 files that one validated, other two skipped
release one solution, 4 files that one deployed to UAT, other two skipped
main all three, 39 files all three to PROD, all validate and UAT stages skipped

Edge case that works: item names containing spaces.

What it still does not do

  • No approval gates
  • Multi-commit pushes inspect only the last commit
  • A shared framework change deploys nothing by design; there is a manual override parameter for rolling one out deliberately

Full walkthrough: [https://www.linkedin.com/pulse/enterprise-microsoft-fabric-cicd-selective-one-azure-devops-ghosh-uxzsf/].
Code, including the test suite:
[https://github.com/vedaforge-team/fabric-cicd-reference/tree/v2.0.0].

Top comments (0)