Azure DevOps
DORA metrics for Azure DevOps: what ADO gives you, and what it doesn't
The short version.
- Azure DevOps does not compute DORA metrics. It ships Analytics views and dashboard widgets, not the four keys.
- It does hold almost all the raw material: Repos, Pipelines, Releases and Boards.
- Deployment frequency and lead time for changes are derivable from ADO data alone.
- Change failure rate and time to restore are not — both need to know which deploy broke production, and ADO does not record that.
- Anyone claiming all four straight out of Azure DevOps has quietly redefined two of them. Ask which.
What are DORA metrics, and where does Azure DevOps hold the data?
The four DORA metrics measure software delivery performance: deployment frequency (how often you ship to production), lead time for changes (how long a commit takes to reach production), change failure rate (what share of deployments cause a degradation needing remediation), and time to restore service (how long recovery takes). They come from the DORA research programme, now published as the annual DORA State of DevOps report.
Azure DevOps is unusually well placed to answer them, because it holds both ends of the delivery pipeline in one product — which is exactly why the gap is frustrating:
| DORA metric | Where the data lives in ADO | Derivable from ADO alone? |
|---|---|---|
| Deployment frequency | Pipelines runs and Releases — each successful production deployment is a record with a timestamp | Yes |
| Lead time for changes | Repos (commit timestamp) joined to the Pipelines/Releases record that carried it | Yes, once you define which stage counts as production |
| Change failure rate | Nothing records "this deployment caused a problem". Closest proxies: failed pipeline runs, or Boards work items typed as bugs or incidents | No — needs an incident signal you supply |
| Time to restore service | Only if you open and close incident work items in Boards with reliable timestamps | No — needs an incident process, not just a tool |
Why do two of the four resist measurement?
Because they are not really about deployments — they are about failure, and failure is a judgement your organisation makes, not an event your CI system emits. A pipeline that goes red is not a change failure: it never reached production, so it is the system working. A pipeline that goes green and takes checkout down for forty minutes is a change failure, and Azure DevOps has no idea.
This is why so many "DORA for Azure DevOps" dashboards quietly substitute pipeline failure rate for change failure rate. The numbers look plausible and move in believable directions. They measure something real — build and release reliability — but it is not the DORA metric, and it will tell you your delivery is healthy on the same week you had three production incidents. If you take one thing from this page, take this: before trusting a change failure rate, find out what it counts.
The honest fix is process, not tooling. To measure change failure rate you need a rule your team actually follows — for example, every production incident gets a work item of a specific type, opened when detected and closed when resolved, with the deployment it followed noted on it. Once that rule holds, both remaining metrics fall out of the data. No tool can install the rule for you.
What are the options for Azure DevOps, and what do they cost you?
The field splits into four shapes, and the trade-off in each case is about where the data lives and how far it can see:
| Approach | Good for | Where it hurts |
|---|---|---|
| Marketplace extension — a DORA widget inside ADO | Fastest possible start; nothing to host; lives where your team already works | Scoped to what one project can see. Cross-project or cross-tool reporting is awkward to impossible, and you inherit the extension's definitions |
| Power BI over Analytics | Total control of definitions; joins to business data; likely already licensed | You are building and maintaining a data model. The four keys are a project, not a checkbox |
| Commercial SaaS — Datadog, Keypup, LinearB, Jellyfish and similar | Polished, supported, usually broader than DORA alone | Per-seat or per-contributor pricing, and your commit and work-item history sits in a vendor's cloud |
| Self-hosted open source | No per-seat cost; data stays on your infrastructure; definitions are auditable because you can read the code | You run it, upgrade it and back it up. Azure DevOps support is thinner than GitHub support across most of this category |
That last row is worth dwelling on, because it is the real gap in this space. Most open-source engineering-metrics tools were built GitHub-first, and Azure DevOps arrived later as a secondary connector — which shows up as shallower work-item support, missing release data, or a repo scope you cannot control. If you are an ADO shop, check the Azure DevOps connector specifically rather than the feature list on the front page.
How does Deckgauge handle Azure DevOps?
Deckgauge is an open-source, self-hosted engineering intelligence platform, and Azure DevOps is a first-class source rather than an afterthought. Being specific about what it does and does not do:
- Work items sync per attached project. Attach an ADO project to a board and
its work items flow in — name, status, owner, description and the work item's
Microsoft.VSTS.Scheduling.DueDate— alongside issues from Jira, GitHub or GitLab on the same board. - Pull request and commit analytics are strictly opt-in. They stay off until you enable code intelligence on the board source and switch on PR and/or commit sync for the project, scoped to either a named repo list or an explicit "all repositories" toggle. There is no wildcard, and a newly attached project pulls no code data at all until you say so.
- Deployment frequency comes from Pipelines and Releases, with which stages count as production decided when the metric is read rather than baked in at ingest — so changing your mind about what production means does not require re-syncing history.
- Change failure rate is not conjured from pipeline failures. Consistent with the section above, Deckgauge will not report a change failure rate it cannot substantiate rather than substitute a proxy that looks like one.
- Status history drives flow metrics. Cycle time, throughput and in-progress time come from the work item's revision history, not from a single current status, so a ticket that bounced between states is measured as it actually moved.
It is free, source-available under the Functional Source License, has no per-seat pricing, and runs on your own infrastructure with Docker. The full Azure DevOps connection guide covers the token scopes, the opt-in switches and the failure modes in detail.
How do you set this up without lying to yourself?
- Define production once, in writing. Name the pipeline stages or release environments that count. Every metric below inherits this decision, and teams that skip it spend the next quarter arguing about the numbers instead of the delivery.
- Start with the two you can actually measure. Deployment frequency and lead time for changes need no new process. Get them right, watch them for a month, and resist reporting the other two until they mean something.
- Install the incident rule. One work item type, opened at detection, closed at resolution, linked to the deployment it followed. This is the whole cost of the remaining two metrics.
- Report at team level, as a trend. DORA metrics compare a team to its own past. Ranking teams against each other with them produces gaming, not improvement — a failure mode worth reading about in DORA metrics without gaming them.
- Check the definition your tool uses before you trust the chart. Especially change failure rate. Especially if it was easy to turn on.
Frequently asked
- Does Azure DevOps calculate DORA metrics natively?
- No. Azure DevOps ships Analytics views, dashboard widgets and Power BI connectivity, but it does not compute the four DORA metrics for you. It holds nearly all of the raw material — Repos for commits, Pipelines and Releases for deployments, Boards for work items — but joining those into deployment frequency, lead time for changes, change failure rate and time to restore is left to you or to a tool on top.
- Which DORA metrics can you get from Azure DevOps data alone?
- Two of the four cleanly. Deployment frequency comes straight from Pipelines and Releases, and lead time for changes is computable because ADO holds both ends of the window — the commit and the deployment that carried it. Change failure rate and time to restore both need to know which deployments caused a production problem, and that is an incident signal Azure DevOps does not record on its own.
- What is the difference between an Azure DevOps DORA extension and a self-hosted tool?
- A Marketplace extension renders inside Azure DevOps and is limited to what one project can see, which makes cross-project and cross-tool views hard. A self-hosted tool pulls ADO through its REST API into its own store, so it can join Azure DevOps to Jira, GitHub or GitLab in one view and keep history beyond ADO retention. The trade-off is that you run and upgrade it.
- Can you measure DORA across Azure DevOps and GitHub together?
- Only with a tool that ingests both, and only if you decide up front what a deployment means on each side. Teams frequently run ADO Boards with GitHub repos, or migrate from ADO to GitHub over months, and during that period a single-platform tool reports half the picture. Deckgauge connects Jira, GitHub, GitLab and Azure DevOps to one board for this reason.
- Do I need Azure DevOps Server, or does Azure DevOps Services work?
- Both expose the same REST API surface for this purpose. Deckgauge authenticates with a personal access token (or basic auth with a username) against your organization URL, so a cloud Azure DevOps Services organization and a self-hosted Azure DevOps Server both work, provided the token carries read scopes for Work Items and — if you want PR and commit analytics — Code.
If you are also pulling work items from Jira, the companion piece is can you get DORA metrics from Jira? — the same honest accounting, applied to a tool that holds even less of the deployment side. For the wider field, see self-hosted DORA metrics: the open-source options.