Period-over-period engineering metrics: are we actually improving?
Period Comparison is the widget built to answer one specific question honestly: is delivery actually getting better, or does it just feel that way because last week was bad? It compares six delivery and quality KPIs between two adjacent 90-day windows, gives each one a direction-aware Improved/Regressed/Flat verdict — and then lets you drill into any row to see whether that verdict is a real trend or a headline about to reverse.
| Metric | Previous | Current | Δ | Verdict |
|---|---|---|---|---|
| Cycle Time | 34h | 29h | -15% | Improved |
| Issue Cycle Time | 6.2d | 5.8d | -6% | Improved |
| Deployment Frequency | 4 | 4 | 0% | Flat |
| Change Failure Rate | 9% | 13% | +44% | Regressed |
| Time to Restore | 3.1h | 3.6h | +16% | Regressed |
| Throughput | 142 | 158 | +11% | Improved |
Trailing 90 days vs. the 90 before, six KPIs.
What it measures, and where the data comes from
Six rows, always in the same order — speed first, then stability, then output: Cycle Time (PR first-commit to merge), Issue Cycle Time, Deployment Frequency, Change Failure Rate, Time to Restore, and Throughput. Issue Cycle Time is the newest row here — it's the median time from a Jira issue or ADO work item being created to reaching a done state, which matters because PR cycle time alone goes blind the moment a team opens and merges PRs in minutes; issue cycle time still catches the slow grind that happens before a PR ever exists. It ignores anything older than 90 days at close, so a mass historical cleanup can't fake an improvement.
- GitHub, GitLab, Azure DevOps — pull/merge request data feeds Cycle Time, Deployment Frequency (a merge-cadence proxy), and Change Failure Rate (a corrective-commit proxy).
- Jira, Azure DevOps — issue/work-item data feeds Issue Cycle Time. Time to Restore (bug-issue open-to-close) and Throughput run over the same full issues union used elsewhere in Deckgauge — Jira, GitHub, GitLab, and Azure DevOps — not just these two.
How to read it
The verdict column is direction-aware, not just "the number went up." A rising Deployment Frequency is Improved; a rising Change Failure Rate is Regressed — same arrow, opposite meaning, because the widget knows which way is good for each metric. Flat only appears when a KPI landed on the exact same value in both windows, which is rarer than it sounds; most rows will read Improved or Regressed even on a small move, so check the % delta before reacting to the color.
Then click the row. Expanding it shows a monthly sparkline of the same metric, graded Improving, Regressing, Flat, Stalling, or Recovering — and this is where the real signal lives. That grade compares the whole window's direction against just the last few months. A metric can Improve on the two-point headline while its trajectory grades Stalling, because the recent months are already reversing even though the average hasn't caught up yet. Trust the drilldown grade over the headline verdict when the two disagree.
What it tells you over time
This widget is already a "two windows" comparison by design, so the extra layer of caution is the drilldown, not a third window. A verdict that holds across several consecutive quarters, with a drilldown that reads Improving rather than Stalling, is the closest thing to proof this dashboard offers. A verdict that flips back and forth every quarter on the same metric usually means the window boundary is catching noise, not a real change — look at the sparkline's shape, not just which side of zero the last dot landed on.
Example situations
1. Multiple rows regressed at once
| Metric | Previous | Current | Δ | Verdict |
|---|---|---|---|---|
| Cycle Time | 28h | 26h | -7% | Improved |
| Issue Cycle Time | 5.1d | 5.0d | -2% | Improved |
| Deployment Frequency | 5 | 6 | +20% | Improved |
| Change Failure Rate | 8% | 17% | +113% | Regressed |
| Time to Restore | 2.9h | 5.4h | +86% | Regressed |
| Throughput | 150 | 162 | +8% | Improved |
Change Failure Rate and Time to Restore both regressed; the rest hold or improve.
What you're seeing: four green rows and a genuinely ugly pair — change failure rate more than doubled, restore time nearly doubled. Speed and volume look great. Stability is quietly falling apart underneath them, which is the classic shape of a team that traded quality for throughput and hasn't noticed yet.
How to react: don't try to fix all six rows in one push — that's how nothing gets fixed. Rank the regressed rows by which one actually threatens the roadmap, not which one is easiest to talk about. Here, Change Failure Rate is the one worth owning first; a climbing restore time is often just the downstream cost of the same underlying problem.
2. Improved on the headline, Stalling on the drilldown
Cycle Time
34h → 29h · Improved -15%
Headline verdict Improved; the trajectory says otherwise.
What you're seeing: the two-point comparison says Cycle Time improved 15% — true, start to end. But the monthly sparkline shows the real path: a sharp drop in the middle of the window, then climbing back up for the last couple of months. The average hasn't caught up to that reversal yet, so the headline still reads green.
How to react: don't take this to a QBR as a finished win. "Improved" on the two-point delta and "Stalling" on the trajectory is the widget telling you the recent months are already undoing the earlier gain — whatever process change caused the mid-window drop may have already stopped working, or quietly gotten reverted.
Frequently asked
- How do I tell whether engineering delivery is actually improving?
- Period Comparison contrasts six delivery and quality KPIs across two adjacent 90-day windows and gives each a direction-aware Improved, Regressed or Flat verdict, so improvement is judged against a comparable prior period rather than against a recent bad week.
- What do the drill-down grades mean?
- Drilling into a row shows whether the verdict reflects a sustained trend or a single distorting period. A headline Improved built on one exceptional week is graded differently from one built on steady movement.
- Why 90-day windows?
- Shorter windows are dominated by noise — holidays, one large release, a single absence. Ninety days is long enough for a genuine direction to separate from week-to-week variance.
Related widgets
- DORA Metrics — the same underlying proxies, read as absolute tiers instead of a delta.
- Velocity with Confidence — checks whether a throughput gain here is also getting more predictable, not just bigger.
- Investment Allocation — explains a Change Failure Rate regression as a mix shift rather than a mystery.
- Compare: Flow — the multi-board cousin of this widget: many boards over one window, instead of one board across two.
- Back to the widget reference.
Last updated