Deploy free

Review mix: splitting bot vs human code review, PR by PR

Review Mix takes each merged PR's first non-comment review — the first approval or changes-requested, not a drive-by comment — and sorts it into bot vs human code review by reviewer login. From there it shows bot share of first reviews, bot share of comment-only activity, and median hours-to-first-review for each type, plus a weekly bot-share line. GitHub only, for now.

deckgauge · Board · Intelligence
Bot review share
58%
Bot pickup (p50)
6m
Bot share of first reviews — weekly

Bot handling most of the first pass, fast.

What it measures, and where the data comes from

The unit here is a PR's first substantive review — an approval or a changes-requested, whichever lands first. Comments-only activity doesn't count toward that classification, but it does get its own separate share (comment share), because a reviewer who only ever leaves comments is doing a different job than one who approves or blocks. Reviewer login gets matched against known bot patterns — service accounts, [bot] suffixes, CI review apps — to decide bot or human.

  • Bot share of first reviews — of all merged PRs with a first review, what fraction were picked up by a bot account.
  • Bot share of comment-only reviews — same idea, restricted to comment-only activity, so you can see whether your bots mostly gate merges or mostly just leave notes.
  • Median hours-to-first-review, bot and human separately — how long each type actually takes to show up.

GitHub only. GitLab and Azure DevOps aren't wired into this one yet — if most of your review activity lives there, this widget will read empty or misleadingly quiet.

Login pattern, not ground truthBot vs human here comes from matching the reviewer's account against known bot login shapes, not a verified flag from the platform. That cuts both ways: a human reviewing through a shared or oddly-named service account can get counted as a bot, and a homegrown review bot with a normal-looking username can slip through as human. Treat the split as directional. Don't quote "58% bot-reviewed" to a VP as an exact headcount — it's a good-faith estimate, not an audited one.

How to read it

Compare the two pickup times before you look at anything else. Bots that pick up in minutes while humans take hours isn't a story about human reviewers being slow — it's usually the bot correctly absorbing the mechanical first pass (lint, format, obvious diffs) so the human reviewer's clock only starts once there's something worth their judgment.

Then check comment share against review share. A bot with high comment share but low review share is mostly annotating, not gating — useful, but it isn't actually reducing the human review queue the way a high review-share bot is.

What it tells you over time

A bot-share line climbing over weeks means more of your first-look coverage is automated — that's usually a good trend, as long as the humans in the loop are still catching the changes that actually matter. Watch it alongside merge velocity: if bot share is climbing and lead time isn't improving, the bot is filling time that wasn't your bottleneck.

Example situations

1. Bots pick up instantly, humans stay slow

deckgauge · Review Mix
Bot pickup (p50)
4m
Human pickup (p50)
11h
Bot share holding steady around 65%

Bot pickup near-instant; human pickup unmoved.

What you're seeing: the bot is absorbing the easy first pass almost instantly, but human pickup hasn't budged — it's still an 11-hour wait for anything that needs a real reviewer. The bot handling volume didn't translate into faster human attention.

How to react: this is fine, not broken — bots and humans are doing genuinely different jobs here, and you shouldn't expect the bot's speed to rub off on the humans automatically. The actual question is whether human pickup time is a problem on its own terms, independent of the bot.

Managerial playLet the bot keep the trivial pass — don't ask a human to double-check what it's already reliably catching, that just adds a queue with no value. Then treat the 11-hour human pickup as its own problem: check Review Pickup Time for the SLA playbook rather than assuming the bot will eventually fix it. Bots absorbing easy work is supposed to protect human review time for substance — it only works if you actually spend that protected time on substance, not on re-reviewing what the bot cleared.

2. Bot review share rising quarter over quarter

deckgauge · Review Mix
Bot review share
76%
Bot comment share
30%
Bot share of first reviews — climbing steadily

Bot share climbing from 40% to 76% over two months.

What you're seeing: more and more of the first-look coverage on merged PRs is going to a bot, almost double where it started. First-look review is increasingly automated across the whole repo, not just on a subset of trivial PRs.

How to react: more automated coverage is good news for throughput, but it raises the stakes on the remaining human review — if the bot is the one deciding what "looks fine," a risky change slipping through on a bot approval is a real gap, not a theoretical one.

Managerial playSet an explicit rule for what always gets a human sign-off regardless of what the bot says — anything touching auth, payments, migrations, or infra, no exceptions. Everything else can keep riding the automation trend. Revisit that carve-out list every quarter as the bot's coverage grows; the goal is a bot handling more of the safe, boring volume, never a bot quietly becoming the only reviewer on something that mattered.

Frequently asked

How do I measure bot versus human code review?
Review Mix takes each merged pull request's first non-comment review — the first approval or changes-requested, not a drive-by comment — and classifies it by reviewer login, then reports bot share of first reviews, bot share of comment-only activity, and median hours to first review for each type.
Why does the bot versus human split matter?
Because a rising bot share of first reviews can make review pickup time look excellent while human scrutiny quietly falls. Separating the two keeps a fast number from hiding a real change in rigour.
Where can the classification misfire?
It works by reviewer login, so a bot using a human-looking account, or an engineer whose account name resembles a bot, will be sorted incorrectly. It is GitHub-only at present.

Related widgets

Last updated