--- slug: risk-based-review type: pattern summary: "Set how much human review a change gets from what it would cost to get wrong, with rules rather than the author deciding the tier." created: 2026-10-23 updated: 2026-10-23 related: code-review: relation: refines note: "Risk-based review keeps code review and varies how many people it takes according to what the change could break." review-bottleneck: relation: mitigates note: "Moving low-risk changes off the human queue frees scarce reviewer attention for the changes that need it." blast-radius: relation: depends-on note: "A change's tier comes mostly from how much would go wrong if it were wrong." bounded-autonomy: relation: specializes note: "Risk-based review is bounded autonomy applied to merging code: the tier decides how far a change travels without a person." approval-policy: relation: extends note: "Approval policy tiers an agent's runtime actions; risk-based review carries the same tiering to the merge gate." agentic-pull-request: relation: depends-on note: "A low-risk lane is only as trustworthy as the tests, rationale, and session record the pull request carries." rollback: relation: depends-on note: "Merging without a person is defensible only when a bad change can be undone quickly." feature-flag: relation: uses note: "Shipping new behavior dark behind a flag lowers a change's tier, because turning the flag on becomes a separate decision." merge-queue: relation: uses note: "The merge queue still retests low-risk changes against everything ahead of them, so a lighter review doesn't mean a lighter integration check." progressive-delivery: relation: uses note: "Staged rollout catches what lighter review misses before it reaches every user." permission-classifier: relation: informed-by note: "A permission classifier makes the same proceed-or-escalate call for an agent's actions and fails the same way, by waving through a dangerous case." human-in-the-loop: relation: refines note: "The tier decides where in the merge path the human stays and where automation stands in." ownership: relation: depends-on note: "Someone still answers for every merged change, whichever lane it took." trusted-monitoring: relation: uses note: "A review agent can read every change and raise its tier, but its approval doesn't stand in for a person's." approval-fatigue: relation: mitigates note: "Taking routine changes out of human queues leaves reviewers fewer, more consequential decisions to make." metric: relation: measured-by note: "Revert and incident rates per tier show whether the scheme saves human time or only moves the cost downstream." --- # Risk-Based Review > **Pattern:** A named solution to a recurring problem. *Set how much human review a change gets from what it would cost to get wrong, so reviewers spend their attention where a mistake would hurt.* *Also known as: Risk-Proportional Review, Risk-Tiered Approval, Tiered Code Review* At nine in the morning a reviewer opens a queue of 40 pull requests. Most are dependency bumps, copy fixes, and test additions an agent wrote overnight. One of them, four lines long, changes how long a login token stays valid. Under the usual rule, every change needs one approval from someone other than the author, so all 40 get the same treatment, and the one that matters gets the same 90 seconds as the rest. ## Understand This First - [Code Review](code-review.md) — the practice this pattern tiers. - [Review Bottleneck](review-bottleneck.md) — the throughput problem that makes uniform review unaffordable. - [Blast Radius](blast-radius.md) — the main property that sets a change's tier. ## Context This is a **tactical** pattern, applied at the point where a change asks to merge into the shared codebase. It matters most where agents write a large share of the code. Agents multiplied the number of changes without multiplying the number of people who can read them. Anthropic's Claude Code team reported that its monorepo went from about 2,500 changes a day in May 2026 to more than 30,000 in September. No staffing plan absorbs a twelvefold jump in reading. Many teams still run one approval rule for every pull request. That rule made sense when people wrote all the code and the volume stayed within what peers could read. It gives a typo fix and a payment-path change the same claim on a person's time. That was always a little wasteful. Now it's the [Review Bottleneck](review-bottleneck.md) that caps what the team can ship. ## Problem How do you keep careful human judgment on the changes that can hurt you when there are far more changes than people can read? A uniform rule fails in both directions. Hold it strict and reviewers drown, start skimming, and slide into [Approval Fatigue](approval-fatigue.md), so the dangerous change gets a rubber stamp along with everything else. Relax it across the board and the dangerous change gets no human at all. ## Forces - Reviewer attention is fixed, and the number of changes arriving keeps growing. - What a mistake costs varies enormously between changes, and diff size is a poor guide to it. - Whoever classifies a change is tempted to call it low risk, and the author or authoring agent is the least reliable judge of its own work. - Automated reviewers read everything and catch a lot, but they miss things and can't be held accountable the way a person can. - Any lane that skips a person needs some way to notice when it starts letting bad changes through. ## Solution **Classify every change by what it would cost to get wrong, and let that class decide how many people must approve it. Rules or a reviewer other than the author set the class; the author never does.** Start with what sets the tier. The questions are the ones you'd ask about any risky operation: what breaks if this is wrong, for whom, and how fast can you undo it? Some paths answer the first question on sight. Treat these as high risk by default: authentication and permissions, billing, database migrations, CI and deployment configuration, infrastructure definitions, audit logging, and wire formats other services depend on. Beyond the path, look at [Blast Radius](blast-radius.md), reversibility (can [Rollback](rollback.md) or a [Feature Flag](feature-flag.md) undo it in minutes?), the test evidence the change carries, and how long the code will live. Line count barely matters. When Rewind built such a system, its engineers found that a pull request's risk had almost nothing to do with its size. A three-line permissions change can outrank a thousand lines of generated documentation. Then decide who sets the tier. Put the floor in deterministic path rules, kept in a file that only designated owners can change, so neither the authoring agent nor a reviewing model can rewrite the policy that governs it. A review agent may read the change and *raise* its tier when it finds something the rules didn't anticipate, but it can't lower a tier the rules set. TradersPost states the principle plainly: rules, not the authoring agent, decide the risk. Authors don't classify their own work either; at Ona, engineers can't mark their own pull requests as low risk. Anthropic lets a classifier pick the tier and guards the lighter lanes another way: only authors on a repository's trusted list, mostly the team that owns it, can use them at all. Commercial review tools are building in the protected-policy idea too. Cursor's approval agent won't approve a pull request scored above a configured risk threshold. When a change edits an approval policy file, the agent reads the base branch's version of the policy or hands the decision to a person. Map each tier to a review requirement. A common shape has three levels: - **Low:** automated checks, security scanning, and a review agent's pass, with no human approval or a light one. - **Ordinary:** one person who didn't write the change. - **High:** one or two people who own the affected area, whatever the size of the diff. Decide, in writing, what a machine's approval is allowed to satisfy. This is where teams disagree most, and the choice should follow how cheap your rollback is. Anthropic lets low-risk changes from trusted authors merge with no human approval once automated review and security checks pass. Rewind and Ona let a bot post the approving review and keep a person on the merge button, because approving and merging are different acts. Ona requires that human click on every change; Rewind requires it whenever an agent wrote the code. TradersPost treats a bot's approval as advisory only; it never satisfies a code-owner requirement. All of them agree on one point: an agent's approval is never counted as a person's. When a tier calls for a human, a review agent's sign-off doesn't fill the seat. Keep a safety net behind the light lanes. A [Merge Queue](merge-queue.md) still retests each change against the ones ahead of it, [Progressive Delivery](progressive-delivery.md) limits who sees a bad change first, and rollback limits how long they see it. Lighter review is a bet that these layers catch what a reader would have caught. If you can't undo a change within minutes, it doesn't belong in a lane with no reader. Watch the scheme with numbers. Track revert and incident rates by tier, and have a person also review a random sample of low-tier changes to see how often they'd have disagreed with the automation. Anthropic's team frames the test bluntly: if revert and incident rates rise, you haven't saved human time, you've only moved it. Ownership doesn't change with the lane, either. Whoever prompted the agent that wrote the change answers for it, and at Anthropic that person counts as one of the change's required reviewers. > **💡 Tip:** Start with path rules alone and route everything they don't match to the ordinary tier. Add a classifier later, and only as a signal that can raise a tier. A rules file you can read in two minutes is easier to audit than a model's judgment, and it fails in ways you can predict. ## How It Plays Out An agent opens a pull request touching 14 files: regenerated API reference pages, two copy fixes, and a new export screen behind a feature flag that defaults to off. None of the paths match the team's high-risk rules. Tests and the security scan pass, and the review agent finds nothing that raises the tier, so the change lands in the low lane. Under this team's policy a bot approves it and the developer who prompted the work clicks merge after a skim of the summary. Nobody spends an hour reading generated documentation. Turning the export screen on next week will be its own change, and the team's rules put every flag flip in the ordinary tier, so a person will see that one. The same morning another agent fixes a failing nightly job by widening its cloud access policy from one storage bucket to a wildcard. The diff is three lines. A size-based heuristic would wave it through, but the path matches the infrastructure-permissions rule, so the change needs two approvals from the platform security owners. The first of them notices that the wildcard also grants write access to the bucket holding audit logs. The agent narrows the grant to the two buckets the job actually reads, and the change merges that afternoon. A third team runs a tiered scheme for a quarter and reviews the numbers. Low-tier reverts are creeping up. The cause turns out to be a shared date-formatting library: each change to it was small and well tested, so the classifier kept rating it low, but invoicing and payroll both depend on it. The team adds the library's path to the high-risk rules and lowers its trust in the classifier for shared code. The scheme did its job here because someone was watching the revert rate. Without that number, the drift would have surfaced as an invoicing incident. ## Consequences **Benefits.** Reviewers spend their limited attention on the changes that can do damage, and they read those more carefully because they aren't wading through dozens of trivial ones first. Routine changes stop waiting. Ona reported that its median time to first approval fell from 2 hours 49 minutes to 3.8 minutes after its bot began approving qualifying pull requests. The rules file also becomes a written statement of what the team considers dangerous, which is useful to new engineers and to the agents reading the repository's instructions. **Liabilities.** Misclassification is the central risk, and the costly direction is a dangerous change rated low. Rules files go stale as code moves, and a classifier trained on yesterday's incidents won't anticipate tomorrow's. Authors, human or agent, may learn to shape changes so they stay under the line, which is why rules should key on what a change touches rather than how big it is. Light lanes are only as safe as the rollback, flags, and staged rollout behind them, and a team without those has no business removing people from the path. Fewer human reads also means fewer people know the code, so the knowledge-spreading benefit of [Code Review](code-review.md) shrinks for low-tier work. The guardrail metrics lag. Revert and incident rates tell you the scheme is failing after it has failed. Anthropic's team says it doesn't yet have circuit breakers that tighten approval automatically when those rates move, and its engineers want them before more code merges with no human. Build the tightening step before you need it, even if it starts as a person checking a dashboard every week. The evidence base is thin. Every public account of this pattern comes from the organization that built the system, the samples are small, and none publishes an independent study of outcomes. Rewind reported that about 40% of pull requests came back auto-approved and that people agreed with the bot's call about 99% of the time when both looked at a change. No revert or incident had been traced to a bot-approved change, and the write-up itself calls these results directional. Anthropic reported that merged pull requests to Claude Code more than doubled from June to September 2026 while the share reverted fell by more than 40%. One team's before-and-after trend can't separate the effect of tiering from anything else that changed in those months. Treat the pattern as sound in principle and young in practice. Start conservative and loosen tiers only as your own numbers justify it. ## Sources - Barry Boehm's "[Software Risk Management: Principles and Practices](https://doi.org/10.1109/52.62930)" (*IEEE Software*, 1991) framed risk as the probability of a loss times its size and argued for spending management attention in proportion to that exposure. Risk-based review applies the same arithmetic to reviewer time. - James Bach's "Heuristic Risk-Based Testing" (*Software Testing and Quality Engineering*, 1999) made the parallel argument for testing: concentrate test effort where failure is both likely and costly instead of spreading it evenly. Risk-based review carries that reasoning from test effort to approval requirements. - Anthropic's Claude Code team described tiered, risk-based human approval for an agent-heavy monorepo in "[How the Claude Code team does code review](https://claude.dev/blog/solving-code-review-at-anthropic/)" (2026). Its rules: the change sets the tier, only trusted authors get the lighter lanes, an agent's approval never counts as a person's, and revert and incident rates are the guardrail. - Dave North's "[Review is the bottleneck now: How we let AI approve pull requests (safely)](https://rewind.com/blog/ai-approve-pull-requests-safely/)" (Rewind, 2026) separated risk from diff size, kept the deterministic policy in a file protected by code ownership, and drew the line between automated approval and automated merge. - Benjamin Stark's account of [auto-approving low-risk pull requests at Ona](https://ona.com/stories/auto-approving-low-risk-prs) (2026) defined eligibility by excluded change types rather than by author judgment and reported the latency effect cited above. - Jonathan Wage's write-up of [risk-proportional review for AI agents at TradersPost](https://blog.traderspost.io/article/risk-proportional-review-for-ai-agents) (2026) stated the principle that rules, not the authoring agent, decide a change's risk, let the agent raise a tier but never lower it, and kept the low tier's bot approval advisory. - Cursor's documentation for [PR Routing & Approval](https://cursor.com/docs/approval-agents) describes a released approval agent that refuses to approve above a configured risk threshold and treats edits to its own approval policy files as a case for the base-branch policy or a human reviewer. - Cristina Buenahora's "[A practical guide to risk-based code review](https://www.cortex.io/post/risk-based-code-review)" (Cortex, 2026) named the practice and tied review rigor to blast radius, reversibility, test coverage, and how long the code will live. --- - [Next: Printf Debugging](printf-debugging.md) - [Previous: Review Bottleneck](review-bottleneck.md)