The PR Pattern That Predicts 73% of Your Incidents

Not all pull requests carry the same risk. A one-line typo fix is not the same as a 500-line refactor of your payment processing code. Yet many teams treat all merges the same way—and then wonder why production incidents keep happening.

This guide shows you how to identify high-risk pull requests before they merge, set up automated detection, and respond appropriately to risk signals without slowing down low-risk changes.

What Makes a Deployment Risky

Risk Is Contextual

A change that's risky for one team might be routine for another. Risk depends on:

• What's being changed: Core business logic vs. documentation
• How it's being changed: Small tweak vs. architectural overhaul
• Who's changing it: Domain expert vs. first-time contributor
• When it's being changed: Monday morning vs. Friday at 5pm
• How it's reviewed: Thorough review vs. rubber stamp

Categories of Risk

Risk signals generally fall into these categories:

Size-based risk:

• Large PRs (500+ lines) are harder to review thoroughly
• Many files changed increases chance of unintended consequences
• Large deletions can accidentally remove needed code

Review-based risk:

• Merged without approval (skipped review)
• Self-merged (author approved their own code)
• Rubber-stamped (approved in under 5 minutes with no comments)
• Single reviewer for critical code

Process-based risk:

• Failing CI checks (tests, linting, security scans)
• Missing required reviewers (CODEOWNERS not satisfied)
• Force-pushed after approval

Timing-based risk:

• After-hours merges (no one around if something breaks)
• Friday deployments (weekend to find and fix issues)
• During incident response (adding changes to already-unstable system)

Content-based risk:

• Sensitive files (authentication, payments, PII handling)
• High-churn files (already problematic areas)
• Configuration changes (can affect entire system)

Risk Signals from PR Patterns

Signal 1: Large PRs

Research consistently shows that defect density increases with PR size. Google's internal data suggests review effectiveness drops significantly after 200-400 lines.

Signal 2: Rubber-Stamp Reviews

A review approved in under 5 minutes with no comments is unlikely to have been thorough. This is especially risky for large or complex PRs.

Signal 3: Merged Without Approval

PRs that merge without any approval bypass your review process entirely. Sometimes this is intentional (hotfix by senior engineer), but it should be tracked and rare.

Signal 4: Self-Merged

When the PR author is also the only approver or merger, there's no independent verification. This defeats the purpose of code review.

Signal 5: Failing Checks

PRs merged with failing CI checks (tests, lint, security scans) carry obvious risk. Even if the failure seems unrelated, it indicates process override.

Signal 6: Sensitive File Changes

Some files warrant extra scrutiny:

• Authentication: auth/, login, session, token files
• Authorization: permissions, roles, access control
• Payments: billing, payment, stripe, checkout
• Data: migrations, schemas, PII handling
• Configuration: .env, config, secrets, deployment files

Signal 7: After-Hours Merges

Code merged outside business hours has less oversight and slower response if something goes wrong. Track and question after-hours merges.

Automated Risk Detection

Building a Risk Scoring System

You can build a simple risk scoring system that evaluates PRs automatically:

Scoring factors: +10 for PR > 500 lines, +5 for PR > 200 lines, +15 merged without approval, +10 self-merged, +10 rubber-stamp review, +5 failing CI, +10 sensitive files, +5 after-hours, +5 first-time contributor.

Implementation Options

Option 1: GitHub Actions

Build a GitHub Action that runs on PR events and comments with risk assessment:

# .github/workflows/risk-check.yml
name: PR Risk Assessment
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  assess-risk:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Calculate risk score
        run: |
          # Check PR size
          ADDITIONS=${{ github.event.pull_request.additions }}
          DELETIONS=${{ github.event.pull_request.deletions }}
          TOTAL=$((ADDITIONS + DELETIONS))

          # Check for sensitive files
          SENSITIVE=$(git diff --name-only HEAD~1 | grep -E 'auth|payment|config' | wc -l)

          # ... more checks

      - name: Comment risk assessment
        uses: actions/github-script@v6
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: 'Risk assessment: ...'
            })

Option 2: Pre-merge Hooks

If your CI/CD system supports it, add risk checks as required status checks:

• Block merges with risk score above threshold
• Require additional approvers for high-risk PRs
• Alert on-call or tech lead for critical risk

Option 3: Post-merge Tracking

Even if you don't block, track risk scores over time to correlate with incidents:

• Log risk score for every merge
• After incidents, check the risk score of related changes
• Use data to refine your risk model

Setting Up Risk Alerts

What to Alert On

Not every risk signal warrants an alert. Focus on:

• Critical combinations: Large PR + no approval + sensitive files
• Process bypasses: Merged without approval, failing checks
• Patterns: Same person repeatedly merging risky changes

Alert Fatigue Prevention

Too many alerts means alerts get ignored. To keep alerts actionable:

• Set meaningful thresholds: Start conservative, adjust based on signal-to-noise
• Aggregate where possible: "3 risky PRs this week" vs. alerting on each one
• Route appropriately: Not every alert needs to go to everyone
• Review regularly: Are alerts being acted on? If not, adjust or remove

Responding to Risk Flags

Low Risk: Normal Process

For low-risk PRs (score 0-10), standard process applies:

• Normal review by any qualified reviewer
• Standard CI checks
• Merge when approved

Medium Risk: Extra Attention

For medium-risk PRs (score 11-20):

• Ensure thorough review (not rushed)
• Consider second reviewer for complex changes
• Verify test coverage for changed code
• Document the "why" in PR description

High Risk: Requires Justification

For high-risk PRs (score 21-30):

• Require explicit justification for risk factors
• Require senior engineer or domain expert review
• Consider breaking into smaller PRs if possible
• Plan deployment timing (not Friday, not after-hours)
• Have rollback plan ready

Critical Risk: Special Handling

For critical-risk PRs (score 31+):

• Require tech lead or staff engineer sign-off
• Mandatory documentation of risk mitigation
• Consider staged rollout or feature flag
• Alert on-call before merge
• Schedule follow-up to verify no issues

Correlating Risk with Incidents

Over time, track which risk factors actually correlate with incidents:

This data helps you refine your risk model and focus on the signals that actually matter for your codebase.

Building a Risk-Aware Culture

The goal isn't to block all risky changes—it's to make risk visible and deliberate:

• Make risk visible: Show risk scores in PR comments or dashboards
• Normalize discussion: "This PR has elevated risk because..."
• Don't punish: Risk awareness should help, not blame
• Learn from incidents: Use incident data to improve detection
• Iterate: Refine your risk model based on what actually matters

High-performing teams ship fast, but they also ship smart. Automated risk detection helps you move quickly on low-risk changes while applying appropriate scrutiny to high-risk ones—catching problems before they become incidents.