Claude Code Review Skill: Setup and Proof It Works

Installing a Claude Code review skill takes about ten minutes. Knowing whether it made your code review better takes a baseline, four metrics, and thirty days of patience. Almost every guide covers the first part and stops. This one covers both, because a review skill that nobody can prove is working is the first thing cut when someone audits the AI budget.

This guide walks through what a review skill actually is, how to write one engineers will not route around, where a skill beats a hosted review bot, and the part the rest of the internet skips: the four numbers that tell you whether any of it helped.

What Is a Claude Code Review Skill?

A skill is a directory in your repository holding a markdown file with frontmatter. The frontmatter carries a name and a description of when the skill applies; the body carries the instructions. When you ask Claude Code to review a change and the task matches the description, the skill loads and its instructions shape the review.

The distinction that matters is between general review ability and house knowledge. Claude arrives already able to spot an off-by-one error, an unhandled promise rejection, or a function doing four things. What it cannot know on arrival is that every database query in your codebase must filter by tenant, that your migrations have to be expand-contract because production carries live customer data, or that three specific files break fifty others when touched. That knowledge normally lives in the heads of two senior engineers and leaves when they take holiday.

"A review skill is your senior engineers' review checklist, written down once and applied every time - including the weeks they are on holiday."

This is why the file belongs in the repository rather than in someone's home directory. A committed skill gets reviewed in a pull request, versioned with the code it describes, and picked up by everyone who clones the repo. A local skill helps one person and drifts immediately.

How Do You Write a Review Skill Engineers Will Not Route Around?

The failure mode of a first attempt is a skill so long and so generic that it produces fifty low-value comments on every change, and engineers learn to ignore all of them. Specificity is the whole game. Write down the things a competent outside reviewer could not guess, and leave the rest to Claude's general ability.

Here is the shape of a skill that stays useful:

# .claude/skills/code-review/SKILL.md
---
name: code-review
description: Use when reviewing a diff, a pull request, or staged changes in this repo.
---

## Hard rules (flag every violation)
- Every DB query filters by `organization_id`. A query without it is a
  tenant-isolation bug, not a style note.
- Migrations must be expand-contract. No column drops in the same release
  as the code that stops writing it.
- `if value:` where value can be 0 - use `if value is not None:`.

## Ask, do not assume
- New endpoint with no test: ask which behavior is covered elsewhere
  before flagging it.

## Out of scope
- Formatting. The linter owns it. Do not comment on it.

Three properties make that skill survive contact with a real team. It states rules a general reviewer cannot infer. It says what is explicitly out of scope, which is what stops the noise that gets skills abandoned. And it distinguishes hard violations from questions, so a reviewer reading the output can tell a blocker from a prompt to think.

Keep it short enough that someone will actually update it. A skill that grows to four hundred lines becomes documentation nobody reads, which is where most house style guides already died. The same discipline applies as in any code reviewer best practices checklist: a short list that gets used beats a thorough one that does not.

Where Does a Skill Beat a Hosted Review Bot?

A skill and a hosted review bot solve different problems, and teams that understand the difference tend to run both. The skill runs on your machine before the pull request exists. The bot runs on the pull request after you push. That timing difference is the entire argument.

The row that decides most rollouts is the second one. A skill is opt-in by nature, so its coverage depends on habit, and habit is uneven. That is not a reason to skip the skill - it is the reason the measurement section below matters, because unmeasured opt-in adoption tends to be far lower than the person who wrote the skill assumes. If you are weighing the hosted options against each other, our comparison of AI code review bots covers detection rates and pricing across BugBot, Codex, and Greptile.

We also build one, so treat this paragraph as disclosure rather than a neutral survey. CodePulse Reviews runs the enforced layer from Slack instead of a dashboard: you paste a pull request link in any channel it is in, Claude reviews the change across the same five axes a good skill encodes (correctness, readability, architecture, security, performance), and it comes back with severity-tagged comments split into Critical, Important, and Suggestion. First review lands in about two minutes. "Fix the issues" opens a commit on the branch with the fixes applied, so you review a commit rather than retyping the feedback yourself.

Pricing starts at $6 per developer per month on BYOK, where you supply your own Anthropic or OpenAI key and pay the model bill directly. Team is $29 and puts a frontier model on every review with unlimited reviews and auto-fix from Slack; Business is $49 and adds Deep Review plus a Semgrep SAST pass before the review. Enterprise adds SSO/SAML, audit logs, and self-hosting. The reason it matters for this guide specifically is that it feeds CodePulse Analytics, so the enforced review layer and the four metrics that judge it read from the same history - you are not standing up a second tool to find out whether the first one worked.

How Do You Prove the Skill Is Actually Working?

Here is the uncomfortable part. Nearly every write-up about AI review skills ends at installation, which means nearly every team running one has no idea whether it helped. You cannot fix that after the fact, because the comparison needs a baseline from before rollout. The good news is that the baseline already exists in your Git history - you just have to read it.

Four metrics, measured 30 days before and 30 days after:

Time to first review is the one that should improve, and it is the honest case for a review skill. When mechanical findings are already fixed before the pull request opens, the first human to look at it has less to wade through and responds sooner. This is the same wait state that dominates PR cycle time for most teams.

Review coverage and merged without approval are guardrails. If either degrades, the skill has become a justification for skipping review rather than a way to improve it. Watch for the sentence "Claude already reviewed it" appearing in your team's vocabulary, because the metrics will show it before anyone says it out loud.

Comment substance is the metric almost nobody tracks and the one that catches the real failure. Count how many review comments are substantive rather than approvals with no body attached. If reviews get faster while comments get thinner and rarer, human attention has left the building.

"Faster approvals with fewer comments is not a productivity win. It is the sound of your review process being quietly switched off."

What Actually Goes Wrong When Teams Roll This Out?

Three failure modes account for most disappointing rollouts, and none of them are about model quality.

The rubber stamp. An approval submitted less than a minute after the pull request opened is not a review, whatever the audit trail says. Once a team believes AI has already checked the change, sub-minute approvals climb. This is measurable and worth an alert, because it is the pattern that turns an AI review win into a production incident nobody saw coming.

Silent non-adoption. Because a local skill is opt-in, the person who wrote it uses it constantly and assumes the team does too. In practice adoption clusters around whoever was in the room. Speed metrics that barely move after rollout usually mean the skill is not being run, not that it does not work - and those are opposite problems with opposite fixes.

Noise-driven abandonment. A skill that comments on formatting, or repeats what the linter already said, trains engineers to skim past its output. By the time it flags something real, nobody is reading. The "out of scope" section in the example above exists precisely to prevent this, and it is the section people leave out.

"An AI reviewer that cries wolf about formatting has spent the credibility it needed for the one comment that mattered."

None of this is an argument against AI review. It is an argument for treating rollout as a change to your review process rather than a tool installation, which is the framing we use across measuring AI coding tool impact.

What Does a 30-Day Rollout Look Like?

A rollout you can evaluate needs a baseline before anything changes and a single variable moving at a time. Four weeks is enough.

Week 0   Baseline. Record the four metrics for the previous 30 days.
         Change nothing. Write the numbers down where you will find them.

Week 1   Write the skill with two senior reviewers. Ship it as a PR so
         the rules themselves get reviewed. Keep it under ~50 lines.

Week 2   One team only. Ask them to run it on staged changes before
         opening a PR. Collect what it got wrong - that is the edit list.

Week 3   Revise from real findings, not from imagination. Cut every rule
         that produced a comment nobody acted on.

Week 4   Compare against the Week 0 baseline. Speed better, coverage and
         substance holding = keep it. Speed better, substance worse =
         you have a disengagement problem, not a review improvement.

The step people skip is Week 0, and skipping it is unrecoverable in the sense that matters: you can still read the history later, but you will read it knowing the answer you want. Write the numbers down before you have an opinion about them.

One more thing worth saying plainly. Use these metrics on the team, never on individuals. A review-comment count next to a person's name turns into a target, and targeted review metrics produce more comments rather than better ones. Schlund and Zitek's 2024 paper in Communications Psychology tested this across four experiments with about 1,200 participants: people under algorithmic rather than human monitoring perceived less autonomy, generated fewer ideas, and reported greater intention to resist. The same paper found the penalty largely disappeared when the monitoring was framed as developmental rather than evaluative. That is the whole distinction. The four metrics exist to tell you whether your process improved, not who to talk to about it.

Where Should You Go Next?

• CodePulse Reviews - our hosted layer: paste a PR link in Slack, get a five-axis review in about two minutes, fix it with one click. From $6/dev/month on BYOK.
• AI Code Review Bots Compared - the third-party alternatives, if you want to shop around first.
• Code Review Culture and Sentiment - how to read comment substance without turning it into a scorecard.
• AI Code Review Tools Tested - which tools caught real bugs and which added noise.
• How to Reduce PR Cycle Time - the wait state a review skill is trying to shorten.