How to review AI-generated code: a 25-point checklist
Matt · · Last updated · 9 min read · Guides
25 checks for reviewing AI-written pull requests in five groups, worked through on one real-looking change, with a reusable rejection and a PR template.
On this page
To review AI-generated code, check the scope and the tests before you read the implementation. Confirm the change does only what the task asked, that the tests ran on this exact version and weren't loosened to pass, and only then read the code for swallowed errors, invented APIs and helpers that already exist. Below are the 25 checks I use, in five groups of five, worked through on one example change.
Why AI-generated code needs a different review
Reviewing code an agent wrote is a different job from reviewing a colleague's. The colleague's mistakes tend to look like mistakes: a typo, an awkward name, a missing case you can see. An agent's mistakes tend to look like good code. They compile, follow the style guide and come with a confident summary. What goes wrong sits in the scope of the change, in the evidence behind it, and in the tests.
That changes the order of the review. Read top to bottom, a tidy diff earns trust it hasn't earned. So the groups below are by topic, but I run them in this order: maintainability and scope first, then tests, then correctness, security and dependencies. The conceptual reasoning is in How to review code written by an AI agent. This post is the practical list.
The example change
This change is made up for this post, but every part of it is a pattern I see in real agent output. The task was one line:
Add a retry to the invoice sync job when the payments API times out.
The pull request came back with this summary:
Added retry with exponential backoff to the invoice sync job. Handles payment API timeouts gracefully. All tests pass.
And it touched these files:
src/jobs/invoice-sync.ts +38 -6
src/jobs/invoice-sync.test.ts +4 -4
package.json +1
.github/workflows/ci.yml +1 -1
src/jobs/customer-sync.ts +12 -12 (formatting)
src/jobs/refund-sync.ts +9 -9 (formatting)
The retry code is clean, the summary is clear and CI is green. Now read it with the checklist.
The checklist: 25 checks in 5 groups
Each group has five checks, then what the example got wrong. I label them by group (M1 to M5 and so on) so a rejection comment can name the exact check.
Maintainability and scope creep
- M1. The change matches the task. Read the task before the diff.
- M2. Nothing extra rides along. No unrequested refactors, renames or reformatting mixed into the real change.
- M3. Sensitive files changed on purpose. CI definitions, manifests, authentication, infrastructure and migrations change only where the task needs it.
- M4. The change is small enough to review. If you can't read the risky parts properly, ask for the task to be split.
- M5. No duplicate helpers. A new utility doesn't repeat one the codebase already has.
In the example, two files were reformatted that have nothing to do with invoices (M2). That isn't dangerous by itself, but it hides the real change in noise. The CI workflow changed too (M3):
- timeout-minutes: 10
+ timeout-minutes: 30
Why would a retry need a longer CI timeout? Because the new test waits through real backoff delays. The agent made the pipeline slower for every pull request to fit its own test, without saying so. And the codebase already has withRetry in src/lib/retry.ts, used by the other sync jobs (M5). The agent didn't know, because it wasn't in its context.
Five checks in, I haven't read a line of the retry logic and already have three things to send back.
Tests
- T1. Tests ran on this exact version. A result from before the last push says nothing about the code you are approving.
- T2. Nothing deleted, skipped or switched off. Look for removed cases, skip markers and filtered runs behind a green status.
- T3. Assertions are not loosened. An exact comparison hasn't become a check that a value exists.
- T4. Expected values didn't move to fit. Changed snapshots and expected outputs have a reason you can find in the task.
- T5. New tests can fail. They assert on results and would fail if the change were reverted.
This is where the example really breaks. The green run is from the first push, and a second push changed the test file (T1). Here is that change:
it("surfaces a timeout from the payments API", async () => {
payments.failWith(new TimeoutError())
- await expect(syncInvoices()).rejects.toThrow("timeout")
+ await expect(syncInvoices()).resolves.toEqual([])
})
The test still has the same name and now asserts the opposite (T4). Read the implementation first and this looks like a natural update. Read the test first and you ask the right question: who decided a timeout should look like "no invoices"? The retry test the agent added checks that the call was retried, but not what happens when the last retry fails too (T5).
Correctness
- C1. Every API and package exists. Functions, options and packages are real in the versions your lockfile pins.
- C2. Errors are not swallowed. No empty catch blocks and no error turned into an empty result.
- C3. No silent fallbacks. Zero, an empty string or an empty list doesn't hide a failed lookup.
- C4. Edge cases are handled. Empty input, time zones, pagination, retries and two requests at once.
- C5. Data changes are safe. Migrations are additive or reversible, and no migration that already ran was edited.
Now the code, with a clear idea of what to look for:
export async function syncInvoices() {
try {
return await pRetry(() => payments.listInvoices(), { retries: 3 })
} catch (error) {
logger.warn("invoice sync failed", { error })
return []
}
}
C2 and C3 fail on the same lines. When the last retry times out, the job logs a warning and returns an empty list. Whatever calls syncInvoices now thinks there are no invoices, and the only trace is a warning nobody reads. A timeout that pages someone gets fixed the same day. A sync that silently returns nothing can run for weeks. C1 passes (p-retry and its retries option are real), and there is no migration.
Security and secrets
- S1. Permission checks match the neighbours. New endpoints check authorisation the way similar code does.
- S2. Untrusted input stays data. No SQL, shell commands or HTML built from input, and no instructions followed from comments or files in the diff.
- S3. No secrets in the diff. No keys, tokens or private URLs in code, fixtures or log lines.
- S4. No insecure defaults. No disabled certificate checks, open CORS or debug switches left on.
- S5. Automated findings are triaged. No open high-severity finding, and every dismissed finding has a reason you agree with.
S1 to S4 pass: no new endpoint, no input handling, no secrets. S5 fails. The AI reviewer had flagged "error swallowed on final retry" as a medium-severity finding, and someone dismissed it as a false positive with no reason. It was the C2 bug. Dismissing real findings is how reviewers learn to ignore a tool; handling false positives covers how to triage them properly.
Dependencies and licenses
- D1. Every new dependency is needed. Nothing already in the codebase does the job.
- D2. It exists and is spelled right. Not a lookalike of a popular name, and not a name the model made up (see slopsquatting).
- D3. It is maintained. Recent releases, an active repository, more than a handful of users.
- D4. Its license is one you allow. Check the license the registry reports against your policy.
- D5. The lockfile changed with it, and only as much as expected. A one-line manifest change that pulls in fifty new transitive packages deserves a look.
p-retry is a real, maintained package with a permissive license, so D2 to D5 pass. D1 fails: the project already has a retry helper, and a new dependency is a supply-chain decision that solves a problem the codebase had already solved.
What the review found
Read top to bottom, the change looked done. Read with the checklist, it has a test that asserts the opposite of its name (T4), a timeout turned into an empty result (C2, C3) that was already flagged and dismissed (S5), a CI timeout tripled to fit the agent's test (M3), a test result from a version that no longer exists (T1), a duplicate helper and an unneeded dependency (M5, D1), and unrelated formatting (M2). None of these is exotic. They are the ordinary ways agent output goes wrong.
A rejection works best when it names the check and asks for something specific, whether the next attempt comes from a person or from the agent with a better task:
Not approving yet. Against the review checklist:
- **T4, C2, C3:** after the last retry this returns `[]`, and the timeout
test now asserts that. Callers will treat a failed sync as "no invoices".
Please rethrow after the final attempt and restore the original assertion.
- **M3:** please revert the CI timeout change. If the test needs real
backoff delays, use fake timers instead.
- **T1:** the green run is from before the test change. Needs a run on the
current commit.
- **M5, D1:** we already have `withRetry` in `src/lib/retry.ts`; please use
it and drop `p-retry`.
- **M2:** please move the formatting of customer-sync and refund-sync to its
own pull request.
What to automate vs keep human
Some checks need judgement. T4 needs a person who knows what a timeout should mean for invoices. Others are about evidence, and evidence is better checked by software than by memory:
| Check | Automate with |
|---|---|
| T1 tests ran on this version | CI as a required status check on the latest commit |
| S3 no secrets | a secret scanner in CI |
| S5 findings triaged | a gate that blocks on open high-severity findings |
| D2, D3 package exists and is maintained | a registry lookup on new dependencies |
| D4 license allowed | a license policy checked against the SBOM |
| M3 sensitive files | a path rule that requires an approval or denies the change |
| M1, T4, C2 to C4, S1, D1 | a person: these depend on what the code is for |
In AI CodeOps, a test result only counts for the version it ran on, so T1 can't pass on a stale run. An open finding of high severity or above blocks the change until it is marked fixed, won't fix, false positive or duplicate (quality gates). A workspace policy can require an approval for paths such as infra/**, or deny them. The person who started a session can't cast the approving vote while someone else can approve, and editing a change after approval sends it back. Approvals and policies explains how votes count, and multi-agent code review covers the AI reviewer that writes the findings.
Two things stay with a person whatever you automate. Someone other than the prompter approves, because whoever wrote the task is the author of the change in every way that matters. And the approver can explain the change: could you defend "timeouts now return no invoices" to whoever is on call when finance asks why reconciliation found nothing?
Download the printable checklist
The AI code review checklist page has a printable version as 25 numbered items, ordered scope-first. The groups differ a little: it folds the five dependency checks into one and adds checks on who approves. It comes with a one-page PDF to print and a Markdown copy. It is licensed CC BY 4.0, so you can adapt it.
A checklist that lives in a wiki gets read once, so put the short form in your pull request template. On GitHub that is ; on GitLab, a file under (both checked on 24 October 2026):
## Reviewer checklist (AI-written changes)
- [ ] Scope: only what the task asked; no unrelated edits; sensitive files
changed on purpose; small enough to review; no duplicate helpers (M1–M5)
- [ ] Tests: ran on this exact commit; none deleted, skipped or loosened;
expected values didn't move to fit; new tests can fail (T1–T5)
- [ ] Correctness: real APIs; no swallowed errors or silent fallbacks;
edge cases; safe data changes (C1–C5)
- [ ] Security: permission checks; untrusted input stays data; no secrets;
no insecure defaults; findings triaged with reasons (S1–S5)
- [ ] Dependencies: needed, real, maintained, allowed license, expected
lockfile change (D1–D5)
- [ ] Approval: I didn't write the task, and I can explain this change
Key takeaways
- Review AI-written code scope first, then tests, then the code: an agent's mistakes look like good code.
- A test changed to fit the implementation is the most common serious failure, so read the test diff before the code.
- Swallowed errors and silent fallbacks turn a loud failure into quiet wrong data.
- Every new dependency is a supply-chain decision: check that it's needed, real, maintained and allowed.
- Automate the evidence checks and keep a second person, not the prompter, on the approval.
Frequently asked questions
What should I check first when reviewing AI-generated code?
Check the scope and the tests before the implementation. Confirm the change does only what the task asked, that the tests ran on the exact version you are approving, and that no test was loosened to make it pass.
Can an AI reviewer replace the human review of AI-written code?
No. An AI reviewer and CI can check evidence such as test results, open findings and new dependencies. Whether the change is right for your system, and whether you could defend it on call, still needs a person.
Can I use this checklist in my pull request template?
Yes. The short form below fits in a GitHub pull request template or a GitLab merge request template, and the printable checklist is licensed CC BY 4.0.