Your own test suite, run against the code the AI wrote

In AI CodeOps, the test stage runs your project's own test suite against the exact code an AI session just wrote, inside an isolated sandbox, and records the result as a test run. A failing test holds the change: it cannot become a pull request until the tests pass on its current version.

An AI agent saying "the tests should pass" is not evidence. So after the code pass, every session that changed a project's code goes through a verify step. It rebuilds the proposed file tree in a sandbox, from the same edits the diff was computed from, and runs your suite there. What runs is what is being reviewed. A test run against the base branch, or against an earlier version of the change, would not tell you anything about this one.

Which agents run

Three built-in agents plan testing. None of them executes code:

• Tester: the test plan and named test cases for the happy path, edge cases and failure modes. It runs in ten built-in workflows, including Plan, Build, Review & Test and Quality-Gated Change.
• E2E Tester: end-to-end scenarios for the user-visible flow, and where coverage is missing. It runs in Feature SDLC and Release Readiness.
• Performance Tester: hot paths and the expected effect of an optimization. It runs at both ends of Performance Tuning.

Their plans go to the code pass, so a session can write the tests along with the change. Then the verify step runs the suite, old tests and new ones.

How the verify step works

1. Pick the command. A command set on the project wins. Otherwise AI CodeOps reads the tree: a package.json test script (run with bun, pnpm, yarn or npm, by lockfile), else pytest -q when it finds pyproject.toml, setup.cfg, pytest.ini or tox.ini, else go test ./... for a go.mod. A project can also be set to skip tests.
2. Install, then go offline. For a project with a repository, the sandbox clones it and installs its dependencies while only your own code is in the tree. Then it is cut off the network for good. If it cannot be isolated, the AI's code is not run.
3. Write the change and run. The AI's files are written into the offline tree and the test command runs. If the change touched a dependency manifest, the install runs again, offline.
4. Record the result. The output is parsed for pytest, Jest, Vitest and JUnit XML, with line coverage when the output reports it. Other runners, Go's included, are recorded by their exit code. The run stores totals, passed, failed, skipped, duration, coverage and the exit code.

Each command is limited to two minutes, and the step stops starting new commands after about six. The step is checked against your policies first: a shell.command rule can allow or deny the install and test commands.

Results appear on the session page and on Build & Test, Quality, which lists recent test runs.

Honest skips

Many things can stop a suite from running that say nothing about the change: no test command, dependencies that will not install, a missing runner, no sandbox available, a timeout, or a policy that denies the command. In each case the step records a skip with its reason and writes no test run. A skip never appears as a failure, and never as a pass. Fix the cause, often by setting the test command on the project, and run the session again. See troubleshooting.

Quality gates involved

Test results are one of the checks in the quality gate:

• Failing tests block. Any recorded test run with a failed test or a non-zero exit code denies the change.
• Only the current version counts. If someone edits the change after its tests ran, the old result no longer applies and the gate reports the new version as untested.
• Coverage floors are optional. A CI call can set minCoveragePct. The most recent run that reported coverage must meet it.
• No tests is not a failure. A skipped verify step does not block on its own.

When your CI asks the gate about a pull request, failing runs come back in testFailures. An excerpt of a denied response from /api/release/check:

{
  "allowed": false,
  "evidenceComplete": true,
  "summary": "Tests failing (vitest): 2 of 148 failed",
  "reasons": ["Tests failing (vitest): 2 of 148 failed", "Coverage 71% is below the required 80%"],
  "testFailures": [
    {
      "id": "tst_01JA8ZB4N6Q1S3V5X7Z9C2E4G6",
      "workflowRunID": "wfr_01JA8YQ4M2C7H5K9P3S6V8X1ZD",
      "framework": "vitest",
      "failed": 2,
      "total": 148,
      "exitCode": 1,
      "coveragePct": 71
    }
  ],
  "coverageShortfall": { "coveragePct": 71, "required": 80 }
}

The API answers HTTP 200 whether the change is allowed or not, so read the allowed field. Tests are only part of it: the change also needs a completed findings review with no blocking findings, and a human approval.

Limits

The sandbox runs your unit and integration tests, whatever your test command covers. AI CodeOps does not run browser tests, load tests or deployments of the change as part of this step, and it does not make up for a project with few tests. Tests that need outside services or network access will not reach them, because the sandbox is offline.