Test AI-generated code before it reaches a release

Read the ticket, approved design and API contract before reading the generated tests. Identify inputs, roles, side effects and failures. Ask the author to list assumptions and affected callers. An unresolved rule should return to the owner; it must not acquire authority because it appears in generated code.

Consider an invoice download fix. The important expectation is that only an authorised member of the owning organisation may download the invoice. A test that checks “download returns 200” misses the risk. Explicitly specify a member of another organisation, a revoked member and an unauthenticated request.

• Check that the diff solves the requested problem rather than a nearby one.
• Separate required behaviour from opportunistic refactoring.
• Identify contracts that other services or older clients depend on.

Review data flow, authorisation, error handling, concurrency, resource limits and compatibility. Confirm that unfamiliar APIs or package names actually exist in the installed version. Generated code can invent a method or use a legitimate method with the wrong semantics.

Look for overly broad catches, permissive defaults, unsafe string interpolation and hidden changes to validation. Inspect dependencies and lockfiles independently. A new library can add licensing, supply-chain or runtime obligations even when the visible patch is small.

For async work, check cancellation and duplicate execution. A retry that performs a payment or sends an email twice is a correctness problem that a happy-path test often misses. Use idempotency requirements from the business contract, not a model’s generic retry advice.

1. 01Establish a baseline: run relevant existing checks on the original commit and distinguish pre-existing failures.
2. 02Run formatting, lint and type checks where the project uses them; these catch structure, not business correctness.
3. 03Add independent unit or API cases for the changed decision boundaries and side effects.
4. 04Exercise the important browser journey when the user interface or integrated flow changed.
5. 05Review access, data handling and dependencies when the patch touches a security boundary.
6. 06Record the tested commit, environment, results and remaining release risks.

Keep test setup reproducible. When a dependency is unavailable, document the missing check rather than mocking away the behaviour that needs verification. A mock can verify how code handles a response; it cannot prove the real integration returns that response.

Generated tests often repeat the implementation’s assumptions. An expected output copied from the function being tested can validate the same mistake twice. Derive expectations from the requirement, an independently worked example or an approved external contract.

Deliberately break the changed behaviour in a controlled copy: remove an authorisation condition, invert a validation branch or skip persistence. The targeted test should fail. If it passes, inspect the assertion and fixture before counting it as protection. This is a focused fault check, not an instruction to alter production code.

The E2E bot guide applies this idea to browser automation. A separate QA reviewer can challenge risk and coverage when the team needs independent assurance.

Confirm what source, logs and fixtures the assistant received and whether that use was approved. Do not upload production records simply because reproducing an issue is convenient. Sanitised fixtures should preserve the failure pattern without exposing customers.

If the assistant ran commands, review the execution boundary. Test credentials should not permit production writes. A generated migration or destructive command requires its own approval and recovery plan; a successful unit test does not authorise execution.

Keep secrets out of generated comments, examples and stored transcripts. If a credential is exposed, removing it from the final diff does not revoke it. Follow the organisation’s incident and rotation process rather than assuming the repository cleanup resolved the exposure.

A release note should state verified behaviour, known limitations and the owner of remaining decisions. The engineer is accountable for the diff even when an assistant wrote it. Existing branch protection and release approvals remain the gate.

Measure net delivery effort over comparable changes: drafting, review, test execution, correction and post-release defects. Report the observation period and task mix. Faster generation alone is not proof of faster delivery, and a small pilot is not a universal productivity claim.

The split between included build checks and additional assurance is explained in QA included in development. Share a risky change or upcoming release in an enquiry to scope the appropriate review.

Sources checked 11 October 2026: Playwright: testing best practices for robust test design and Anthropic: Claude Code security for coding-tool access controls. This release workflow is engineering guidance, not a certification of generated code.