Skip to content
steelabs

Guide

Build an E2E testing bot without manufacturing green builds

Generate and maintain browser tests from approved behaviours, then independently check whether those tests detect a broken journey rather than mirror the implementation.

Published 7 min read6 chapters

Contents

  1. 01Keep the expected result outside the generated test
  2. 02Generate a proposal, run it, then challenge it
  3. 03An assertion must observe the promised effect
  4. 04Do not let maintenance erase a regression
  5. 05Evaluate fault detection and suite cost
  6. 06Budget generation and ongoing execution together

Build an E2E testing bot without manufacturing green builds

An E2E bot can draft browser tests and propose repairs after an interface changes. The expensive failure is a test that always passes. If the bot reads the new implementation and quietly changes the expected behaviour to match it, the suite can lose protection while looking healthier.

Start with one stable, important journey and an expectation approved independently of the code. The proposed architecture below uses Playwright examples, but the ownership of expected results matters more than the browser framework.

Chapter 01

Keep the expected result outside the generated test

Write a behaviour contract with preconditions, actions and observable effects. For a discount journey, specify the eligible basket, the applied reduction, the resulting total and the ineligible case. Use synthetic accounts and explicit test data. Record who approved the business rule.

The bot can inspect the interface to find accessible controls; it cannot decide that a changed displayed total is the new correct total. Protect acceptance fixtures or require a separate owner’s approval for changes. Assertions should cover a business result, not only the presence of a heading.

  • Prefer API or unit checks where the behaviour does not require a rendered browser.
  • Reserve E2E coverage for meaningful journeys, integration boundaries and user-visible failures.
  • Use isolated fixtures so execution order does not become an invisible dependency.

Chapter 02

Generate a proposal, run it, then challenge it

  1. 01Supply the approved behaviour, supported browsers and a disposable test environment.
  2. 02Let the bot draft a test with resilient locators and explain its assertions.
  3. 03Run it against the known-good build and retain real runner output.
  4. 04Run controlled broken variants to verify the critical assertions fail.
  5. 05Have a tester review the contract, assertions and fixtures before merging the test.

A successful run against the current build checks compatibility, not sensitivity. Deliberately disable discount application or return the wrong total in a controlled test environment. If the generated test remains green, reject it. Keep this independent check separate from the bot’s own claims about coverage.

Chapter 03

An assertion must observe the promised effect

Illustrative Playwright test: requires your own route, fixture and accessible labels; not executed against a real store.
import { test, expect } from '@playwright/test';

test('approved discount changes the basket total', async ({ page }) => {
  // Fixture: basket has a 100.00 subtotal and an eligible SAVE10 code.
  await page.goto('/basket?fixture=eligible-basket');
  await page.getByLabel('Discount code').fill('SAVE10');
  await page.getByRole('button', { name: 'Apply discount' }).click();
  await expect(page.getByTestId('basket-total')).toHaveText('90.00');
});

The example is deliberately not a drop-in production test. A real suite must create the fixture securely, verify currency and rounding rules, and include an ineligible basket. Avoid public query parameters that seed privileged data in a production application. Test-only fixture access belongs in the isolated environment.

Use framework assertions that wait for the expected state. Arbitrary sleeps hide timing problems and slow the suite. A bot-generated locator should be inspected for stability and for whether it identifies the intended element.

Chapter 04

Do not let maintenance erase a regression

Classify a failure before repairing the test: product defect, intentional requirement change, locator change, test data or environment. A bot may propose a locator update when the accessible name changed, but deleting an assertion or increasing a timeout needs a reason a reviewer can verify.

If the expected total changes because the business rule changed, update the independent contract first. Retain the previous rule in history and review the new cases. Never offer a “heal all red tests” mode that accepts whatever the current interface displays.

Capture traces only with approved data and retention. Screenshots and network traces can contain personal information or credentials. Redact sensitive headers and keep artifacts out of public review comments.

Chapter 05

Evaluate fault detection and suite cost

Use known-good journeys plus deliberately broken variants: skipped persistence, incorrect permissions, duplicate submissions and stale displayed data. Record which faults each test detects. Measure flakiness under repeated clean runs, execution time and triage effort alongside coverage.

A green suite after five reruns is not a dependable gate. Separate deterministic failures from environmental noise and assign an owner to each flaky test. Quarantine with a visible risk record rather than allowing the bot to silently remove coverage.

Keep generation read-only until results earn trust. The runner uses test credentials, not deployment credentials, and cannot reach production resources. Define a safe stop when the environment is unavailable; “not executed” must remain distinct from “passed”.

Chapter 06

Budget generation and ongoing execution together

Model calls are only one part of the bill. Browser workers, test accounts, artifact storage, repeated runs and maintenance often dominate. Generate on changed requirements rather than on every commit, reuse approved fixtures and set a limit on repair attempts. Track the cost per protected journey, including human review.

For a maintained suite, see test automation and AI SDLC integration. Describe your critical journey in an enquiry.

Testing guidance checked 11 October 2026: Playwright: testing best practices. Platform isolation references: GitHub: secure use of Actions and GitLab: merge request pipelines. The workflow is a proposed control design, with no claimed client defect-reduction metric.

Going deeper

The individual pieces

Practical companion resources

Contact

Turn the guide into a scoped next step

Bring the task, approved examples and existing tools. We can review feasibility, define the evaluation and estimate the first useful deliverable.

Start the conversationsales@steelabs.tech