Visual regression testing

Visual regression testing captures what a component or page actually renders, compares it pixel by pixel against a stored baseline, and fails when the difference exceeds a threshold. It exists because functional assertions are blind to a large category of real defects. A test that finds a button by its accessible name and clicks it passes perfectly well when the button is white on white, positioned off screen, or overlapped by a banner — the behaviour is intact and the page is unusable.

The technique lives or dies on false positives, and the failure mode is social rather than technical. Rendering varies with fonts, with graphics drivers, with the sub-pixel decisions a browser makes on a different machine, so a suite configured naively fails constantly for reasons nobody cares about. Within a few weeks the team is approving diffs without reading them, at which point a genuine regression is approved with the rest and the suite has become a ritual that costs money and catches nothing.

Making the comparison trustworthy

• Render in one controlled environment, normally a container pinned to specific browser and font versions, so baselines captured on a laptop are never compared against a run in CI.
• Freeze everything that legitimately changes: dates, animation states, avatars, anything randomised. Most early instability comes from content rather than from rendering.
• Target components rather than full pages. A component baseline fails for one reason; a page baseline fails whenever anything anywhere on it moves.
• Treat baseline approval as a review step with an owner. Bulk-accepting a batch of diffs is the moment the suite stops providing information.

This technique proves that the rendering has not changed, which is a different claim from proving it is correct: an approved baseline of a broken layout passes indefinitely.