03 /Test Automation
CI/CD Test Integration
Wiring an automated suite into your pipeline so it executes on every change, blocks what it ought to block, and reports somewhere people already look. A suite living on a laptop is a hobby. The value arrives when a red result stops a merge and somebody is accountable for the colour.
What you get
A job per test tier
Fast checks on every push, the browser tier on a pull request, and the long tail on a schedule. Running everything on every commit is how teams end up waiting a quarter of an hour for feedback on a typo.
Containerised, reproducible runs
Browsers, dependencies and system libraries pinned into an image so a run behaves identically on a developer machine and on the runner. A good deal of what teams call flakiness is an environment difference nobody has examined.
Gating rules written into the repository
An agreed answer to what a failure actually prevents, encoded in branch protection rather than in a convention people remember differently. A gate that can be waived quietly is not a gate.
Artefacts and reports attached to each run
Traces, screenshots and a published report living on the run itself, so whoever triages does not have to reproduce anything locally first. Findings requiring a checkout before they can be read do not get read.
Secrets and environment handling
Credentials kept in the pipeline’s own store, test accounts that can be reset between runs, and a documented answer to which environment each tier points at. Suites quietly running against production are more common than anybody admits.
How it runs
- 01
Map the pipeline you have
We look at how a change currently reaches production, where the waiting already happens, and what people do when a build turns red. Adding a stage to a pipeline nobody trusts simply spreads the distrust.
You provide: Pipeline access, runner details, and the name of whoever owns a broken build today
- 02
Set tiers and budgets
Each group of tests gets a trigger and a time budget agreed with the people who will be waiting for it. A number fixed in advance is what stops the suite from slowly becoming the release process.
We produce: A tiering plan with triggers · An agreed run-time budget per tier
- 03
Wire it up and observe
Jobs are built, containerised and connected to branch protection, then watched across repeated runs before anything is permitted to block a merge. Gating on a suite whose stability is unproven teaches a team to bypass it.
We produce: Pipeline configuration in your repository · Reporting and notifications · A quarantine mechanism for unreliable tests
- 04
Hand over the rules
The triage path, the quarantine rule and the time budget are written down and owned by a named role on your side. A pipeline convention with no owner reverts within a release or two.
We produce: Written triage and quarantine policy
Where this stops
- Not DevOps consulting. We add and maintain the testing stages; your deployment strategy, infrastructure and release tooling stay exactly where they are.
- Not a migration between CI providers. Moving from one platform to another is a project in its own right, and it is rarely worth undertaking on account of tests.
- Not monitoring. A pipeline tells you a change broke something before it ships; knowing that production is unwell right now needs observability, which is a separate budget line.
- Not a licence to gate on a suite that has not earned it. We will decline to switch on branch protection over tests that have not proven themselves across repeated runs, because one false block is enough to teach a team to ignore the signal.
Related reading
- Test automation
CI test gating: deciding what a red build is allowed to stop
Wiring a suite into a pipeline takes a day. Deciding what it may block is the real project, and it is a social agreement with a configuration file attached.
- Test automation
Diagnosing flaky tests: a procedure, not a retry
A retry policy converts an unreliable suite into a slow one that sounds confident. Four families of flake, the tell that identifies each, and the order to work through them.
- Test automation
Test data management: five strategies and what each one costs
How a test obtains the state it needs decides whether a suite survives parallel execution. Five approaches, the failure mode of each, and the problems nobody puts in the estimate.
FAQ
CI/CD Test Integration questions we hear often
Should a failing test block the merge?
Eventually yes, and not on the first day. A gate only works if the team believes a red result means something, so the suite runs in reporting mode until its stability across repeated runs has earned the switch. Turning gating on immediately is the fastest route to somebody adding a bypass, and bypasses are never removed afterwards.
How long should the pipeline suite take?
Short enough that nobody starts something else while waiting. In practice the tier guarding a pull request has to fit inside the window a developer will actually sit through, and everything heavier moves to a nightly or on-demand run. When the budget is blown, the answer is usually to push checks down to the service layer rather than to rent bigger runners.
Can you work with our CI provider?
If it runs containers and can hold secrets, almost certainly. We work most often in GitHub Actions and GitLab CI, and the concepts travel: triggers, caching, matrices, artefacts and protected branches all exist elsewhere under different names. What we will not do is recommend switching provider as part of a testing engagement, because that is a decision with costs attached that have nothing to do with us.
Who fixes the pipeline when it breaks at an inconvenient hour?
Your team, and the setup is deliberately arranged so that is a fair ask. Configuration lives in your repository, the triage path is written down, and the quarantine rule lets one person pull a single unreliable test out of the gate without convening a meeting. Ongoing maintenance is available, but the default is that you can operate this without us.
Contact
Have a project in mind?
Tell us what you are building — or what keeps breaking. You will get a considered reply from an engineer, not an autoresponder.