Smoke testing

A smoke test is a small set of checks across the most important paths of an application, run immediately after a build is deployed in order to answer one question: is this worth testing further. It is deliberately shallow and deliberately wide. The application starts, the front page renders, somebody can sign in, the primary workflow reaches its second step, the service can reach its database. The name comes from hardware, where powering up a new board and watching for smoke told you whether to carry on.

Its value is almost entirely about protecting other people’s time. Running a full regression pass against a build whose authentication is broken produces pages of failures that share one cause and take a day to triage back to it. A gate at the front prevents that. It also gives a deployment pipeline a definition of success beyond the fact that a process started and did not immediately exit. This is the easiest testing investment to justify, because the argument is arithmetic rather than philosophy: a few minutes at the front of a pipeline set against a day of triage behind it.

Keeping one useful

• Keep it quick. Once it takes longer than a few minutes it stops being a gate and becomes a stage people find reasons to skip.
• Keep it shallow. It exists to detect catastrophe rather than to find defects, and depth belongs in the suites that run behind it.
• Run it wherever a build lands, production included, where it doubles as confirmation that the release you thought you shipped is the one now serving traffic.
• Fail loudly and stop. A smoke failure should halt the pipeline, because everything downstream of it is now measuring the same broken thing repeatedly.

A passing smoke test says the build is testable, which is a much weaker statement than the build working, and reporting it as the latter is how a release gets approved on five minutes of evidence.