# What Is Test Quarantine? — Definition, How to Do It, and the Rules That Keep It Honest

Canonical: https://commareports.com/glossary/test-quarantine
Published: 2026-11-02

> Test quarantine moves a known-flaky test out of the build gate while it keeps running and reporting. How it differs from skipping, how to set it up in Playwright, pytest and Cypress, and the rules that stop it becoming a graveyard.

# Test quarantine

**Test quarantine is moving a known-flaky test out of the build gate
while it keeps running and reporting.** The merge stops depending on
it; the test does not stop producing evidence.

It exists because the two obvious responses to a [flaky
test](/glossary/flaky-test) are both bad. Leave it in the gate and the
team learns to re-run red builds without reading them. Delete or skip it
and the coverage — and the evidence needed to fix it — disappears.

## Quarantine vs skip vs retry

|                           | Still runs | Still reported | Can fail the build      |
| ------------------------- | ---------- | -------------- | ----------------------- |
| Normal test               | Yes        | Yes            | Yes                     |
| **Quarantined**           | Yes        | Yes            | **No**                  |
| Skipped (`skip`, `fixme`) | No         | As skipped     | No                      |
| Retried until green       | Yes        | Usually hidden | Only if every try fails |

Retries are the trap. They keep the test in the gate but throw away the
signal, and most setups keep only the passing attempt's artifacts.

## How to set it up

The pattern is the same everywhere: tag, exclude from the gate, run the
tag in a job that is allowed to fail.

**Playwright** — tags are built in:

```ts
test("checkout applies coupon", { tag: "@quarantine" }, async ({ page }) => {
  /* … */
});
```

```bash
npx playwright test --grep-invert @quarantine   # gating job
npx playwright test --grep @quarantine          # non-blocking job
```

**pytest** — register a marker so typos fail loudly:

```ini
# pytest.ini
[pytest]
markers = quarantine: known flaky, does not gate merges
```

```bash
pytest -m "not quarantine"   # gating job
pytest -m quarantine         # non-blocking job
```

**Cypress** — tag the test and filter with a grep plugin such as
`@cypress/grep`, using an inverted tag in the gating run.

In CI, the non-blocking job is `continue-on-error: true` on GitHub
Actions or `allow_failure: true` on GitLab — it still shows red, it just
does not block.

## The rules that keep it honest

Quarantine without rules becomes a second suite that nobody reads. The
ones that work:

- **An owner and a ticket** for every quarantined test, at the moment it
  is quarantined.
- **A review date.** Past it, the test is fixed, deleted, or explicitly
  re-quarantined by someone who looked.
- **A size cap.** When the quarantine set is full, nothing new goes in
  until something comes out.
- **Exit on evidence.** A test leaves quarantine after a run of
  consecutive passes, not after someone thinks it is fixed.

## The evidence is the point

Quarantine only works if somebody reads the quarantined job's output.
That output — the trace, the failure screenshot, the timings across runs
— lives in the runner's [HTML report](/glossary/html-report), and in
most pipelines that report is a [build artifact](/glossary/build-artifact)
that expires before the review date comes round.

Publish the quarantine job's report to the same link on every run and
each build becomes a revision: the owner opens one URL and sees the
latest failure, with earlier runs still in the history. Notes like
"fails only on the 4-core runner" go on the failing test itself, where
the next person to pick it up will see them ([comment on
HTML](/comment-on-html)).

## Try it

Comma is free — unlimited reports, unlimited commenters, unlimited
revision history.

**[Publish a test report →](https://commareports.com/)**

### Related

- [Flaky test](/glossary/flaky-test) · [JUnit XML](/glossary/junit-xml)
- [Allure trend and history missing](/fix/allure-report-history-trend-missing)
- [Cypress mochawesome shows only the last spec](/fix/cypress-mochawesome-report-only-last-spec)
- [Share a Playwright report](/share-playwright-report) · [pytest](/share-pytest-report)
- [Glossary](/glossary)