Skip to main content
Quarantine keeps flaky tests from blocking CI. For some code, that’s the wrong trade. You may want every test for payments, authentication, or a database migration to block a merge when it fails, even if it’s flaky. This recipe sets a Never Quarantine override on every test under the paths you choose, and keeps it in place:
  • A scheduled script finds every test whose file matches your paths and sets the override on the ones that don’t have it. Tests added since the last run get it too.
  • An optional webhook handler puts the override back as soon as someone changes it, and tells your team in Slack.
  • A check confirms that no test under your paths is quarantined.

Before you begin

  • Quarantining must be on for each test collection. An override has no effect in a collection where quarantining is off.
  • Create an API key with the Quarantine tests permission. Reading tests needs no permission. If a collection’s Manual Quarantine Permissions setting is Admins only, an API key still works, since only an admin can create one.
  • The script needs Node.js 18 or later.

1. Write down your protected paths

Keep the paths in your repository, so changing the policy goes through code review like any other change. Each entry is a path prefix. A test is protected when its file, the path your test reports upload, starts with one of them:
quarantine-policy.json
End a directory prefix with /, so src/auth/ doesn’t also protect src/authors/. reason is saved with each override. People see it on the test’s page in Trunk, so say where the policy lives.

2. Find the tests under those paths

Call GET /v2/tests with file set to each prefix, paging through the results with nextCursor. file also matches a path that contains the prefix further in, such as lib/src/payments/, so the script keeps only the tests whose file starts with the prefix. Without testCollectionId, the list covers every test collection in your organization. A test that uploads to two collections, such as one for pull requests and one for the default branch, appears once in each, with a different id. Both need the override.

3. Set the override

Call POST /v2/tests/quarantine with up to 100 tests at a time, each with mode set to neverQuarantine. Each test in the response either succeeded, was unchanged because it already had that override, or failed with an error. A failure doesn’t undo the other tests in the request, and a test that failed with INTERNAL_ERROR can be sent again. The request is safe to repeat: a test that already has the override is left alone. That’s what makes the script safe to run on a schedule.
The script also replaces an Always Quarantine override on a protected test, because the policy says those tests are never quarantined. To leave those tests alone and only list them, set REPLACE_ALWAYS_QUARANTINE to false in the script.
This script does steps 2 and 3. Run it with DRY_RUN=1 first to see which tests it would change without changing them.
protect-paths.mjs

4. Run it every day

New tests don’t have the override until the script runs again, so run it at least daily. This GitHub Actions workflow runs every day at 06:00 UTC, and whenever the policy file changes on the default branch. Store the API key as a repository secret:
.github/workflows/protect-paths.yml
The number of requests depends on how many tests are under your paths, not on how many tests you have. Requests share your organization’s rate limit.
Removing a path from the policy doesn’t remove the overrides the script set. To clear them, send the same tests to POST /v2/tests/quarantine with mode set to clear.

5. Put overrides back right away (optional)

Between runs, a person can remove the override from a protected test in the web app. If the test is flaky, Trunk can quarantine it before the next run. To catch that right away, subscribe an endpoint you run to two webhook events: Both payloads include the test’s file_path, so the handler can check the policy without another request. A change your API key makes doesn’t send v2.test_case.quarantining_setting_changed, so the handler’s own writes don’t trigger it again. This handler reuses the functions from protect-paths.mjs. Call it from your web server with each webhook’s parsed body, after you verify the webhook’s signature:
handle-webhook.mjs
The Slack message goes to an incoming webhook. The person who changed the override sees it set back on the test’s Events tab, with your policy’s reason.

6. Check that it’s working

Call GET /v2/tests with quarantineState=quarantined and file set to each prefix, keeping the tests whose file starts with it. The result should be empty. A test that turns up is quarantined under a protected path, usually because it’s new since the last run, or because its collection’s override settings refused the API key. Never Quarantine means a flaky test under a protected path blocks CI until someone fixes it. To find those tests, call the same endpoint with status=flaky,broken, and make sure each has an owner. Post a Weekly Flaky Test Digest groups them by code owner, and POST /v2/tests/{testId}/create-ticket files a ticket for one.