Quarantining

Quarantining isolates known flaky tests to prevent them from blocking CI jobs while continuing to run and track their results. The system identifies flaky tests at runtime and overrides their exit codes when they fail, allowing your CI pipeline to pass without requiring code changes to disable problematic tests.

Why use quarantining: It acts as a crucial stopgap, minimizing the disruption from known flaky tests while your team works on fixing them. By quarantining flaky tests, you unblock critical CI pipelines—especially your merge queue—and regain development velocity without losing visibility, as these tests continue to run and upload results. This constant stream of data allows you to prioritize fixing the worst offenders based on their ongoing impact.

What does "Quarantined" mean?

A quarantined test continues running in CI and uploading results to Trunk Flaky Tests, but its failures won't block your pipeline. The Trunk Analytics CLI checks with Trunk's backend to determine if failed tests are quarantined, then overrides the exit code for those failures. When all failures in a CI job come from quarantined tests, the entire job passes.

Why this matters: You maintain complete test coverage and historical data while preventing known problematic tests from disrupting your development cycle.

How tests get quarantined

Tests can be quarantined through two methods:

1. Manual Quarantine - You explicitly select specific tests using override settings

2. Auto-Quarantine (when enabled) - Tests already flagged by Trunk's flaky detection are automatically quarantined

Tests are auto-quarantined only if detected as flaky or manually marked as flaky. For manually quarantined tests, all failures are quarantined regardless of test state.

Enable quarantining

With quarantining enabled, the Analytics Uploader will compare failed test cases against known flaky tests. If a test is known to be flaky, it will be quarantined. If all failed tests are quarantined, the exit code of the test command will be overridden to return 0 and the CI job will pass.

Quarantining settings

To enable quarantining, navigate to Settings > Repositories > repository > Flaky Tests > toggle on Enable Test Quarantining.

Here's what each of these options does when enabled:

Updates in CI

If you're using the provided GitHub Actions workflow to upload test results to Flaky Test, you can quarantine flaky tests by wrapping the test command or as a follow-up step.

If you're using the Trunk CLI directly or other CI providers, check the instructions in the Using The Trunk CLI Directly tab.

name: Run Tests And Upload Results
on:
  workflow_dispatch:
jobs:
  upload-test-results:
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:   
    - name: Run Tests
      id: unit_tests
      shell: bash
      run: <COMMAND TO RUN TESTS>
      continue-on-error: true
        
    - name: Upload test results
      if: always()
      uses: trunk-io/analytics-uploader@v1
      with:
        junit-paths: <TEST OUTPUT PATH>
        org-slug: my-trunk-org-slug
        token: ${{ secrets.TRUNK_API_TOKEN }}

name: Run Tests And Upload Results
on:
  workflow_dispatch:
jobs:
  upload-test-results:
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Run tests and upload results
        uses: trunk-io/analytics-uploader@v1
        with:
          junit-paths: <TEST OUTPUT PATH>
          run: <COMMAND TO RUN TESTS>
          token: ${{ secrets.TRUNK_API_TOKEN }}
          org-slug: my-trunk-org-slug

<run my tests> || true # doesn't fail job on failure
|
    ./trunk flakytests upload \
        --org-url-slug $TRUNK_ORG_SLUG \
        --token $TRUNK_API_TOKEN \
        --junit-paths $JUNIT_PATH

./trunk flakytests test \
    --org-url-slug <TRUNK_ORG_SLUG> \
    --token $TRUNK_API_TOKEN \
    --junit-paths $JUNIT_PATH \
    --allow-empty-test-results \
    <Test Command>

Quarantining with sharded or parallelized tests

If your CI runs multiple Playwright shards, wrap each npx playwright test invocation with Trunk’s quarantine support. The shard’s post-quarantine exit code becomes the source of truth.

1. Wrap each shard run

# Example per-shard step (args come from your CI; not shown here)
./trunk-quarantine -- npx playwright test <your usual args> --output=./pw-report-$SHARD_ID
echo $? > shard-status.txt

2. Use merge for reporting only

Run npx playwright merge-reports unconditionally to produce a single report, but do not use its result to gate CI.

3. Gate on wrapped shard codes

Aggregate the stored shard-status.txt files and fail if any is non-zero; otherwise succeed.

# Shards are created however you already do it (matrix, Playwright, etc.)
- name: Run Playwright (wrapped with Trunk quarantine)
  run: |
    set +e
    ./trunk-quarantine -- npx playwright test $PLAYWRIGHT_ARGS --output=pw-report-$SHARD_ID
    echo $? > shard-status.txt
  continue-on-error: true

- name: Upload shard artifacts
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: pw-$SHARD_ID
    path: |
      pw-report-$SHARD_ID/
      shard-status.txt

# Merge reports for reporting-only
- uses: actions/download-artifact@v4
  with: { path: ./artifacts }
- name: Merge reports (reporting-only)
  if: always()
  run: |
    reports=$(find ./artifacts -type d -name 'pw-report-*' -maxdepth 2)
    npx playwright merge-reports --reporter html $reports
- name: Set final status from wrapped shard codes
  if: always()
  run: |
    bad=0
    for f in $(find ./artifacts -name shard-status.txt); do c=$(cat "$f"); [ "$c" -ne 0 ] && bad=1; done
    exit $bad

Overriding individual tests

If you have tests that should never be quarantined or should always be quarantined regardless of their current health status, you can do this by overriding individual tests.

You can manually control a test's quarantine status from its details page.

• To set an override: Click the Quarantine (or Override) button, then select either Always Quarantine or Never Quarantine.

• To remove an override: Click the Remove Override button.

When a manual override is active, a banner shows who set it and when.

To review a history of all quarantine changes on a test, use the Quarantine Events filter within the Test History section. This will show every override, setting change, and comment, along with the author and timestamp for each entry.

Tracking quarantined jobs in the dashboard

Once quarantining is active, the Quarantining tab provides a central hub for monitoring its impact and effectiveness. This tab serves as a complete audit log of every CI job saved by the feature, allowing you to:

• Visualize Trends: A 30-day chart shows the number of jobs quarantined per day.

• Inspect Individual Jobs: A detailed table lists every quarantined job. Click any entry to see the specific tests that were quarantined.

• Isolate Critical Workflows: Use the filter to see how quarantining impacts specific branches, such as preventing flaky failures in your Merge Queue.

• Measure ROI: Use the data to quantify the number of builds saved and developer time reclaimed for your organization.

Audit logs

Trunk provides audit logs for all setting changes and overwrites for individual tests. You can access the audit log by navigating to Settings > Repositories > repository > Flaky Tests > Audit logs under the Enable Test Quarantining heading.

Quarantining API and webhooks

For advanced use cases, you can interact with quarantining features programmatically.

• API: Use the Flaky Tests API to fetch a list of all currently quarantined tests in your project.

• Webhooks: Subscribe to the test_case.quarantining_setting_changed event to trigger automated workflows whenever a test's quarantine override is modified. Learn more about Webhooks.