Get Started
Last updated
Last updated
Trunk Flaky Tests works by changing all of your CI jobs that run tests to upload their test results to Trunk. This setup looks slightly different depending on which CI system and test framework(s) you use, but we have guides for all the popular systems.
If you prefer to see an example onboarding flow, watch a video here:
For Trunk to start detecting flaky tests, it needs to ingest the results from all of your CI test runs. To get started, configure your test framework (Jest, Pytest, JUnit, etc.) to output results. You'll do this only for CI jobs, not for running tests locally.
You can follow along using the in-app onboarding wizard or read about it in our docs.
After you've set your test framework to output test results, you're ready to update your CI jobs to upload test results to Trunk.
You can follow along using the in-app onboarding wizard or read about it in our docs.
Once you've updated your CI configuration to upload to Trunk, run a test job and ensure it completes without errors.
Once you've confirmed data is flowing properly into Trunk, let data accumulate for several days and check back to analyze the results. We'll also email you when your first flaky test is detected.
You can learn more about how Trunk detects flaky tests in our documentation:
Flaky Tests can post comments on GitHub pull requests to provide a summary of all the tests run on this PR, across multiple CI jobs or even CI systems, as well as whether those failures are due to flakiness, the history of the failures of that test, and more.
To enable PR Comments, see our documentation:
Quarantining is an advanced feature that allows your CI test jobs to pass even if a flaky test is failing. This eliminates the pain developers feel from dealing with flaky tests.
To enable Quarantining, see our documentation:
You must and Trunk organization to complete the steps below.
If you run the same tests on different environments or architectures, for example, running tests for a mobile app on iOS and Android, use the during upload to detect flakes on each environment independently.
In , select the repo you've just updated and navigate to Flaky Tests > Uploads. Confirm that you see the upload(s) from the previous step.
Trunk needs to ingest a large amount of test data from many different pull request branches and , main for example, before it can detect flaky tests accurately. This may happen quickly if you have a very high-velocity repo and run test jobs hundreds of times a day.
