LogoLogo
SlackLogin
  • Overview
    • Welcome
  • Setup & Configuration
    • Connecting to Trunk
    • Managing Your Organization
      • GitHub App Permissions
  • Integration with Slack
  • Flaky Tests (Beta)
    • Overview
    • Get Started
      • Test Frameworks
        • Android
        • Bazel
        • Behave
        • cargo-nextest
        • Cypress
        • Dart Test
        • Go
        • GoogleTest
        • Gradle
        • Jasmine
        • Jest
        • Karma
        • Maven
        • minitest
        • Mocha
        • Nightwatch
        • NUnit
        • Pest
        • PHPUnit
        • Playwright
        • Pytest
        • Robot Framework
        • RSpec
          • RSpec (Manual Uploads)
        • Swift Testing
        • Vitest
        • XCTest
        • Other Test Frameworks
      • CI Providers
        • Azure DevOps Pipelines
        • BitBucket Pipelines
        • Buildkite
        • CircleCI
        • Drone CI
        • GitHub Actions
        • GitLab
        • Jenkins
        • Semaphore CI
        • TeamCity
        • Travis CI
        • Other CI Providers
    • Dashboard
    • Flaky Test Detection
    • Quarantining
    • PR Comments
    • Ticketing Integrations
      • Jira Integration
      • Linear Integration
      • Other Ticketing Platforms
    • Webhooks
      • Slack Integration
      • Microsoft Teams Integration
      • GitHub Issues Integration
      • Linear Integration
    • Uploader CLI Reference
  • Merge Queue
    • Overview
    • How does it work?
    • Setup
      • Quick Start
      • Settings
      • Integration for Slack
    • Concepts and Optimizations
      • Predictive Testing
      • Optimistic Merging
      • Pending Failure Depth
      • Anti-Flake Protection
      • Batching
      • Parallel Queues
        • Bazel
        • Nx
        • API
      • FAQ
    • Priority
    • Managing Merge Queue
      • Using the Merge UI
      • Metrics
      • Command Line
    • Webhooks
    • Reference
  • Code Quality
    • Overview
    • Why Metalinters?
      • How does it work?
      • Why Code Quality?
    • Setup & Installation
      • Initialize Trunk
      • Local Linting
      • Linting in CI
      • Nightly Report (Deprecated)
    • IDE Integration
      • VSCode
      • Neovim
      • GitHub Codespaces
    • Linters
      • Supported Linters
        • Actionlint
        • Ansible-lint
        • Autopep8
        • Bandit
        • Biome
        • Black
        • Brakeman
        • buf
        • Buildifier
        • cfnlint
        • Checkov
        • circleci
        • ClangFormat
        • clang-tidy
        • Clippy
        • cmake-format
        • codespell
        • cspell
        • cue-fmt
        • dart
        • deno
        • Detekt
        • djlint
        • dotenv-linter
        • dotnet-format
        • dustilock
        • ESLint
        • Flake8
        • git-diff-check
        • Gitleaks
        • Gofmt
        • gofumpt
        • goimports
        • gokart
        • golangci-lint
        • golines
        • google-java-format
        • graphql-schema-linter
        • hadolint
        • haml-lint
        • isort
        • iwyu
        • ktlint
        • kube-linter
        • markdown-link-check
        • markdown-table-prettify
        • Markdownlint
        • markdownlint-cli2
        • mypy
        • nancy
        • nixpkgs-fmt
        • opa
        • OSV-Scanner
        • Oxipng
        • perlcritic
        • perltidy
        • php-cs-fixer
        • phpstan
        • pmd
        • pragma-once
        • pre-commit-hooks
        • Prettier
        • prisma
        • psscriptanalyzer
        • Pylint
        • pyright
        • regal
        • remark-lint
        • renovate
        • rome
        • rubocop
        • Ruff
        • rufo
        • rustfmt
        • scalafmt
        • semgrep
        • ShellCheck
        • shfmt
        • sort-package-json
        • sourcery
        • sql-formatter
        • SQLFluff
        • sqlfmt
        • squawk
        • standardrb
        • stringslint
        • stylelint
        • stylua
        • SVGO
        • swiftformat
        • swiftlint
        • taplo
        • Terraform
        • terragrunt
        • terrascan
        • TFLint
        • tfsec
        • tofu
        • Trivy
        • Trufflehog
        • txtpbfmt
        • vale
        • Yamllint
        • yapf
      • Run Linters
      • Manage Linters
      • Configure Linters
      • Ignoring Issues and Files
      • Custom Linters
      • Shared Configs
      • Upgrades
    • Debugging
    • Licensing
  • CLI & API References
    • CLI Reference
      • Install
      • Getting Started
        • Code Quality
        • Merge Queue
        • Flaky Tests
        • Tools
        • Actions
          • Git Hooks
        • Announce
      • Compatibility
      • Caching
      • Commands Reference
        • Code Quality
        • Actions
        • Merge
      • Configuration
        • Plugins
          • Share Config Between Codebases
          • Exporting Linter Configs
        • Runtimes
        • Tools
        • Lint
          • Definitions
          • Commands
          • Output
          • Output Parsing
          • Files and Caching
          • Dependencies
          • Auto-Enable
        • Actions
          • Notifications
          • Logging and Troubleshooting
        • Merge
        • Telemetry
        • Per User Overrides
    • API Reference
      • Flaky Tests
      • Merge Queue
      • Webhooks Reference
  • Pricing & Security
    • Security
  • Billing
  • Community & Support
  • Links
    • Open App
    • Slack Community
    • Changelog
    • Feature Requests
On this page
  • Guides
  • Installing the CLI
  • Organization Slug and Token
  • Uploading Using the CLI
  • Testing Using the CLI
  • Validating Reports Locally
  • Upgrade
  • Full Command Reference
Edit on GitHub
  1. Flaky Tests (Beta)

Uploader CLI Reference

Last updated 1 month ago

Trunk detects and tracks flaky tests in your repos by receiving uploads from your test runs in CI, uploaded from the Uploader CLI. These uploads happen in the CI jobs used to run tests in your nightly CI, post-commit jobs, and PR checks.

Guides

If you're setting up Trunk Flaky Tests for the first time, you can follow the guides for your CI provider and test framework.

Installing the CLI

The CLI should be downloaded as part of your test workflow in your CI system. The launcher is platform agnostic and will download the latest version of the uploader for your platform.

You should always use the latest version of the uploader CLI by downloading it fresh in your CI jobs for the best detection results.

You can download the uploader CLI and mark it executable with the following command:

curl -fsSLO --retry 3 https://trunk.io/releases/trunk && chmod +x trunk

And you can verify that it's been downloaded properly by running:

./trunk flakytests -V

Under-the-hood, this downloads the which will download the appropriate binaries for your environment.

Organization Slug and Token

The CLI requires your Trunk organization slug and token passed through --org-url-slug and --token to upload results to the correct organzation.

You can find your organization slug and token by going to Settings > Manage > Organization.

Uploading Using the CLI

Trunk can accept JUnit XMLs through the --junit-paths argument:

./trunk flakytests upload --junit-paths "test_output.xml" \
   --org-url-slug <TRUNK_ORG_SLUG> \
   --token $TRUNK_API_TOKEN

Trunk can accept Bazel through the --bazel-bep-path argument:

./trunk flakytests upload --bazel-bep-path <BEP_JSON_PATH> \
   --org-url-slug <TRUNK_ORG_SLUG> \
   --token $TRUNK_API_TOKEN

Trunk can accept XCode through the --xcresults-path argument:

./trunk flakytests upload --xcresults-path <BEP_JSON_PATH> \
   --org-url-slug <TRUNK_ORG_SLUG> \
   --token $TRUNK_API_TOKEN

Testing Using the CLI

You can also execute tests and upload results to Trunk in a single step using the ./trunk flakytest test command to wrap your test command.

Trunk can accept JUnit XMLs through the --junit-paths argument:

./trunk flakytests test --junit-paths "test_output.xml" \
   --org-url-slug <TRUNK_ORG_SLUG> \
   --token $TRUNK_API_TOKEN \
   <YOUR_TEST_COMMAND>

Trunk can accept Bazel through the --bazel-bep-path argument:

./trunk flakytests test --bazel-bep-path <BEP_JSON_PATH> \
   --org-url-slug <TRUNK_ORG_SLUG> \
   --token $TRUNK_API_TOKEN \
   <YOUR_TEST_COMMAND>

Trunk can accept XCode through the --xcresults-path argument:

./trunk flakytests test --xcresults-path <XCRESULT PATH> \
   --org-url-slug <TRUNK_ORG_SLUG> \
   --token $TRUNK_API_TOKEN \
   <YOUR_TEST_COMMAND>

Validating Reports Locally

You can validate the test reports produced by your test frameworks before you set up Trunk in your CI jobs. This is currently only available for XML reports.

You can run the validate command like this:

./trunk flakytests validate --junit-paths "test_output.xml"

The ./trunk flakytests validate command will output any problems with your reports so you can address them before setting up Trunk in CI.

Validating the following 1 files:
  File set matching junit.xml:
    junit.xml

junit.xml - 1 test suites, 0 test cases, 0 validation errors

All 1 files are valid! ✅
Navigate to https://app.trunk.io/onboarding?intent=flaky+tests to continue using Trunk Flaky Tests! 🚀🧪

Upgrade

If you're using the flakytests CLI subcommand using the Trunk CLI locally, you can upgrade with this command:

./trunk flakytests --upgrade

Full Command Reference

The trunk command-line tool can upload and analyze test results. The trunk flakytests command accepts the following subcommands:

Command
Description

trunk flakytests upload

Upload data to Trunk Flaky Tests.

trunk flakytests validate

Validates if the provided JUnit XML files and prints any errors.

trunk flakytests test <COMMAND>

The upload and test commands accept the following options:

Argument
Description

--junit-paths <JUNIT_PATHS>

Path to the test output files. File globs are supported. Remember to wrap globs in "" quotes

--bazel-bep-path <BEP_JSON_PATH>

--xcresults-path <XCRESULT_PATH>

Path to a .xcresult directory, which contains test reports from xcodebuild.

--org-url-slug <ORG_URL_SLUG>

Trunk Organization slug, from the Settings page.

--token <TOKEN>

Trunk Organization (not repo) token, from the Settings page. Defaults to the TRUNK_API_TOKEN variable.

-h, --help

Additional detailed description of the upload command.

--repo-root

Path to the repository root. Defaults to the current directory.

--repo-url <REPO_URL>

Value to override URL of repository. Optional.

--repo-head-sha <REPO_HEAD_SHA>

Value to override SHA of repository head. Optional.

--repo-head-branch <REPO_HEAD_BRANCH>

Value to override branch of repository head. Optional.

--repo-head-commit-epoch <REPO_HEAD_COMMIT_EPOCH>

Value to override commit epoch of repository head. Optional.

--tags <TAGS>

Comma separated list of custom tag=value pairs. Optional.

--print-files

Print files which will be uploaded to stdout.

--dry-run

Run metrics CLI without uploading to API. Optional.

--team <TEAM>

Value to tag team owner of upload. Optional.

--codeowners-path <CODEOWNERS_PATH>

Value to override CODEOWNERS file or directory path. Optional.

--allow-empty-test-results

Don't fail commands if test results are empty or missing. Use it when you sometimes skip all tests for certain CI jobs. Defaults to true.

--variant <VARIANT_NAME>

Upload tests to a specific variant group. Optional.

The uploaded tests are processed by Trunk periodically, not in real-time. Wait for at least an hour after the initial upload before they’re displayed in the . Multiple uploads are required before a test can be accurately detected as broken or flaky.

Trunk accepts uploads in three main report formats, , , and XCode XCResult paths. You can upload each of these test report formats using the ./trunk flakytest upload command like this:

This is especially useful for , where the Trunk CLI will override the exit code of the test command if all failures can be quarantined, preventing flaky tests from failing your builds in CI.

If you installed the CLI in your CI jobs following the instructions in the , the CI job will automatically install the latest version of the CLI. You should always download a fresh copy of the latest CLI in CI.

Runs tests using the provided command, uploads results, checks whether the failures are tests, and correct the exit code based on that.

Path to a JSON serialized . Trunk will use the BEP file to locate test reports. Your test frameworks must still output .

XML
Bazel Event Protocol JSONs
Quarantining
Installing the CLI step
Bazel Build Event Protocol
compatible report formats

Guides by Test Frameworks

Guides by CI Provider

Uploads tab
Trunk CLI Launcher
quarantined
Make sure you are getting your Organization Slug, not the Organization Name.
Ensure you get your Organization API Token, not your repo token.