XCTest

A guide for generating Trunk-compatible test reports for XCode and xcodebuild

Test written in Objective-C and Swift projects using the XCTest framework can be run from the command line with xcodebuild. Trunk Flaky Tests accepts both .xcresult and JUnit XML files.

Using XCResult

xcodebuild produces a test result directory in the .xcresult default format. You can upload .xcresult directories directly to Trunk Flaky Tests. The trunk flakytests subcommand takes .xcresult paths using the --xcresult-path argument.

Using JUnit XML Output (Optional)

You can optionally configure xcodebuild to produce the JUnit XML format, which is more commonly used by other test frameworks. Using the xcbeautify open source tool, xcodebuild can produce JUnit compatible XML output that Trunk can ingest.

Install xcbeautify it on macOS with Homebrew. Alternative installation instructions here.

brew install xcbeautify

Then pipe the output of xcodebuild to xcbeautify with the --report junit option.

xcodebuild test -scheme Testo2 | xcbeautify --report junit 

This will produce a build/reports/junit.xml output file.

Test Suite Naming

xcbeautify will use the name of file the tests are in as the name of the output <testsuite> and the function name as the name attribute of each <testcase> element.

Disable Retries

You need to disable automatic retries if you previously enabled them. Retries compromise the accurate detection of flaky tests.

If you run tests in CI with the -retry-tests-on-failure option, disable it for more accurate results.

Next Step

Once you've configured your test runner to output JUnit XML or you have a .xcresult file, you're ready to modify your CI test jobs to actually upload test results to Trunk. See CI Providers for instructions to do this for the CI system you use.