Anti-Flake Protection
Using optimistic merging and pending-failure-depth to protect your Merge Queue from flaky failures
Sometimes, a pull request (PR) fails testing for reasons outside of the actual code change - a test was flaky, a CI runner was disconnected, etc.. If a second PR that depends on the first does pass, then it is very likely that the first PR was actually good and simply experienced a transient failure. Trunk Merge Queue can use the combination of Optimistic Merging and Pending Failure Depth to merge pull requests that would otherwise be rejected from the queue. In the video below you can see an example of this anti-flake protection:
what's happening? | queue |
---|---|
A, B, C begin predictive testing |
|
B fails testing |
|
predictive failure depth keeps B from being evicted while C tests |
|
C passes |
|
optimistic merging allows A, B, C to merge |
|
Optimistic Merging only works when the Pending Failure Depth is set to a value greater than zero. When zero or disabled, Merge will not hold any failed tests in the queue.
Enabling anti-flake protection
Achieve anti-flake protection works by enabling Optimistic Merge and setting Pending Failure Depth greater than 0 in the Merge UI settings:
The Fine Print There is a small tradeoff to be made when optimistic merging is used. You can get into a situation where an actually broken test in say change 'B' is corrected by a change in 'C'. In this case if you later reverted 'C' your build would be broken.
Last updated