LogoLogo
SlackLogin
  • Overview
    • Welcome
  • Setup & Configuration
    • Connecting to Trunk
    • Managing Your Organization
      • GitHub App Permissions
  • Integration with Slack
  • Flaky Tests (Beta)
    • Overview
    • Get Started
      • Test Frameworks
        • Android
        • Bazel
        • Behave
        • cargo-nextest
        • Cypress
        • Dart Test
        • Go
        • GoogleTest
        • Gradle
        • Jasmine
        • Jest
        • Karma
        • Maven
        • minitest
        • Mocha
        • Nightwatch
        • NUnit
        • Pest
        • PHPUnit
        • Playwright
        • Pytest
        • Robot Framework
        • RSpec
        • Swift Testing
        • Vitest
        • XCTest
        • Other Test Frameworks
      • CI Providers
        • Azure DevOps Pipelines
        • BitBucket Pipelines
        • Buildkite
        • CircleCI
        • Drone CI
        • GitHub Actions
        • GitLab
        • Jenkins
        • Semaphore CI
        • TeamCity
        • Travis CI
        • Other CI Providers
    • Dashboard
    • Flaky Test Detection
    • Quarantining
    • PR Comments
    • Ticketing Integrations
      • Jira Integration
      • Linear Integration
      • Other Ticketing Platforms
    • Webhooks
      • Slack Integration
      • Microsoft Teams Integration
      • GitHub Issues Integration
      • Linear Integration
    • Uploader CLI Reference
  • Code Quality
    • Overview
    • Why Metalinters?
      • How does it work?
      • Why Code Quality?
    • Setup & Installation
      • Initialize Trunk
      • Local Linting
      • Linting in CI
      • Nightly Report (Deprecated)
    • IDE Integration
      • VSCode
      • Neovim
      • GitHub Codespaces
    • Linters
      • Supported Linters
        • Actionlint
        • Ansible-lint
        • Autopep8
        • Bandit
        • Biome
        • Black
        • Brakeman
        • buf
        • Buildifier
        • cfnlint
        • Checkov
        • circleci
        • ClangFormat
        • clang-tidy
        • Clippy
        • cmake-format
        • codespell
        • cspell
        • cue-fmt
        • dart
        • deno
        • Detekt
        • djlint
        • dotenv-linter
        • dotnet-format
        • dustilock
        • ESLint
        • Flake8
        • git-diff-check
        • Gitleaks
        • Gofmt
        • gofumpt
        • goimports
        • gokart
        • golangci-lint
        • golines
        • google-java-format
        • graphql-schema-linter
        • hadolint
        • haml-lint
        • isort
        • iwyu
        • ktlint
        • kube-linter
        • markdown-link-check
        • markdown-table-prettify
        • Markdownlint
        • markdownlint-cli2
        • mypy
        • nancy
        • nixpkgs-fmt
        • opa
        • OSV-Scanner
        • Oxipng
        • perlcritic
        • perltidy
        • php-cs-fixer
        • phpstan
        • pmd
        • pragma-once
        • pre-commit-hooks
        • Prettier
        • prisma
        • psscriptanalyzer
        • Pylint
        • pyright
        • regal
        • remark-lint
        • renovate
        • rome
        • rubocop
        • Ruff
        • rufo
        • rustfmt
        • scalafmt
        • semgrep
        • ShellCheck
        • shfmt
        • sort-package-json
        • sourcery
        • sql-formatter
        • SQLFluff
        • sqlfmt
        • squawk
        • standardrb
        • stringslint
        • stylelint
        • stylua
        • SVGO
        • swiftformat
        • swiftlint
        • taplo
        • Terraform
        • terragrunt
        • terrascan
        • TFLint
        • tfsec
        • tofu
        • Trivy
        • Trufflehog
        • txtpbfmt
        • vale
        • Yamllint
        • yapf
      • Run Linters
      • Manage Linters
      • Configure Linters
      • Ignoring Issues and Files
      • Custom Linters
      • Shared Configs
      • Upgrades
    • CI Setup
      • GitHub Integration
      • Manual Setup
    • Debugging
    • Licensing
  • Merge Queue
    • Overview
    • How does it work?
    • Setup
      • Quick Start
      • Settings
      • Integration for Slack
    • Concepts and Optimizations
      • Predictive Testing
      • Optimistic Merging
      • Pending Failure Depth
      • Anti-Flake Protection
      • Batching
      • Parallel Queues
        • Bazel
        • Nx
        • API
      • FAQ
    • Priority
    • Managing Merge Queue
      • Using the Merge UI
      • Metrics
      • Command Line
    • Webhooks
    • Reference
  • CLI & API References
    • CLI Reference
      • Install
      • Getting Started
        • Code Quality
        • Merge Queue
        • Flaky Tests
        • Tools
        • Actions
          • Git Hooks
        • Announce
      • Compatibility
      • Caching
      • Commands Reference
        • Code Quality
        • Actions
        • Merge
      • Configuration
        • Plugins
          • Share Config Between Codebases
          • Exporting Linter Configs
        • Runtimes
        • Tools
        • Lint
          • Definitions
          • Commands
          • Output
          • Output Parsing
          • Files and Caching
          • Dependencies
          • Auto-Enable
        • Actions
          • Notifications
          • Logging and Troubleshooting
        • Merge
        • Telemetry
        • Per User Overrides
    • API Reference
      • Flaky Tests
      • Merge Queue
      • Webhooks Reference
  • Pricing & Security
    • Security
  • Billing
  • Community & Support
  • Links
    • Open App
    • Slack Community
    • Changelog
    • Feature Requests
On this page
  • 1. Create a GitHub Token
  • 2. Add a New Webhook
  • 3. Add Custom Headers
  • 4. Customize Your Transformation
  • 5. Test Your Webhook
  • 6. Monitoring Webhooks
  • Congratulations!
Edit on GitHub
  1. Flaky Tests (Beta)
  2. Webhooks

GitHub Issues Integration

Learn how to automatically create GitHub Issues with Flaky Test webhooks

Last updated 1 month ago

Trunk allows you to automate GitHub Issue creation through webhooks. This will allow you to create GitHub issues and auto-assign them to using Webhooks.

This guide will walk you through integrating Trunk Flaky Tests with GitHub Issues through webhooks. You will be able to automatically generate GitHub issues for new flaky or broken tests. This guide should take 15 minutes to complete.

1. Create a GitHub Token

Before you can create a webhook to automate GitHub Issue creation, you need to create an API token to authorize your requests.

  1. Navigate to GitHub Developer Settings under Settings > Developer settings

  2. Under Personal access token > Fine-grained tokens > Click Generate new token

  3. Name the token something like Trunk Flaky Tests so you can recognize your token and set it never to expire.

  4. Select the repositories you need to create issues to under Repository access

  5. Under Permissions > Repository Permissions, select Read and Write access for Issues.

  6. Click Generate Token and copy your API token.

2. Add a New Webhook

Trunk uses Svix to integrate with other services, such as GitHub Issues through webhooks.

You can create a new endpoint by:

  1. From your profile on the top right, navigate to Settings

  2. Under Organization > Webhooks, click Automate GitHub Issue Creation

  3. Create the new endpoint. You will be redirected to the endpoint configuration view.

3. Add Custom Headers

The GitHub Issues API requires some custom headers. You can configure custom headers in the endpoint configuration:

  1. You can add custom headers under Webhooks > Advanced > Custom Headers.

  2. Fill in the Key and Value referencing the table below, and click the + button to add each header.

You'll need to configure the following headers.

Key
Value

Accept

application/vnd.github+json

Authorization

Bearer <YOUR_API_TOKEN>

X-GitHub-Api-Version

2022-11-28

4. Customize Your Transformation

Transformations are custom code snippets you can write to customize the GitHub issues created by the webhook. A working template transformation will be added automatically for your webhook, but you can further customize the behavior of this webhook.

  1. In the endpoint configuration view, navigate to the Advanced tab. Under Transformation, toggle the Enabled switch.

  2. Click Edit transformation to update your transformation code, and click Save to update the transformation.

The generated webhook template contains several configurable constants out of the box:

Constant
Description

GITHUB_ISSUE_LABEL_IDS

(Optional) GitHub labels that will be assigned to issues created by Trunk.

PRS_IMPACTED_THRESHOLD

Issues will be created only for flaky tests that have impacted more PRs than the PRS_IMPACTED_THRESHOLD. You can adjust this value if you see many issues about low-impact flaky tests.

/**
 * @param webhook the webhook object
 * @param webhook.method destination method. Allowed values: "POST", "PUT"
 * @param webhook.url current destination address
 * @param webhook.eventType current webhook Event Type
 * @param webhook.payload JSON payload
 * @param webhook.cancel whether to cancel dispatch of the given webhook
 */
// IDs of any labels you want added to the GitHub issue.
const GITHUB_ISSUE_LABEL_IDS = [];

// Below are various configs to fine-tune when an issue is created.

// At least this many PRs need to be impacted for an issue to be created.
const PRS_IMPACTED_THRESHOLD = 2;

function handler(webhook) {
  const impacted_prs = webhook.payload.test_case.pull_requests_impacted_last_7d;
  const newStatus = webhook.payload.status_change.current_status.value;

  // Filter for only flaky tests that impact more than the provided threshold
  if (newStatus !== "flaky" || impacted_prs < PRS_IMPACTED_THRESHOLD) {
    webhook.payload = "canceled";
    webhook.cancel = true;
    return webhook;
  }
  webhook.payload = {
    "title":`Flaky Test: ${webhook.payload.test_case.name.substring(0, 25)} transitioned to ${webhook.payload.status_change.current_status.value}`,
    "body": summarizeTestCase(webhook.payload),
    "labels": GITHUB_ISSUE_LABEL_IDS,
    // Uncomment this function for auto asignment
    // "assignees": webhook.payload.test_case.codeowners.map((assignee)=>{
    //     // Strip the `@` symbol from codeowners
    //     return assignee.slice(1)
    // })
  }
  return webhook
}

function summarizeTestCase(payload) {
    const {
        status_change: {
            previous_status
        },
        test_case: {
            name,
            file_path,
            status,
            quarantine,
            repository,
            codeowners,
            failure_rate_last_7d,
            most_common_failures,
            pull_requests_impacted_last_7d,
            ticket,
            html_url
        }
    } = payload;
    // Construct a comprehensive issue body with key details
    const issueBody = `See all details on the [Trunk Test Detail page](${html_url})
 
Transition time: ${status.timestamp}
 
Latest failure: Dec 9, 2024
 
Severity (last 7 days): ${(failure_rate_last_7d * 100).toFixed(2)}% failure rate; impacting ${pull_requests_impacted_last_7d} PRs
 
Ownership: this test is owned by ${(codeowners || ['@unassigned']).join(', ')}

___
__The  most common failure reason (out of ${most_common_failures.length} identified failure reason) are:__

${
  most_common_failures.map((failure, index) => {
    return `**Reason #${index + 1}**: "${failure.summary}" \n`
  })
}
    `
    return issueBody
}

Automatically Assign Issues with CODEOWNERS

"assignees": webhook.payload.test_case.codeowners.map((assignee)=>{
         // Strip the `@` symbol from codeowners
         return assignee.slice(1)
     })

Limitations of CODEOWNERS

  1. CODEOWNERS supports assigning files to teams, but GitHub doesn't support assigning issues to teams. If you have team owners in your CODEOWNERS file, the requests will fail.

  2. If your code owners do not map 1:1 with GitHub users, you will need to provide your own mapping, or webhooks will fail.

5. Test Your Webhook

You can create test issues by delivering a mock webhook. You can do this by:

  1. In the endpoint configuration view, navigate to the Testing tab and select a Send event

  2. Under Subscribed events, select test_case.status_changedas the event type to send.

  3. Click Send Example to test your webhook

6. Monitoring Webhooks

You can monitor the events and the webhook's delivery logs in the Overview tab of an endpoint configuration view.

You can see an overview of how many webhook deliveries have been attempted, how many are successful, how many are in flight, and how many fail in the Attempt Delivery Status modal.

Congratulations!

A GitHub Issue will now be created when a test's health status changes. You can further modify your transformation script to customize your issues.

Login to

Paste your GitHub repo's Issues endpoint into Endpoint URL. Your Endpoint URL should be formatted as: https://api.github.com/repos/{OWNER}/{REPO}/issues. You can verify the URL by visiting it in your browser, such as .

Review the transformation code automatically generated for GitHub issues. You can customize this transformation at any time. Learn more about .

If you're having trouble adding a new webhook endpoint with Svix, please see the .

You can test the transformation by selecting the test_case.status_changed payload and clicking Run Test. This will test the transformation but not send a message. You will learn to send a test message in .

Here is the provided transformation for context. You can customize your GitHub Issues integration by following the and documentation.

If you have configured for your GitHub repo, you can create issues with assignees using CODEOWNERS. You can uncomment the code block on lines 31-35 or use a snippet similar to:

The example payload provided for testing has the CODEOWNERS assigned to @backend. If you're testing following the instructions in , the delivery attempt can fail.

You can see a list of past delivery attempts in the Message Attempts modal. You can filter this list by Succeeded and Failed status, and you can click on each message to see the Message content, response code, and error message of each attempt. You can learn more about and in the Svix docs.

Trunk Flaky Tests
https://api.github.com/repos/trunk-io/docs/issues
Adding Endpoint docs from Svix
GitHub
Svix transformations
CODEOWNERS
replaying messages
filtering logs
See the Trunk webhook event catalog
Learn more about consuming webhooks in the Svix docs
Learn more about the GitHub Issues API
customizing transformations
step 5
step 5
CODEOWNERS
GitHub Issue created automatically with webhooks.