Skip to main content

GitHub Integration

Connect your GitHub repositories to DebuggAI for automated test runs on pull requests, status checks, and intelligent code change detection.

Why Connect GitHub?​

DebuggAI's GitHub App enables:

  • Automated PR Testing: Tests run automatically when pull requests are opened or updated
  • Status Checks: Pass/fail results appear directly on your PRs
  • Test Result Comments: Detailed reports posted as PR comments
  • Code Change Detection: Smart analysis of which tests need to run based on changed files

Installation Steps​

1. Navigate to GitHub Settings​

From the DebuggAI dashboard:

  1. Click Settings in the left sidebar
  2. Select GitHub

2. Connect GitHub​

  1. Click Connect GitHub
  2. You'll be redirected to GitHub to authorize the DebuggAI GitHub App
  3. Review the requested permissions:
    • Read access to code and metadata
    • Read and write access to checks and pull requests
    • Webhook access for push and PR events

3. Select Repositories​

After authorization:

  1. Choose All repositories or Only select repositories
  2. For selective access, check the repositories you want DebuggAI to monitor
  3. Click Install to complete setup

Your connected repositories will appear in DebuggAI within 1-2 minutes.

Features Once Connected​

PR-Triggered Test Runs​

When a pull request is opened, updated, or reopened:

  1. DebuggAI receives a webhook from GitHub
  2. Relevant test suites are queued based on your configuration
  3. Tests execute against the PR branch
  4. Results are reported back to GitHub

Status Checks​

DebuggAI posts status checks directly to your pull requests:

  • Pending: Tests are queued or running
  • Success: All tests passed
  • Failure: One or more tests failed

You can configure branch protection rules in GitHub to require DebuggAI checks to pass before merging.

Test Result Comments​

After test runs complete, DebuggAI posts a summary comment on the PR including:

  • Total tests run, passed, failed, skipped
  • Links to detailed test reports
  • Screenshots and videos for failed tests
  • AI-generated debugging insights for failures

Branch Pattern Matching​

Configure which branches trigger test runs:

PatternExample Matches
mainOnly the main branch
feature/*feature/login, feature/checkout
release/*release/v1.0, release/2024-01
**All branches

Push Notifications​

For branches like main or master, configure tests to run on every push, not just PRs. This ensures your default branch stays healthy.

Configuration Options​

Branch Triggers​

In your project settings, configure which branches trigger tests:

  1. Navigate to your project in DebuggAI
  2. Go to Settings > Triggers
  3. Add branch patterns for:
    • PR branches: Which PR target branches trigger tests
    • Push branches: Which direct pushes trigger tests

Test Suite Selection​

Choose which test suites run on PRs:

  1. Go to Settings > Test Suites
  2. For each suite, toggle Run on PR
  3. Optionally configure different suites for different branch patterns

Status Check Settings​

Configure how DebuggAI reports to GitHub:

SettingDescription
RequiredMark the check as required (configure in GitHub branch protection)
Report pendingShow pending status while tests queue
Comment on PRPost detailed results as PR comment
Comment on failure onlyOnly comment when tests fail

Environment Configuration​

Adding Secrets​

Your tests need access to environment variables and secrets:

  1. Navigate to your project
  2. Go to Environment tab
  3. Add secrets like:
    • DATABASE_URL
    • API_KEY
    • Third-party service credentials

Use the same secrets from your staging environment.

Per-Branch Environments​

For different configurations per branch:

  1. Go to Environment > Branch Overrides
  2. Add branch pattern and environment variables
  3. These override base environment for matching branches

Troubleshooting​

Project Not Appearing​

Symptoms: Repository connected but no project in DebuggAI

Solutions:

  1. Wait 1-2 minutes for initial webhook processing
  2. Open a PR or push to trigger project creation
  3. Check GitHub webhook delivery in repository settings
  4. Verify the DebuggAI app has access to the repository

Webhooks Not Firing​

Symptoms: PRs opened but no test runs triggered

Solutions:

  1. Go to GitHub repository Settings > Webhooks
  2. Find the DebuggAI webhook
  3. Check Recent Deliveries for errors
  4. Common issues:
    • Webhook URL unreachable (temporary outage)
    • Payload too large (simplify PR)
    • Authentication failed (reconnect GitHub)

Permission Errors​

Symptoms: "Insufficient permissions" errors in test runs

Solutions:

  1. Go to GitHub Settings > Applications > Installed GitHub Apps
  2. Find DebuggAI and click Configure
  3. Ensure the repository has access granted
  4. Check that required permissions haven't been revoked

Status Checks Not Appearing​

Symptoms: Tests run but no status on PR

Solutions:

  1. Verify Report to GitHub is enabled in project settings
  2. Check the GitHub App still has checks permission
  3. Look for errors in the test run logs

Reconnecting GitHub​

If the integration stops working:

  1. Go to DebuggAI Settings > GitHub
  2. Click Disconnect
  3. Wait 30 seconds
  4. Click Connect GitHub and re-authorize
  5. Select repositories again

Your existing project configurations are preserved.

Rate Limiting​

GitHub has API rate limits. If you see rate limit errors:

  • Reduce concurrent PR activity
  • Stagger test triggers across repositories
  • Contact support if limits are consistently hit

Managing Repository Access​

Adding Repositories​

After initial setup, add more repositories:

  1. Go to GitHub Settings > Applications > Installed GitHub Apps
  2. Click Configure on DebuggAI
  3. Under Repository access, add new repositories

Removing Repositories​

To stop DebuggAI from monitoring a repository:

  1. Same steps as above
  2. Remove the repository from access list
  3. Existing project data in DebuggAI is preserved but no new runs trigger

Next Steps​