Skip to main content

Commit-Based Testing

Automatically generate and run tests when you commit code.

Enable Commit Testing​

VS Code Extension​

  1. Settings (Cmd+,) → Search "DebuggAI"
  2. Enable "Commit Testing"
  3. Set "Test Generation": Automatic

Configuration​

settings.json
{
  "debuggAI.commitTesting": true,
  "debuggAI.autoGenerate": true,
  "debuggAI.testOnPush": true
}

How It Works​

1. AI Analyzes Changes​

When you commit, AI reviews:

  • Modified files
  • Added components
  • Changed functionality
  • Removed features

2. Generates Relevant Tests​

AI creates tests like:

  • "Test new signup validation logic"
  • "Test updated checkout flow"
  • "Test removed feature doesn't break existing flows"

3. Runs Tests Automatically​

  • Tests execute against localhost
  • Results appear in VS Code
  • Failed tests block push (optional)

Smart Test Generation​

Component Changes​

// Modified: components/LoginForm.tsx
// AI generates: "Test login form with new validation rules"

Route Changes​

// Added: pages/checkout/success.tsx
// AI generates: "Test checkout success page displays correctly"

API Changes​

// Modified: api/auth/login.ts
// AI generates: "Test login API with updated authentication flow"

Configuration Options​

Test Triggers​

settings.json
{
  "debuggAI.commitTesting": true,      // Test on commit
  "debuggAI.testOnPush": false,        // Test on push
  "debuggAI.testOnPR": true,           // Test on pull request
  "debuggAI.testScheduled": "daily"    // Scheduled tests
}

Test Scope​

settings.json
{
  "debuggAI.testScope": "changed",     // Only test changed areas
  "debuggAI.fullSuite": false,         // Run full test suite
  "debuggAI.criticalOnly": true        // Only critical path tests
}

Blocking Behavior​

settings.json
{
  "debuggAI.blockOnFailure": true,     // Block push if tests fail
  "debuggAI.requireApproval": false,   // Manual approval for tests
  "debuggAI.retryFailures": 2          // Auto-retry failed tests
}

Git Integration​

Pre-commit Hook​

Automatically runs tests before commit:

.git/hooks/pre-commit
#!/bin/sh
# Run DebuggAI tests before commit
npx @debugg-ai/cli test --commit

Pre-push Hook​

Block pushes if tests fail:

.git/hooks/pre-push
#!/bin/sh
# Verify tests pass before push
npx @debugg-ai/cli verify --exit-on-failure

GitHub Actions Integration​

.github/workflows/debuggai.yml
name: DebuggAI Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run DebuggAI Tests
        env:
          DEBUGGAI_API_KEY: ${{ secrets.DEBUGGAI_API_KEY }}
        run: |
          npm install
          npm run dev &
          npx @debugg-ai/cli test --wait-for-server

Team Workflows​

Code Review Integration​

  • Tests auto-run on PR creation
  • Results appear in PR comments
  • Block merge on test failures
  • Team notifications on issues

Branch Protection​

branch-protection.json
{
  "required_status_checks": {
    "strict": true,
    "contexts": ["debuggai/tests"]
  },
  "enforce_admins": true,
  "required_pull_request_reviews": {
    "required_approving_review_count": 1
  }
}

Advanced Patterns​

Feature Branch Testing​

# Test feature branch against main
git checkout feature/new-auth
npx @debugg-ai/cli test --compare-branch main

Release Testing​

# Comprehensive tests before release
git checkout release/v2.0
npx @debugg-ai/cli test --full-suite --critical-path

Hotfix Testing​

# Quick critical tests for hotfixes
git checkout hotfix/login-bug
npx @debugg-ai/cli test --critical-only --fast

Monitoring & Alerts​

Slack Integration​

Configure through VS Code extension settings:

  • Slack Webhook URL: Add in DebuggAI extension settings
  • Channel: Specify notification channel
  • Alert Types: Choose when to notify (failures, successes, etc.)

Email Alerts​

Set up through debugg.ai dashboard:

  • Team Settings → Notifications
  • Add email addresses for test result notifications
  • Configure alert frequency and failure thresholds

Performance Optimization​

Parallel Testing​

settings.json
{
  "debuggAI.parallelTests": 3,         // Run 3 tests simultaneously
  "debuggAI.testTimeout": 30000,       // 30 second timeout
  "debuggAI.smartRetry": true          // Intelligent retry logic
}

Test Caching​

settings.json
{
  "debuggAI.cacheResults": true,       // Cache test results
  "debuggAI.skipUnchanged": true,      // Skip tests for unchanged code
  "debuggAI.incrementalTesting": true  // Only test affected areas
}

Troubleshooting​

Tests Not Triggering​

  1. Check commit testing is enabled
  2. Verify git hooks are installed
  3. Ensure localhost is running
  4. Check API key is valid

Performance Issues​

  1. Reduce parallel test count
  2. Enable test caching
  3. Use incremental testing
  4. Increase timeout values

False Positives​

  1. Review test descriptions
  2. Update test patterns
  3. Add test exclusions
  4. Report to AI training

Team Adoption​

  1. Start with optional testing
  2. Gradually enable blocking
  3. Train team on test descriptions
  4. Monitor and adjust settings