Test Suites

Test suites help you organize, manage, and execute collections of related E2E tests efficiently.

What Are Test Suites

Test suites are organized collections of related E2E tests that can be executed together. They allow you to:

• Group related tests by feature, workflow, or test type
• Run multiple tests with a single action
• Track aggregate metrics across test collections
• Integrate with CI/CD for automated testing on commits

Creating a Test Suite

Steps to Create

1. Navigate to your project in DebuggAI
2. Select Test Suites from the sidebar
3. Click Create Suite
4. Fill in the suite details:
   • Name: A descriptive name (e.g., "Checkout Flow Tests")
   • Description: Purpose and scope of the suite
   • Category: Feature area or test type
5. Click Save to create the suite

Adding Tests to a Suite

Once created, add tests to your suite:

1. Open the suite detail page
2. Click Add Tests
3. Select tests from your existing E2E tests
4. Confirm the selection

Tests can belong to multiple suites, allowing flexible organization.

Organizing Tests

Group by Feature

Organize tests around product features for focused testing:

Suite Name	Tests Included
Authentication	Login, logout, password reset, MFA
User Profile	Profile edit, avatar upload, settings
Payments	Card addition, checkout, refunds

Group by Test Type

Categorize by testing purpose:

• Smoke Tests: Quick validation of core functionality
• Regression Tests: Comprehensive coverage for releases
• Happy Path Tests: Standard user flows without edge cases
• Edge Case Tests: Boundary conditions and error handling

Using Tags

Apply tags for cross-cutting concerns:

• critical - Tests that must pass before deployment
• slow - Tests with longer execution times
• flaky - Tests requiring stability improvements
• mobile - Mobile-specific test scenarios

Running Suites

Manual Execution

1. Navigate to the suite
2. Click Run Suite
3. Select execution options:
   • Environment (staging, production)
   • Browser configuration
4. Monitor progress in real-time

Parallel Execution

Suites run tests in parallel based on your plan limits:

Plan	Parallel Tests
Free	1
Pro	3
Enterprise	10+

Parallel execution significantly reduces total suite runtime.

Viewing Results

After execution:

• Summary view: Overall pass/fail counts
• Test details: Individual test results with screenshots
• Timeline: Execution order and duration
• Logs: Detailed execution logs for debugging

Commit-Based Suites

Trigger on Commits

Configure suites to run automatically when code changes:

1. Open suite settings
2. Enable Commit Triggers
3. Select trigger conditions:
   • All commits to specific branches
   • Commits matching patterns
   • Pull request events

CI/CD Integration

Integrate with your CI/CD pipeline:

# Example GitHub Actions workflow
- name: Run DebuggAI Suite
  run: |
    curl -X POST https://api.debugg.ai/v1/suites/{suite_id}/run \
      -H "Authorization: Bearer ${{ secrets.DEBUGGAI_TOKEN }}" \
      -d '{"commit": "${{ github.sha }}"}'

Blocking Deployments

Configure suites as deployment gates:

• Set suite as required in project settings
• Failed suites block merge/deploy actions
• Receive notifications on suite completion

Suite Statistics

Pass Rate Tracking

Monitor suite health over time:

• Current pass rate: Percentage of passing tests
• Historical trends: Pass rate changes over time
• Per-test breakdown: Identify consistently failing tests

Execution Time Trends

Track performance metrics:

• Average duration: Suite execution time
• Trend analysis: Identify slowdowns
• Per-test timing: Find bottlenecks

Flakiness Detection

DebuggAI automatically identifies flaky tests:

• Tests that pass/fail inconsistently
• Flakiness score based on recent runs
• Recommendations for stabilization

Example: E-Commerce Test Organization

Here is an example of organizing test suites for an e-commerce application:

Suite Structure

E-Commerce Test Suites
|
|-- Smoke Tests (run on every commit)
|   |-- Homepage loads
|   |-- User can log in
|   |-- Product search works
|   |-- Cart adds item
|
|-- Authentication Suite
|   |-- User registration
|   |-- Login with email
|   |-- Social login (Google, Facebook)
|   |-- Password reset flow
|   |-- Session timeout handling
|
|-- Product Catalog Suite
|   |-- Category navigation
|   |-- Product search
|   |-- Filter and sort
|   |-- Product detail view
|   |-- Image gallery
|
|-- Checkout Suite
|   |-- Add to cart
|   |-- Update quantities
|   |-- Apply discount code
|   |-- Shipping selection
|   |-- Payment processing
|   |-- Order confirmation
|
|-- Regression Suite (run before releases)
|   |-- All tests from above suites
|   |-- Edge cases
|   |-- Error handling scenarios

Execution Strategy

Trigger	Suite	Purpose
Every commit	Smoke Tests	Quick validation
PR to main	Feature suites	Targeted testing
Pre-release	Regression Suite	Full coverage
Nightly	All suites	Comprehensive check

This organization ensures fast feedback on commits while maintaining thorough coverage for releases.

Best Practices

1. Keep smoke suites small: Under 10 tests for fast feedback
2. Review flaky tests weekly: Stabilize or remove unreliable tests
3. Update suites with features: Add tests as new features ship
4. Monitor execution times: Investigate sudden slowdowns
5. Use descriptive names: Make suite purpose clear at a glance