Skip to main content

Understanding Results

Interpret test results to debug issues and improve your app.

Test Results Panel​

Access via View → Test Results or Cmd+Shift+Y → "Test Results" tab.

Result Components​

Status Icons

  • ✅ Pass: Test completed successfully
  • ❌ Fail: Test encountered errors or unexpected behavior
  • ⏳ Running: Test in progress
  • ⚠️ Warning: Test passed but with concerns

Execution Details

  • Duration: How long the test took
  • Steps: AI actions taken (click, type, wait, verify)
  • Screenshot: Final page state when test completed
  • Logs: Detailed AI reasoning and decisions

Reading Test Failures​

Common Failure Types​

Element Not Found

❌ Could not locate login button

Possible causes: Element selector changed, page didn't load, slow network

Unexpected Page State

❌ Expected dashboard page, found error page instead

Possible causes: Authentication failed, redirect issue, server error

Timeout Errors

❌ Page load exceeded 30 second timeout

Possible causes: Slow server, network issues, infinite loading

Debugging with Screenshots​

Screenshots show exactly what AI "saw" when test failed:

  1. Compare with expected: Does page match your expectations?
  2. Check for errors: Look for error messages, 404 pages, loading spinners
  3. Verify elements: Are buttons, forms, links visible and clickable?

Performance Insights​

Metrics Provided​

  • Page load time: Time to interactive
  • Element discovery: How quickly AI found key elements
  • User flow duration: Total time for multi-step processes

Performance Warnings​

  • ⚠️ Slow page loads (>5 seconds): May indicate backend issues
  • ⚠️ Element delays (>3 seconds to find): Possible UI/UX problems
  • ⚠️ Long form submissions (>10 seconds): Database or validation bottlenecks

AI Decision Insights​

Understanding the Logs​

Element Selection Logic

✓ Found 3 buttons on page
✓ Selected "Sign In" based on test context
✓ Button clicked successfully

User Flow Reasoning

✓ Detected login form with email/password fields
✓ Filled email: test@example.com  
✓ Generated secure test password
✓ Form submitted, awaiting redirect

Error Recovery

⚠️ First attempt to find "Submit" failed
✓ Retrying with alternative selectors
✓ Found button with text "Create Account"

Visual Testing Features​

Screenshot Analysis​

  • Before/after comparisons for UI changes
  • Element highlighting showing what AI interacted with
  • Mobile vs desktop rendering differences
  • Error state captures for debugging

Video Recordings​

When available, videos show:

  • Complete user interaction flow
  • Timing of AI decisions
  • Real-time page responses
  • Exact failure points

Improving Test Quality​

Using Results to Refine Tests​

If test is too broad:

❌ "Test the app works"
✅ "Test user can log in and see dashboard"

If test is too specific:

❌ "Click the blue button in the top right corner"
✅ "Test user can access their profile settings"

If test assumes too much:

❌ "Test checkout process" (assumes items in cart)
✅ "Test complete purchase flow starting from product page"

Iterating Based on Failures​

  1. Read failure reason carefully
  2. Examine screenshot for clues
  3. Adjust test description based on findings
  4. Re-run to validate improvement

Team Collaboration​

Sharing Results​

  • Copy test URL from results panel
  • Export screenshots for bug reports
  • Share execution logs with developers
  • Create issues directly from failed tests

Result History​

  • View previous runs of same test
  • Compare results across code changes
  • Track improvement/regression trends
  • Identify flaky vs consistent failures

Next: Explore different Workflows to match your development process.