Browser testing with screenshots

Browser Testing with Screenshots

Overview

Automate Chrome browser testing with visual verification using browser-tools. Connect to Chrome DevTools Protocol for navigation, interaction, and screenshot capture to confirm application functionality.

Prerequisites

REQUIRED: Install agent-tools from https://github.com/badlogic/agent-tools

# Clone and install agent-tools
git clone https://github.com/badlogic/agent-tools.git
cd agent-tools
# Follow installation instructions in the repository
# Ensure all executables (browser-start.js, browser-nav.js, etc.) are in your PATH

Verify installation:

# Check that browser tools are available
which browser-start.js
which browser-nav.js
which browser-screenshot.js

All browser-* commands referenced in this skill come from the agent-tools repository and must be properly installed and accessible in your system PATH.

When to Use

Use this skill when:

• Testing web application UI flows
• Verifying visual changes or layouts
• Automating repetitive browser interactions
• Documenting application behavior with screenshots
• Testing localhost applications during development
• Need to interact with elements that require human-like selection

Don't use for:

• API testing (use direct HTTP calls)
• Headless testing where visuals don't matter
• Simple page content validation (use curl/wget)

Quick Reference

Task	Command	Purpose
Start browser	browser-start.js	Launch Chrome with debugging
Navigate	browser-nav.js http://localhost:5172/dashboard	Go to specific URL
Take screenshot	browser-screenshot.js	Capture current viewport
Pick elements	browser-pick.js "Select the login button"	Interactive element selection
Run JavaScript	browser-eval.js 'document.title'	Execute code in page context
Extract content	browser-content.js	Get readable page content
View cookies	browser-cookies.js	List session cookies

Setup and Basic Workflow

1. Start Chrome with Remote Debugging

# Launch Chrome with debugging enabled (preserves user profile)
browser-start.js

# Or start fresh (no cookies, clean state)
browser-start.js --fresh

Expected Result: Chrome opens on port 9222 with DevTools Protocol enabled

2. Navigate to Application

# Go to your application starting point
browser-nav.js http://localhost:5172/dashboard

Verify: Browser navigates to dashboard page

3. Capture Baseline Screenshot

# Take initial screenshot to confirm page loaded
browser-screenshot.js

Output: Returns path to screenshot file (e.g., screenshot_20231203_141532.png)

Testing Workflow with Screenshots

Complete Test Scenario Example

#!/bin/bash
# Test login and dashboard functionality

echo "🚀 Starting browser test..."

# 1. Launch browser
browser-start.js --fresh

# 2. Navigate to login page
browser-nav.js http://localhost:5172/login
sleep 2

# 3. Take screenshot of login page
LOGIN_SHOT=$(browser-screenshot.js)
echo "📸 Login page: $LOGIN_SHOT"

# 4. Fill login form (interactive element picking)
browser-pick.js "Click the username field"
browser-eval.js 'document.activeElement.value = "testuser"'

browser-pick.js "Click the password field"
browser-eval.js 'document.activeElement.value = "password123"'

# 5. Screenshot filled form
FORM_SHOT=$(browser-screenshot.js)
echo "📸 Filled form: $FORM_SHOT"

# 6. Submit form
browser-pick.js "Click the login button"
sleep 3

# 7. Verify dashboard loaded
browser-nav.js http://localhost:5172/dashboard
DASHBOARD_SHOT=$(browser-screenshot.js)
echo "📸 Dashboard: $DASHBOARD_SHOT"

# 8. Verify specific dashboard elements
browser-pick.js "Select the navigation menu"
browser-eval.js 'console.log("Navigation found:", !!document.querySelector(".nav"))'

echo "✅ Test complete. Screenshots saved."

Element Interaction Pattern

# Interactive element selection (best for dynamic content)
browser-pick.js "Select the submit button"
# User clicks element in browser → returns CSS selector

# Use returned selector for automation
SELECTOR=$(browser-pick.js "Select the submit button" | grep "selector:")
browser-eval.js "document.querySelector('$SELECTOR').click()"

# Take screenshot to verify action
browser-screenshot.js

Advanced Usage

JavaScript Evaluation for Complex Interactions

# Check if element exists before interaction
browser-eval.js 'document.querySelector("#login-form") !== null'

# Wait for dynamic content
browser-eval.js '
  new Promise(resolve => {
    const check = () => {
      if (document.querySelector(".loaded")) resolve(true);
      else setTimeout(check, 100);
    };
    check();
  })
'

# Extract form data
browser-eval.js 'JSON.stringify(Object.fromEntries(new FormData(document.querySelector("form"))))'

Screenshot with Timing

# Navigate and wait before screenshot
browser-nav.js http://localhost:5172/slow-page
sleep 5  # Wait for animations/loading
browser-screenshot.js

Content Extraction for Verification

# Get page title
PAGE_TITLE=$(browser-eval.js 'document.title')
echo "Current page: $PAGE_TITLE"

# Extract readable content
browser-content.js > page_content.md

# Check for specific text
browser-eval.js 'document.body.textContent.includes("Welcome to Dashboard")'

Common Mistakes

Mistake	Problem	Solution
No sleep after navigation	Screenshots of loading page	Add sleep 2-5 after nav
Hardcoded selectors	Breaks when UI changes	Use browser-pick.js for selection
Missing Chrome setup	"Connection refused" errors	Run browser-start.js first
Wrong localhost port	Navigation fails	Verify application is running on correct port
Screenshot timing	Captures before content loads	Wait for page load or specific elements
Not preserving state	Login lost between commands	Use default profile, not --fresh

Error Handling

# Check if Chrome is running
if ! browser-eval.js 'true' 2>/dev/null; then
  echo "❌ Chrome not connected. Running browser-start.js..."
  browser-start.js
fi

# Verify navigation succeeded
if browser-eval.js 'location.href.includes("dashboard")'; then
  echo "✅ Navigation successful"
else
  echo "❌ Navigation failed"
  exit 1
fi

File Output Patterns

• Screenshots: screenshot_YYYYMMDD_HHMMSS.png in current directory
• Content: Markdown format via stdout from browser-content.js
• Selectors: CSS selectors from browser-pick.js interaction
• JavaScript results: JSON or string values from browser-eval.js

Integration with Testing Frameworks

# Create test evidence directory
mkdir -p test-results/$(date +%Y%m%d_%H%M%S)
cd test-results/$(date +%Y%m%d_%H%M%S)

# Run tests with organized screenshots
browser-start.js
for page in login dashboard profile; do
  browser-nav.js "http://localhost:5172/$page"
  sleep 2
  screenshot=$(browser-screenshot.js)
  mv "$screenshot" "${page}_page.png"
  echo "✅ $page page tested"
done

Real-World Impact

Benefits:

• Visual verification: Screenshots provide immediate feedback on UI state
• Interactive debugging: Element picker works with dynamic/complex selectors
• State preservation: Maintains login sessions between commands
• Evidence collection: Automated screenshot capture for test documentation
• Development workflow: Quick verification of localhost changes

Results:

• Faster UI testing iteration (visual confirmation vs manual checking)
• Reliable element selection (human picks, automation uses)
• Test documentation with visual proof
• Catches visual regressions immediately

---

Key principle: Combine automated navigation with human element selection for robust, maintainable browser testing.