Testdriver find

<!-- Generated from find.mdx. DO NOT EDIT. -->

Overview

Locate UI elements on screen using AI-powered natural language descriptions. Returns an Element object that can be interacted with.

Syntax

const element = await testdriver.find(description)
const element = await testdriver.find(description, options)

Parameters

<ParamField path="description" type="string" required> Natural language description of the element to find </ParamField> <ParamField path="options" type="object | number"> Optional configuration for finding and caching <Expandable title="properties"> <ParamField path="cacheKey" type="string"> Custom cache key for storing element location. Use this to prevent cache pollution when using dynamic variables in prompts, or to share cache across tests. </ParamField>

<ParamField path="cacheThreshold" type="number" default={0.05}>
  Similarity threshold (0-1) for cache matching. Lower values require more similarity. Set to -1 to disable cache.
</ParamField>

<ParamField path="timeout" type="number" default={10000}>
  Maximum time in milliseconds to poll for the element. Retries every 5 seconds until found or timeout expires. Defaults to `10000` (10 seconds). Set to `0` to disable polling and make a single attempt.
</ParamField>

<ParamField path="confidence" type="number">
  Minimum confidence threshold (0-1). If the AI's confidence score for the found element is below this value, the find will be treated as a failure (`element.found()` returns `false`). Useful for ensuring high-quality matches in critical test steps.
</ParamField>

<ParamField path="type" type="string">
  Element type hint that wraps the description for better matching. Accepted values:
  - `"text"` — Wraps the prompt as `The text "..."`
  - `"image"` — Wraps the prompt as `The image "..."`
  - `"ui"` — Wraps the prompt as `The UI element "..."`
  - `"any"` — No wrapping, uses the description as-is (default behavior)
</ParamField>

<ParamField path="zoom" type="boolean" default={true}>
  Two-phase zoom mode for better precision in crowded UIs with many similar elements. Enabled by default.
</ParamField>

<ParamField path="ai" type="object">
  AI sampling configuration for this find call (overrides global `ai` config from constructor).
  
  <Expandable title="properties">
    <ParamField path="temperature" type="number">
      Controls randomness. `0` = deterministic. Default: `0` for find verification.
    </ParamField>
    
    <ParamField path="top" type="object">
      Sampling parameters
      
      <Expandable title="properties">
        <ParamField path="p" type="number">
          Top-P (nucleus sampling). Range: 0-1.
        </ParamField>
        
        <ParamField path="k" type="number">
          Top-K sampling. `1` = most deterministic.
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

</Expandable> </ParamField>

Returns

Promise<Element> - Element instance that has been automatically located

Examples

Basic Element Finding

// Find by role
const button = await testdriver.find('submit button');
const input = await testdriver.find('email input field');

// Find by text content
const link = await testdriver.find('Contact Us link');
const heading = await testdriver.find('Welcome heading');

// Find by visual appearance
const icon = await testdriver.find('red warning icon');
const image = await testdriver.find('company logo image');

Finding with Context

// Provide location context
const field = await testdriver.find('username input in the login form');
const button = await testdriver.find('delete button in the top right corner');

// Describe nearby elements
const input = await testdriver.find('input field below the email label');
const checkbox = await testdriver.find('checkbox next to "Remember me"');

// Describe visual position
const menu = await testdriver.find('hamburger menu icon in the top left');

Interacting with Found Elements

// Find and click
const submitBtn = await testdriver.find('submit button');
await submitBtn.click();

// Find and verify
const message = await testdriver.find('success message');
if (message.found()) {
  console.log('Success message appeared');
}

// Find and extract info
const price = await testdriver.find('product price');
console.log('Price location:', price.coordinates);
console.log('Price text:', price.text);

Element Object

The returned Element object provides:

Methods

• found() - Check if element was located
• click(action) - Click the element
• hover() - Hover over the element
• doubleClick() - Double-click the element
• rightClick() - Right-click the element
• find(newDescription) - Re-locate with optional new description

Properties

• coordinates - Element position {x, y, centerX, centerY}
• x, y - Top-left coordinates
• centerX, centerY - Center coordinates
• text - Text content (if available)
• screenshot - Base64 screenshot (if available)
• confidence - AI confidence score
• width, height - Element dimensions
• boundingBox - Complete bounding box

See Elements Reference for complete details.

JSON Serialization

Elements can be safely serialized using JSON.stringify() for logging and debugging. Circular references are automatically removed:

const element = await testdriver.find('login button');

// Safe to stringify - no circular reference errors
console.log(JSON.stringify(element, null, 2));

// Output includes useful debugging info:
// {
//   "description": "login button",
//   "coordinates": { "x": 100, "y": 200, "centerX": 150, "centerY": 225 },
//   "found": true,
//   "threshold": 0.01,
//   "x": 100,
//   "y": 200,
//   "cache": {
//     "hit": true,
//     "strategy": "pixel-diff",
//     "createdAt": "2025-12-09T10:30:00Z",
//     "diffPercent": 0.0023,
//     "imageUrl": "https://..."
//   },
//   "similarity": 0.98,
//   "confidence": 0.95,
//   "selector": "button#login",
//   "aiResponse": "Found the blue login button..."
// }

This is useful for:

• Debugging element detection issues
• Logging test execution details
• Sharing element information across processes
• Analyzing cache performance

Best Practices

<Check> **Be specific in descriptions**

More specific descriptions improve accuracy:

// ✅ Good
await testdriver.find('blue submit button below the email field');

// ❌ Too vague
await testdriver.find('button');

</Check> <Check> **Always check if found**

Verify elements were located before interacting:

const element = await testdriver.find('login button');
if (!element.found()) {
  throw new Error('Login button not found');
}
await element.click();

</Check> <Check> **Include visual or positional context**

// Include color
await testdriver.find('red error icon');

// Include position
await testdriver.find('search button in the top navigation bar');

// Include nearby text
await testdriver.find('checkbox next to "I agree to terms"');

</Check>

Confidence Threshold

Require a minimum AI confidence score for element matches. If the confidence is below the threshold, find() treats the result as not found:

// Require at least 90% confidence
const element = await testdriver.find('submit button', { confidence: 0.9 });

if (!element.found()) {
  // AI found something but wasn't confident enough
  throw new Error('Could not confidently locate submit button');
}

await element.click();

This is useful for:

• Critical test steps where an incorrect click could cause cascading failures
• Distinguishing between similar elements (e.g., multiple buttons)
• Failing fast when the UI has changed unexpectedly

// Combine with timeout for robust polling with confidence gate
const element = await testdriver.find('success notification', {
  confidence: 0.85,
  timeout: 15000,
});

<Tip> The `confidence` value is a float between 0 and 1 (e.g., `0.9` = 90%). The AI returns its confidence with each find result, which you can also read from `element.confidence` after a successful find. </Tip> ## Element Type

Use the type option to hint what kind of element you're looking for. This wraps your description into a more specific prompt for the AI, improving match accuracy — especially when users provide short or ambiguous descriptions.

// Find text on the page
const label = await testdriver.find('Sign In', { type: 'text' });
// AI prompt becomes: The text "Sign In"

// Find an image
const logo = await testdriver.find('company logo', { type: 'image' });
// AI prompt becomes: The image "company logo"

// Find a UI element (button, input, checkbox, etc.)
const btn = await testdriver.find('Submit', { type: 'ui' });
// AI prompt becomes: The UI element "Submit"

// No wrapping — same as omitting the option
const el = await testdriver.find('the blue submit button', { type: 'any' });

Type	Prompt sent to AI
"text"	The text "..."
"image"	The image "..."
"ui"	The UI element "..."
"any"	Original description (no wrapping)

<Tip> This is particularly useful for short descriptions like `"Submit"` or `"Login"` where the AI may not know whether to look for a button, a link, or visible text. Specifying `type` removes the ambiguity. </Tip> ## Polling for Dynamic Elements

By default, find() polls for up to 10 seconds (retrying every 5 seconds) until the element is found. You can customize this with the timeout option:

// Uses default 10s timeout - polls every 5 seconds
const element = await testdriver.find('login button');
await element.click();

// Custom timeout - wait up to 30 seconds
const element = await testdriver.find('login button', { timeout: 30000 });
await element.click();

// Disable polling - single attempt only
const element = await testdriver.find('login button', { timeout: 0 });

The timeout option:

• Defaults to 10000 (10 seconds)
• Retries finding the element every 5 seconds
• Stops when the element is found or the timeout expires
• Logs progress during polling
• Returns the element (check element.found() if not throwing on failure)
• Set to 0 to disable polling and make a single attempt

Zoom Mode

Zoom mode is enabled by default. It uses a two-phase approach for better precision when locating elements, especially in crowded UIs with many similar elements.

To disable zoom for a specific find call, pass zoom: false:

// Zoom is on by default — no option needed
const extensionsBtn = await testdriver.find('extensions puzzle icon in Chrome toolbar');
await extensionsBtn.click();

// Disable zoom for a specific call if needed
const largeButton = await testdriver.find('big hero button', { zoom: false });

How Zoom Mode Works

1. Phase 1: AI identifies the approximate location of the element
2. Phase 2: A 30% crop of the screen is created around that location
3. Phase 3: AI performs precise location on the zoomed/cropped image
4. Result: Coordinates are converted back to absolute screen position

This two-phase approach gives the AI a higher-resolution view of the target area, improving accuracy when multiple similar elements are close together.

<Tip> You may want to disable zoom with `zoom: false` when: - Targeting large, isolated elements where the extra precision isn't needed - You want to speed up find calls in simple UIs </Tip>

Cache Options

Control caching behavior to optimize performance, especially when using dynamic variables in prompts.

Custom Cache Key

Use cacheKey to prevent cache pollution when prompts contain variables:

// ❌ Without cacheKey - creates new cache entry for each email value
const email = '[email protected]';
await testdriver.find(`input for ${email}`); // Cache miss every time

// ✅ With cacheKey - reuses cache regardless of variable
const email = '[email protected]';
await testdriver.find(`input for ${email}`, {
  cacheKey: 'email-input'
});

// Also useful for dynamic IDs, names, or other changing data
const orderId = generateOrderId();
await testdriver.find(`order ${orderId} status`, {
  cacheKey: 'order-status'  // Same cache for all orders
});

Cache Threshold

Control how similar a cached result must be to reuse it:

// Default: 95% similarity required
await testdriver.find('submit button');

// Strict threshold - 99% similarity required
await testdriver.find('submit button', {
  cacheThreshold: 0.01
});

// Disable cache entirely for this call
await testdriver.find('submit button', {
  cacheThreshold: -1
});

// Combine cacheKey with threshold
await testdriver.find('submit button', {
  cacheKey: 'submit-btn',
  cacheThreshold: 0.01
});

<Tip> By default, TestDriver auto-generates a cache key from the SHA-256 hash of your test file. When you modify your test file, the hash changes automatically, invalidating stale cache entries. </Tip>

Manual Polling (Alternative)

If you need custom polling logic:

async function waitForElement(testdriver, description, timeout = 30000) {
  const startTime = Date.now();
  
  while (Date.now() - startTime < timeout) {
    const element = await testdriver.find(description);
    if (element.found()) return element;
    await new Promise(r => setTimeout(r, 1000));
  }
  
  throw new Error(`Element "${description}" not found after ${timeout}ms`);
}

// Usage
const button = await waitForElement(testdriver, 'submit button', 10000);
await button.click();

Use Cases

<AccordionGroup> <Accordion title="Form Fields"> ```javascript const emailField = await testdriver.find('email input field'); await emailField.click(); await testdriver.type('[email protected]');

const passwordField = await testdriver.find('password input');
await passwordField.click();
await testdriver.type('MyP@ssw0rd');
```

</Accordion> <Accordion title="Buttons and Links"> ```javascript const submitBtn = await testdriver.find('submit button'); await submitBtn.click();

const cancelLink = await testdriver.find('cancel link');
await cancelLink.click();

const menuIcon = await testdriver.find('hamburger menu icon');
await menuIcon.click();
```

</Accordion> <Accordion title="Dynamic Content"> ```javascript // Wait for loading to complete let content; for (let i = 0; i < 30; i++) { content = await testdriver.find('results table'); if (content.found()) break; await new Promise(r => setTimeout(r, 1000)); }

// Interact with loaded content
const firstRow = await testdriver.find('first row in the results table');
await firstRow.click();
```

</Accordion> <Accordion title="Complex UI Elements"> ```javascript // Modals and dialogs const modal = await testdriver.find('confirmation dialog'); if (modal.found()) { const confirmBtn = await testdriver.find('confirm button in the dialog'); await confirmBtn.click(); }

// Dropdown menus
const dropdown = await testdriver.find('country dropdown');
await dropdown.click();

const option = await testdriver.find('United States option');
await option.click();
```

</Accordion> </AccordionGroup>

Complete Example

import { beforeAll, afterAll, describe, it, expect } from 'vitest';
import TestDriver from 'testdriverai';

describe('Element Finding', () => {
  let testdriver;

  beforeAll(async () => {
    client = new TestDriver(process.env.TD_API_KEY);
    await testdriver.auth();
    await testdriver.connect();
  });

  afterAll(async () => {
    await testdriver.disconnect();
  });

  it('should find and interact with elements', async () => {
    await testdriver.focusApplication('Google Chrome');
    
    // Find login form elements
    const usernameField = await testdriver.find('username input field');
    expect(usernameField.found()).toBe(true);
    
    await usernameField.click();
    await testdriver.type('testuser');
    
    // Find with context
    const passwordField = await testdriver.find('password input below username');
    await passwordField.click();
    await testdriver.type('password123');
    
    // Find button
    const submitBtn = await testdriver.find('green submit button');
    expect(submitBtn.found()).toBe(true);
    
    console.log('Button location:', submitBtn.centerX, submitBtn.centerY);
    
    await submitBtn.click();
    
    // Wait for success message
    let successMsg;
    for (let i = 0; i < 10; i++) {
      successMsg = await testdriver.find('success notification');
      if (successMsg.found()) break;
      await new Promise(r => setTimeout(r, 1000));
    }
    
    expect(successMsg.found()).toBe(true);
  });
});

Related Methods

• click() - Click on found elements
• hover() - Hover over elements
• assert() - Verify element states
• Elements Reference - Complete Element API

---

findAll()

Locate all elements matching a description, rather than just one.

Syntax

const elements = await testdriver.findAll(description, options)

Parameters

<ParamField path="description" type="string" required> Natural language description of elements to find </ParamField> <ParamField path="options" type="object | number"> Optional cache options (same as `find()`) <Expandable title="properties"> <ParamField path="cacheKey" type="string"> Cache key for storing element location </ParamField>

<ParamField path="cacheThreshold" type="number" default={-1}>
  Similarity threshold (0-1) for cache matching. Set to -1 to disable cache.
</ParamField>

</Expandable> </ParamField>

Returns

Promise<Element[]> - Array of Element instances

Examples

Basic Usage

// Find all matching elements
const buttons = await testdriver.findAll('button');
console.log(`Found ${buttons.length} buttons`);

// Interact with specific element
if (buttons.length > 0) {
  await buttons[0].click(); // Click first button
}

// Iterate over all
for (const button of buttons) {
  console.log(`Button at (${button.x}, ${button.y})`);
}

Finding Multiple Items

// Find all list items
const items = await testdriver.findAll('list item');

// Find specific item by index
const thirdItem = items[2];
await thirdItem.click();

// Check all items
for (let i = 0; i < items.length; i++) {
  console.log(`Item ${i + 1}: ${items[i].text || 'No text'}`);
}

With Caching

// Cache element locations for faster subsequent runs
const menuItems = await testdriver.findAll('menu item', {
  cacheKey: 'main-menu-items'
});

// First run: ~2-3 seconds (AI call)
// Subsequent runs: ~100ms (cache hit)

Empty Results

// Returns empty array if nothing found (doesn't throw error)
const errors = await testdriver.findAll('error message');

if (errors.length === 0) {
  console.log('No errors found - test passed!');
} else {
  console.log(`Found ${errors.length} errors`);
}

Differences from find()

Feature	find()	findAll()
Return type	Single Element	Array of Element[]
If nothing found	Throws ElementNotFoundError	Returns empty array []
Chainable	✅ Yes: await find('button').click()	❌ No (returns array)
Use case	One specific element	Multiple similar elements
Cache support	✅ Yes	✅ Yes

Use Cases

<AccordionGroup> <Accordion title="Table Rows"> ```javascript // Find all rows in a table const rows = await testdriver.findAll('table row');

// Click every row
for (const row of rows) {
  await row.click();
  await new Promise(r => setTimeout(r, 500)); // Wait between clicks
}

// Or click specific row
await rows[2].click(); // Click third row
```

</Accordion> <Accordion title="Checkboxes/Radio Buttons"> ```javascript // Find all checkboxes const checkboxes = await testdriver.findAll('checkbox');

// Check all boxes
for (const checkbox of checkboxes) {
  await checkbox.click();
}

// Or select first unchecked
const unchecked = checkboxes[0];
await unchecked.click();
```

</Accordion> <Accordion title="Navigation Links"> ```javascript // Find all navigation links const navLinks = await testdriver.findAll('navigation link');

// Validate all are present
expect(navLinks.length).toBeGreaterThan(0);

// Click specific link by text
const homeLink = navLinks.find(link => 
  link.text?.toLowerCase().includes('home')
);

if (homeLink) {
  await homeLink.click();
}
```

</Accordion> <Accordion title="Conditional Interactions"> ```javascript // Check if any error messages exist const errors = await testdriver.findAll('error message');

if (errors.length > 0) {
  console.log(`Found ${errors.length} validation errors`);
  
  // Log each error location
  errors.forEach((error, i) => {
    console.log(`Error ${i + 1} at (${error.x}, ${error.y})`);
  });
} else {
  console.log('Form validation passed!');
}
```

</Accordion> </AccordionGroup>

Complete Example

import { test, expect } from 'vitest';
import { chrome } from 'testdriverai/presets';

test('select multiple items from list', async (context) => {
  const { testdriver } = await chrome(context, {
    url: 'https://example.com/products'
  });
  
  // Find all product cards
  const products = await testdriver.findAll('product card');
  
  expect(products.length).toBeGreaterThan(0);
  console.log(`Found ${products.length} products`);
  
  // Click first 3 products
  const productsToSelect = Math.min(3, products.length);
  
  for (let i = 0; i < productsToSelect; i++) {
    await products[i].click();
    console.log(`Selected product ${i + 1}`);
    await new Promise(r => setTimeout(r, 500)); // Brief pause
  }
  
  // Verify selections
  const selectedBadges = await testdriver.findAll('selected badge');
  expect(selectedBadges.length).toBe(productsToSelect);
});

Best Practices

<Check> **Handle empty arrays gracefully**

// ✅ Good - check length first
const items = await testdriver.findAll('list item');
if (items.length > 0) {
  await items[0].click();
}

// ❌ Bad - may throw error
const items = await testdriver.findAll('list item');
await items[0].click(); // Error if array is empty!

</Check> <Check> **Use find() for single elements**

// ✅ Use find() when you need exactly one
const submitBtn = await testdriver.find('submit button');
await submitBtn.click();

// ❌ Unnecessary - findAll() returns array
const buttons = await testdriver.findAll('submit button');
await buttons[0].click(); // Extra array handling

</Check> <Check> **Cache for performance**

// First run - slow (AI call)
const items = await testdriver.findAll('menu item', {
  cacheKey: 'menu-items'
});

// Subsequent runs - fast (cache hit)
// ~10-20x faster than without cache

</Check>