Browser

AgentInBrowser Skill

Overview

AgentInBrowser is a REST API service that provides remote control of a Chrome browser. It wraps Selenium WebDriver and exposes HTTP endpoints for browser operations (find elements, click, type, screenshot, execute JS, etc.).

Project Location: D:\Develop\AgentInBrowser

When to Use

• Building browser automation scripts (auto-login, course watching, form filling, etc.)
• Need to control browser via HTTP interface instead of using Selenium directly
• Need to run browser tasks in background without detection

Quick Reference

Start Server

# In project directory (using uv)
cd D:\Develop\AgentInBrowser
source .venv/bin/activate  # Linux/macOS
# or .venv\Scripts\activate  # Windows
aib

# Or use directly after global install
aib

Server listens on: http://127.0.0.1:5000

Check Server Status

import requests
resp = requests.get("http://127.0.0.1:5000/status")
# → {"status": "running", "browser_active": false, "current_url": null}

Stop Server

requests.post("http://127.0.0.1:5000/shutdown")  # Close browser + exit server
requests.post("http://127.0.0.1:5000/quit")      # Close browser only, server stays alive

API Endpoints

Endpoint	Method	Description
/status	GET	Get server status
/env	GET	Get server environment info (project dir, venv path, etc.)
/init	POST	Initialize browser and navigate to URL
/execute	POST	Execute a Selenium command
/quit	POST	Close browser (server continues)
/shutdown	POST	Close browser and stop server

---

Agent Usage Workflow (Important!)

Since the server uses uv for dependency management, the virtual environment .venv is inside the project directory. Agent's bash tool runs in a local directory, so you need to get server environment info first before activating venv.

Step 1: Query Server Environment

import requests

env = requests.get("http://127.0.0.1:5000/env").json()
print(env)
# → {
#     "project_dir": "D:\\Develop\\AgentInBrowser",
#     "venv_path": "D:\\Develop\\AgentInBrowser\\.venv",
#     "python_executable": "D:\\Develop\\AgentInBrowser\\.venv\\Scripts\\python.exe",
#     "platform": "Windows",
#     "is_windows": true,
#     "activate_cmd": "D:\\Develop\\AgentInBrowser\\.venv\\Scripts\\activate.bat",
#     "activate_ps": "D:\\Develop\\AgentInBrowser\\.venv\\Scripts\\Activate.ps1"
#   }

Step 2: Activate venv (Execute Before Using CLI)

In Agent's bash command, activate venv first before using aib-client:

Windows PowerShell:

& "D:\Develop\AgentInBrowser\.venv\Scripts\Activate.ps1"
aib-client init https://example.com

Windows cmd:

D:\Develop\AgentInBrowser\.venv\Scripts\activate.bat && aib-client init https://example.com

Linux/macOS:

source /path/to/AgentInBrowser/.venv/bin/activate
aib-client init https://example.com

Step 3: Use CLI Client

# Now aib-client is available
aib-client init https://example.com
aib-client find button
aib-client click 0
aib-client quit

---

Response Format

All endpoints return JSON with unified format:

// Success
{"success": true, "data": {...}}

// Failure
{"success": false, "error": "error message"}

Commands via /execute

Request body format:

{"cmd": "command_name", "params": {"key": "value"}}

Finding Elements

find — Find elements by CSS selector

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "find",
    "params": {"selector": "button.btn-primary"}
})
# → {"success": true, "data": {
#     "count": 3,
#     "elements": [
#       {"index": 0, "tag": "button", "text": "Login", "class": "btn-primary", "id": ""},
#       ...
#     ]
#   }}

Note: find results are cached, subsequent click/send_keys/get_html reference by index.

inputs — Get all input elements

requests.post("http://127.0.0.1:5000/execute", json={"cmd": "inputs"})
# → {"success": true, "data": {
#     "count": 2,
#     "inputs": [
#       {"index": 0, "type": "text", "name": "username", "placeholder": "Username", ...},
#       ...
#     ]
#   }}

buttons — Get all button elements

requests.post("http://127.0.0.1:5000/execute", json={"cmd": "buttons"})
# → {"success": true, "data": {
#     "count": 1,
#     "buttons": [
#       {"index": 0, "text": "Submit", "class": "submit-btn", "id": ""}
#     ]
#   }}

Element Interaction

click — Click element (by index from find/inputs/buttons)

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "click",
    "params": {"index": 0}
})
# → {"success": true, "data": {"current_url": "https://..."}}

Auto-scrolls to element visible area, falls back to JS click on normal click failure. Built-in random delays mimic human behavior.

send_keys — Type text into element

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "send_keys",
    "params": {"index": 0, "text": "hello"}
})
# → {"success": true, "data": {"success": true}}

Calls clear() first before typing, built-in random delays.

Page Information

page_info — Get current page information

requests.post("http://127.0.0.1:5000/execute", json={"cmd": "page_info"})
# → {"success": true, "data": {"url": "...", "title": "...", "source_length": 12345}}

get_text — Get page text

requests.post("http://127.0.0.1:5000/execute", json={"cmd": "get_text"})
# → {"success": true, "data": {"text": "page body text (first 2000 chars)"}}

get_html — Get element innerHTML

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "get_html",
    "params": {"index": 0}
})
# → {"success": true, "data": {"html": "...(max 5000 chars)"}}

Utility Commands

screenshot — Take screenshot

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "screenshot",
    "params": {"filename": "my_screenshot.png"}  # optional, defaults to timestamp filename
})
# → {"success": true, "data": {"filename": "my_screenshot.png"}}

execute_script — Execute JavaScript

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "execute_script",
    "params": {"script": "return document.title"}
})
# → {"success": true, "data": {"result": "page title"}}

sleep — Wait (also saves page HTML)

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "sleep",
    "params": {"seconds": 5}
})
# → {"success": true, "data": {"slept": 5}}

Window Management

switch_window — Switch window

requests.post("http://127.0.0.1:5000/execute", json={
    "cmd": "switch_window",
    "params": {"index": -1}  # -1 = newest window, 0/1/2 = specific index
})
# → {"success": true, "data": {"current_url": "...", "title": "...", "window_count": 2}}

close_window — Close current window (auto-switch back to main window)

requests.post("http://127.0.0.1:5000/execute", json={"cmd": "close_window"})
# → {"success": true, "data": {"current_url": "...", "title": "..."}}

Typical Workflow

1. Initialize browser and open page

import requests

SERVER = "http://127.0.0.1:5000"

# Initialize browser and navigate to URL
resp = requests.post(f"{SERVER}/init", json={"url": "https://example.com"})
print(resp.json())

2. Find and interact

# Find all links
resp = requests.post(f"{SERVER}/execute", json={
    "cmd": "find", "params": {"selector": "a"}
})
elements = resp.json()["data"]["elements"]

# Click first one
requests.post(f"{SERVER}/execute", json={
    "cmd": "click", "params": {"index": 0}
})

# Wait for page load
requests.post(f"{SERVER}/execute", json={
    "cmd": "sleep", "params": {"seconds": 3}
})

3. Fill form

# Get input fields
requests.post(f"{SERVER}/execute", json={"cmd": "inputs"})

# Fill them
requests.post(f"{SERVER}/execute", json={
    "cmd": "send_keys", "params": {"index": 0, "text": "username"}
})
requests.post(f"{SERVER}/execute", json={
    "cmd": "send_keys", "params": {"index": 1, "text": "password"}
})

# Get buttons and click
requests.post(f"{SERVER}/execute", json={"cmd": "buttons"})
requests.post(f"{SERVER}/execute", json={
    "cmd": "click", "params": {"index": 0}
})

4. Handle new window (e.g., popup login)

# Switch to newest window
requests.post(f"{SERVER}/execute", json={
    "cmd": "switch_window", "params": {"index": -1}
})

# ... interact in new window ...

# Close new window, return to main window
requests.post(f"{SERVER}/execute", json={"cmd": "close_window"})

5. Cleanup

# Close browser, server continues running
requests.post(f"{SERVER}/quit")

# Or completely shutdown
requests.post(f"{SERVER}/shutdown")

Important Notes

1. Element indices are ephemeral — Each find/inputs/buttons call resets internal cache, subsequent click/send_keys/get_html reference the most recent find result
2. Server must be started first — Client scripts should check /status before sending commands
3. Anti-detection is built-in — Server automatically injects anti-suspend scripts and anti-automation detection, no client-side work needed
4. Random delays included — click and send_keys include random delays to mimic human behavior
5. Auto-logging — All operations logged to server_log.txt in working directory
6. Click auto-fallback — Falls back to JavaScript click when normal click fails