agentsclimarketplace

Agent device

Skill bidah/react-native-hifi/skills/agent-device

Automates interactions for iOS simulators/devices and Android emulators/devices. Use when navigating apps, taking snapshots/screenshots, tapping, typing, scrolling, or extracting UI info on mobile targets.From its SKILL.md

Install
npx -y skills add bidah/react-native-hifi --skill agent-device

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

10.1 KB, as published. Nobody here has run it

Mobile Automation with agent-device

For exploration, use snapshot refs. For deterministic replay, use selectors. For structured exploratory QA bug hunts and reporting, use ../dogfood/SKILL.md.

Start Here (Read This First)

Use this skill as a router, not a full manual.

  1. Pick one mode:
    • Normal interaction flow
    • Debug/crash flow
    • Replay maintenance flow
  2. Run one canonical flow below.
  3. Open references only if blocked.

Decision Map

  • No target context yet: devices -> pick target -> open.
  • Normal UI task: open -> snapshot -i -> press/fill -> diff snapshot -i -> close
  • Debug/crash: open <app> -> logs clear --restart -> reproduce -> network dump -> logs path -> targeted grep
  • Replay drift: replay -u <path> -> verify updated selectors
  • Remote multi-tenant run: allocate lease -> point client at remote daemon base URL -> run commands with tenant isolation flags -> heartbeat/release lease
  • Device-scope isolation run: set iOS simulator set / Android allowlist -> run selectors within scope only

Target Selection Rules

  • iOS local QA: use simulators unless the task explicitly requires a physical device.
  • iOS local QA in mixed simulator/device environments: run ensure-simulator first and pass --device, --udid, or --ios-simulator-device-set on later commands.
  • Android local QA: use install or reinstall for .apk/.aab files, then relaunch by installed package name.
  • Android React Native + Metro flows: prefer open <package> --remote-config <path> --relaunch.
  • In mixed-device environments, always pin the exact target with --serial, --device, --udid, or an isolation scope.
  • For session-bound automation runs, prefer a pre-bound session/platform instead of repeating selectors on every command: set AGENT_DEVICE_SESSION, set AGENT_DEVICE_PLATFORM, and the daemon will enforce the shared lock policy across CLI, typed client, and RPC entry points.
  • Use --session-lock reject|strip (or AGENT_DEVICE_SESSION_LOCK) only when you need to override the default reject behavior. Lock mode applies to nested batch steps too.

Canonical Flows

1) Normal Interaction Flow

agent-device open Settings --platform ios
agent-device snapshot -i
agent-device press @e3
agent-device diff snapshot -i
agent-device fill @e5 "test"
agent-device close

1a) Local iOS Simulator QA Flow

agent-device ensure-simulator --platform ios --device "iPhone 16" --boot
agent-device open MyApp --platform ios --device "iPhone 16" --session qa-ios --relaunch
agent-device snapshot -i
agent-device press @e3
agent-device close

Use this when a physical iPhone is also connected and you want deterministic simulator-only automation.

1b) Android React Native + Metro QA Flow

agent-device reinstall MyApp /path/to/app-debug.apk --platform android --serial emulator-5554
agent-device open com.example.myapp --remote-config ./agent-device.remote.json --relaunch
agent-device snapshot -i
agent-device close

Do not use open <apk|aab> --relaunch on Android. Install/reinstall binaries first, then relaunch by package.

1c) Session-Bound Automation Flow

export AGENT_DEVICE_SESSION=qa-ios
export AGENT_DEVICE_PLATFORM=ios
export AGENT_DEVICE_SESSION_LOCK=strip

agent-device open MyApp --relaunch
agent-device snapshot -i
agent-device batch --steps-file /tmp/qa-steps.json --json
agent-device close

Use this for orchestrators that must preserve one bound session/device across many plain CLI calls without a wrapper script.

1d) Android Emulator Session-Bound Flow

export AGENT_DEVICE_SESSION=qa-android
export AGENT_DEVICE_PLATFORM=android

agent-device reinstall MyApp /path/to/app-debug.apk --serial emulator-5554
agent-device --session-lock reject open com.example.myapp --relaunch
agent-device snapshot -i
agent-device close --shutdown

2) Debug/Crash Flow

agent-device open MyApp --platform ios
agent-device logs clear --restart
agent-device network dump 25
agent-device logs path

Logging is off by default. Enable only for debugging windows. logs clear --restart requires an active app session (open <app> first).

3) Replay Maintenance Flow

agent-device replay -u ./session.ad

4) Remote Tenant Lease Flow (HTTP JSON-RPC)

export AGENT_DEVICE_DAEMON_BASE_URL=http://mac-host.example:4310
export AGENT_DEVICE_DAEMON_AUTH_TOKEN=<token>

# Allocate lease
curl -sS "${AGENT_DEVICE_DAEMON_BASE_URL}/rpc" \
  -H "content-type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{"jsonrpc":"2.0","id":"alloc-1","method":"agent_device.lease.allocate","params":{"runId":"run-123","tenantId":"acme","ttlMs":60000}}'

# Use lease in tenant-isolated command execution
agent-device \
  --tenant acme \
  --session-isolation tenant \
  --run-id run-123 \
  --lease-id <lease-id> \
  session list --json

# Heartbeat and release
curl -sS "${AGENT_DEVICE_DAEMON_BASE_URL}/rpc" \
  -H "content-type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{"jsonrpc":"2.0","id":"hb-1","method":"agent_device.lease.heartbeat","params":{"leaseId":"<lease-id>","ttlMs":60000}}'
curl -sS "${AGENT_DEVICE_DAEMON_BASE_URL}/rpc" \
  -H "content-type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{"jsonrpc":"2.0","id":"rel-1","method":"agent_device.lease.release","params":{"leaseId":"<lease-id>"}}'

Command Skeleton (Minimal)

Session and navigation

agent-device devices
agent-device devices --platform ios --ios-simulator-device-set /tmp/tenant-a/simulators
agent-device devices --platform android --android-device-allowlist emulator-5554,device-1234
agent-device ensure-simulator --device "iPhone 16" --ios-simulator-device-set /tmp/tenant-a/simulators
agent-device ensure-simulator --device "iPhone 16" --runtime com.apple.CoreSimulator.SimRuntime.iOS-18-4 --ios-simulator-device-set /tmp/tenant-a/simulators --boot
agent-device open [app|url] [url]
agent-device open [app] --relaunch
agent-device close [app]
agent-device install <app> <path-to-binary>
agent-device install-from-source <url> [--header "name:value"]
agent-device reinstall <app> <path-to-binary>
agent-device session list

Use boot only as fallback when open cannot find/connect to a ready target.

Snapshot and targeting

agent-device snapshot -i
agent-device diff snapshot -i
agent-device find "Sign In" click
agent-device press @e1
agent-device fill @e2 "text"
agent-device is visible 'id="anchor"'

press is canonical tap command; click is an alias.

Utilities

agent-device appstate
agent-device clipboard read
agent-device clipboard write "token"
agent-device keyboard status
agent-device keyboard dismiss
agent-device perf --json
agent-device network dump [limit] [summary|headers|body|all]
agent-device push <bundle|package> <payload.json|inline-json>
agent-device trigger-app-event screenshot_taken '{"source":"qa"}'
agent-device get text @e1
agent-device screenshot out.png
agent-device settings permission grant notifications
agent-device settings permission reset camera
agent-device trace start
agent-device trace stop ./trace.log

Batch (when sequence is already known)

agent-device batch --steps-file /tmp/batch-steps.json --json

Performance Check

Guardrails (High Value Only)

  • Re-snapshot after UI mutations (navigation/modal/list changes).
  • Prefer snapshot -i; scope/depth only when needed.
  • Use refs for discovery, selectors for replay/assertions.
  • find "<query>" click --json returns { ref, locator, query, x, y } — all derived from the matched snapshot node.
  • Use fill for clear-then-type semantics; use type for focused append typing.
  • Use install for in-place app upgrades, and reinstall for deterministic fresh-state runs.
  • App binary format support: Android .apk/.aab, iOS .app/.ipa.
  • Android .aab requires bundletool in PATH.
  • network dump is best-effort and parses HTTP(s) entries from the session app log file.
  • For AndroidTV/tvOS selection, always pair --target with --platform.
  • Permission settings are app-scoped and require an active session app: settings permission <grant|deny|reset> <camera|microphone|photos|contacts|notifications> [full|limited]
  • iOS simulator permission alerts: use alert wait then alert accept/dismiss — retries internally for up to 2s.

Common Failure Patterns

  • Failed to access Android app sandbox for /path/app-debug.apk: Android relaunch flow received an APK path instead of package name. Use reinstall first, then open <package> --relaunch.
  • Failed to terminate iOS app: flow may have selected a physical iPhone. Re-run with ensure-simulator, then pin with --device or --udid.

Common Mistakes

  • Mixing debug flow into normal runs (keep logs off unless debugging).
  • Continuing to use stale refs after screen transitions.
  • Using URL opens with Android --activity (unsupported combination).
  • Treating boot as default first step instead of fallback.

Security and Trust Notes

  • Prefer a preinstalled agent-device binary over on-demand package execution.
  • If install is required, pin an exact version.
  • Keep logging off unless debugging and use least-privilege environments for autonomous runs.

References

Keep looking

Skills are one crate of 613,872. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.