Robotframework platynui skill
Skill manykarim/robotframework-agentskills/skills/robotframework-platynui-skill
Guide AI agents in creating native desktop UI tests with PlatynUI (new_core) using the PlatynUI.BareMetal library — cross-platform desktop automation over Windows UIA and Linux AT-SPI2 with an XPath query model. Use when asked to automate desktop applications (windows, buttons, menus, text fields), locate native UI elements by XPath, or write Robot Framework tests with PlatynUI.From its SKILL.md
npx -y skills add manykarim/robotframework-agentskills --skill robotframework-platynui-skillAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
SKILL.md
12.6 KB, as published. Nobody here has run it
PlatynUI Library Skill (new_core / BareMetal)
Quick Reference
PlatynUI is cross-platform native desktop UI automation for Robot Framework. It normalizes Windows UIA, Linux AT-SPI2 (and planned macOS AX) into one desktop UI model that you query with XPath-2.0-style selectors.
⚠️ Preview / pre-release. This skill targets the
new_corerewrite. The only usable keyword library today isPlatynUI.BareMetal; the high-levelPlatynUIlibrary is a not-yet-implemented placeholder. Keywords, CLI output, and behavior may change.
Installation
Get the version with the right pin — this is the #1 mistake:
# ✅ new_core (PlatynUI.BareMetal, documented keywords) — pin OR use --pre:
pip install robotframework-PlatynUI==0.12.0.dev330
pip install --pre robotframework-PlatynUI # latest pre-release
# ❌ WRONG — installs old 0.9.2 (a different library, NO BareMetal):
pip install robotframework-PlatynUI # no --pre, no pin → 0.9.2
- A plain
pip install robotframework-PlatynUIresolves the old stable0.9.2, whose surface is a different, near-undocumentedPlatynUIlibrary with noBareMetal. You only getnew_coreby opting into the pre-release (--pre) or pinning an exact dev version. - The wheel is prebuilt — it bundles the compiled
platynui-nativeruntime (_native.abi3.so). No Rust toolchain and no git checkout required. Python 3.12+ only. - Optional CLI / inspector tools (for locator development — see below):
uv tool install --prerelease allow platynui-cli uv tool install --prerelease allow platynui-inspector - Linux additionally needs a running AT-SPI2 accessibility bus and (recommended) an X11 session — see
references/platform-setup.md.
Library Import
*** Settings ***
Library PlatynUI.BareMetal
Import arguments (all keyword-only, all optional):
| Argument | Default | Purpose |
|---|---|---|
auto_activate | True | Bring the target element's window to the foreground before pointer/keyboard actions. Override per-call with activate=. |
use_mock | False | Use the Rust mock provider (deterministic, no desktop). Note: the published wheel is built without mock support and raises ProviderError; needs a --features mock-provider source build. |
keyboard_profile / pointer_profile / pointer_settings | None | Tune input timing/motion realism. Accept a native object or a plain &{dict} (e.g. press_delay_ms, acceleration_profile, motion). |
Library PlatynUI.BareMetal auto_activate=${True} pointer_profile=${POINTER_PROFILE}
*** Variables ***
&{POINTER_PROFILE} acceleration_profile=EASE_OUT motion=JITTER jitter_amplitude=100.0
Library scope is SUITE. The native runtime is created lazily on first keyword use (so libdoc and import work without a desktop).
Essential Concepts
The normalized desktop UI model
PlatynUI exposes every application's accessibility tree as one queryable model rooted at the Desktop. Each node has a normalized PascalCase role (Window, Button, Edit, ListItem…), a Name, an optional developer Id, and typed attributes (Bounds, IsEnabled, Value…). Native roles are normalized cross-platform (UIA Button / AT-SPI PUSH_BUTTON → Button); the original stays under native:Role.
Four query namespaces
control: UI controls (Window, Button, Edit, …) ← DEFAULT (unprefixed = control:)
item: container children (ListItem, TreeItem, TabItem, Cell, Row)
app: Application/process nodes
native: raw platform attributes (native:HWND, native:Role, …)
Critical rule: an unprefixed query matches control: only. You MUST prefix app: and item: explicitly — //ListItem finds nothing; //item:ListItem works.
XPath query language
Elements are referenced by an XPath string passed directly as a keyword argument (auto-converted to an element descriptor), or by a UiNode captured from Query ... only_first=${True}.
Pointer Click //control:Button[@Name="OK"]
Pointer Click app:Application[@Name="Calculator"]//Button[@Id="num5Button"]
${count}= Query count(//control:Button) # value expression
${node}= Query //control:Edit[@Name="Search"] only_first=${True}
Supports the full XPath-2.0 axis/function set: // (descendant), ../parent::, following-sibling::, predicates ([@Name='OK'], [1]), and functions (contains(), matches(), count(), starts-with()). See references/locators-and-queries.md.
Lazy resolution with built-in wait
There is no explicit "wait for element" keyword. When a descriptor is first used, PlatynUI polls the query (default ~30 s, retrying ~0.1 s) until the element resolves or it raises ElementNotFoundError. Write the correct selector and let the runtime wait — don't add Sleep.
Coordinate semantics (pointer keywords)
| You pass | Behavior |
|---|---|
| descriptor, no x/y | acts at the element's ActivationPoint (else bounds center) |
| descriptor + x/y | x/y are offsets from the element's top-left bounds |
| no descriptor, x/y | x/y are absolute screen coordinates |
| only one of x/y | ValueError — pass both together |
Window keywords need a window node
Activate/Minimize/Maximize/Restore/Close/Move/Resize Window operate on a node that supports the window surface — target a Window node, not an arbitrary control, or they raise PatternError.
Core Keywords Quick Reference
Query & scope
${node}= Query //control:Button[@Name="OK"] only_first=${True}
${all}= Query //item:ListItem # list of nodes
${n}= Query count(//control:Button) # computed value
Set Root app:Application[@Name="Calculator"]//control:Window # scope later queries
Set Root ${None} # reset to desktop
Pointer / mouse
Pointer Click //control:Button[@Name="OK"]
Pointer Click //control:Button[@Name="OK"] button=RIGHT activate=${False}
Pointer Multi Click //item:ListItem[@Name="file.txt"] clicks=${2}
Pointer Move To ${None} x=${100} y=${200} # absolute screen
Pointer Press //control:Slider
Pointer Release ${None}
${pos}= Get Pointer Position
Keyboard
Keyboard Type //control:Edit[@Name="Search"] Hello <Ctrl+A> world${\n}
Keyboard Type ${None} <Ctrl+A><Delete> # type into whatever has focus
Keyboard Press ${None} <Shift>
Keyboard Release ${None} <Shift>
Key syntax: <Ctrl+A>, <Return>, <ESC>, <Shift+Tab>, plain text, ${\n}. Pass ${None} as the descriptor to skip focusing. Discover valid key names with platynui-cli keyboard list.
Focus, attributes, windows
Focus //control:Button[@Name="New File"]
${bounds}= Get Attribute //control:Window[@Name="Calc"] Bounds
Get Attribute //control:Button[@Name="OK"] Name == OK # built-in assertion
Bring To Front //control:Window[@Name="Calc"]
Activate Window //control:Window[@Name="Calc"]
Maximize Window //control:Window[@Name="Calc"]
Move And Resize Window //control:Window[@Name="Calc"] ${0} ${0} ${800} ${600}
Diagnostics
Take Screenshot # full screen → ${OUTPUT DIR}
Take Screenshot //control:Window[@Name="Calc"] # just the element
Take Screenshot filename=EMBED # inline base64 in the log
Highlight //control:Button[@Name="OK"] duration=${2} # draw an overlay (debug)
Locator Strategy
- Default namespace is
control—//Button≡//control:Button. Addcontrol:only for clarity. - Always prefix
app:anditem:— applications and container items are NOT in the default namespace. Forgetting this is the most common cause of "element not found." - Prefer
@Idover@Name—[@Id="num5Button"]is language-independent and stable (UIA AutomationId / AT-SPI accessible_id / AX AXIdentifier). Use@Nameonly when no Id exists. - Anchor on the application, then descend —
app:Application[@Name='…']//control:Window//control:Button[@Id='…']. OrSet Rootonce on the app/window and use short relative queries (.//Button[@Id='…']). - Handle localization with
or—[@Name='Calculator' or @Name='Rechner']. - Fuzzy match with functions —
contains(@Name,'Save'),matches(@Name,'.*Save.*'),[contains(@*,'Close')](search across all attributes). - Step axes when structure beats identity —
…/Label[@Name='5']/parent::Button. - Let resolution wait — don't
Sleep; the descriptor retries for ~30 s.
Common Patterns
Launch an app, scope to it, act
*** Settings ***
Library PlatynUI.BareMetal
Library Process
Test Setup Start Calculator
*** Keywords ***
Start Calculator
Start Process calc.exe
# Prefer letting the next Query wait over a fixed Sleep.
Enter Digit
[Arguments] ${d}
Set Root app:Application[@Name="ApplicationFrameHost"]/control:Window[@Name="Calculator" or @Name="Rechner"]
Pointer Click .//control:Button[@Id="num${d}Button"]
Type into a field and assert
Keyboard Type //control:Edit[@Name="Search"] robot framework${\n}
Get Attribute //control:Edit[@Name="Search"] Value == robot framework
Verify targeting before automating
${node}= Query //control:Button[@Name="Submit"] only_first=${True}
Highlight ${node} duration=${2} # visually confirm you matched the right node
Pointer Click ${node}
CLI Tooling (locator development loop)
Develop and debug selectors with platynui-cli before baking them into tests:
platynui-cli list-providers # confirm a provider is active
platynui-cli query "//control:Button[@Name='OK']" # test a selector live
platynui-cli snapshot "//control:Window" --pretty # dump a UI subtree
platynui-cli highlight "//control:Button[@Name='OK']" # visually confirm on screen
platynui-inspector is a GUI that shows the live UI tree, lets you test XPath interactively, and highlights matches. See references/cli-and-inspector.md.
Troubleshooting
pip install robotframework-PlatynUIgives a library with noBareMetal→ you got0.9.2; reinstall with--preor pin==0.12.0.dev330.- Linux: queries return nothing / empty tree → AT-SPI2 not running, or accessibility not enabled. See
references/platform-setup.md(platynui-cli list-providersshould show theatspiprovider active). - Wayland: clicks land wrong / screenshots fail → use an X11 (or XWayland) session; Wayland is degraded for input injection and screen coordinates.
ElementNotFoundError→ check the namespace prefix (item:/app:), prefer@Id, and verify the selector withplatynui-cli query.ProviderError: requires building with feature 'mock-provider'→use_mock=${True}needs a source build; the published wheel has no mock provider.
When to Load Additional References
| Need | Reference File |
|---|---|
| Full keyword signatures, args, return types | references/keywords-reference.md |
| XPath syntax, namespaces, roles, axes, locator best practices | references/locators-and-queries.md |
platynui-cli subcommands + the inspector | references/cli-and-inspector.md |
| Install matrix, Linux AT-SPI/X11/Wayland, mock, Docker | references/platform-setup.md |
| BareMetal-vs-high-level status, version footgun, migration | references/status-and-migration.md |
Companion Skills
| Need | Skill |
|---|---|
| Search for keywords across libraries | rf-libdoc-search |
| Explain keyword arguments in detail | rf-libdoc-explain |
| Generate user keywords | rf-keyword-builder |
| Generate test cases | rf-testcase-builder |
| Design resource file layout | rf-resource-architect |
| Parse test results from output.xml | rf-results |