Read and translate sign

Skill sumitaich1998/jarvisvr/skills/perception/read-and-translate-sign

An AI agentic operating system for mixed reality on the Meta Quest 3 — your own J.A.R.V.I.S. Multi-agent orchestration, multimodal perception (sight/hearing/gaze), 42 holographic widgets, 20 LLM providers.

Install

npx -y skills add sumitaich1998/jarvisvr --skill read-and-translate-sign

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Read text visible in the passthrough camera (a sign, menu, label, or document) and optionally translate it into the user's language. Use for "read this", "what does this say", "translate this sign/menu", or OCR of something in view. Triggers: read this, what does this say, translate this sign, menu, label, OCR, foreign text.

The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

2.8 KB, as published. Nobody here has run it

Read & Translate Sign

OCR text from the current view, then (if a target language is implied or asked) translate it, surfacing both on a translator panel in sign mode.

Steps

Pick the path:
- Read only ("what does this say?") → read_text → shows a panel.
- Read + translate ("translate this sign") → translate_view (does OCR + translation in one call) → shows a translator.
- Translate provided text (not from camera) → translate_text.
Determine target language. Default to the session locale from client.hello; honor explicit "…into French".
Call the tool. translate_view returns data.source_text, data.translated, data.target_lang, an observation, and a translator directive.
Narrate with agent.observation/agent.speech and render the panel.

Output

agent.observation:

{ "text": "The sign reads: 非常口. In English: Emergency Exit.", "final": true, "annotations": [] }

translator hologram (show_translator, props per registry.json):

{ "widget_type": "translator",
  "transform": { "anchor": "head", "position": [0.0,0.0,1.0], "billboard": true },
  "props": { "source_lang": "auto", "target_lang": "en",
             "source_text": "非常口", "translated_text": "Emergency Exit", "mode": "sign" },
  "interactions": ["tap","grab","resize","toggle"] }

Read-only path uses a panel:

{ "widget_type": "panel", "props": { "title": "Read", "body": "Gate B12 — Boarding 14:05" } }

Edge cases

No legible text → "I can't make out any text from here — move a little closer or steady the view." Spawn nothing.
Mixed languages / long passages → translate the dominant language; for long documents suggest document_viewer instead of a translator panel.
Live two-way conversation (not a static sign) → hand to communication-agent's live-translate.
Source == target language → skip translation, just read it back.
Numbers/codes (gate, room, price) → preserve verbatim; don't "translate" digits.
Privacy → if you warmed the camera with perception.request{action:"once"}, stop it when done.