Skip to the rows

agentsclimarketplace

Annotate reality

Skill sumitaich1998/jarvisvr/skills/stage/annotate-reality

An AI agentic operating system for mixed reality on the Meta Quest 3 — your own J.A.R.V.I.S. Multi-agent orchestration, multimodal perception (sight/hearing/gaze), 42 holographic widgets, 20 LLM providers.

Install

npx -y skills add sumitaich1998/jarvisvr --skill annotate-reality

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Place world-anchored labels, callouts, pins, and boxes onto real objects and locations so information sticks to the physical world. Use for labeling things in the room, pinning a spot, boxing a detected object, or step-by-step "point at the X" guidance. Triggers: label this, pin here, mark that, point to, highlight, call out, annotate, put a label on.

The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

3.0 KB, as published. Nobody here has run it

Annotate Reality

Anchor information to the physical world. The perception-agent decides what a real thing is; you render the spatial label/box/pin in the right place.

Pick the annotation

Goal	Widget	Tool
Name a real object with a callout	`vision_annotation`	`annotate_object`
Box a detected object's extent	`bounding_box_3d`	`draw_bounding_box`
Drop a pin/marker at a spot	`scene_label`	`drop_scene_label`
Free-floating heading/caption	`text_label`	`show_text`

Steps

Get the target position (world meters) from the perception annotation, gaze hit point, or a scene surface.
Choose the widget from the table; keep labels short.
Anchor in world with billboard:true so the label faces the user; lift the callout slightly above the object (e.g. +0.15 m in Y) with a leader_line.
Spawn, then keep it anchored as the user moves (world anchor handles this).
Clean up stale annotations via close_hologram when no longer relevant.

Output

vision_annotation (annotate_object, props per registry.json):

{ "widget_type": "vision_annotation",
  "transform": { "anchor": "world", "position": [0.3,0.95,0.7], "billboard": true },
  "props": { "label": "coffee mug", "confidence": 0.78, "detail": "ceramic, ~350 ml",
             "leader_line": true, "target_position": [0.3,0.8,0.7], "color": "#7FE7FF" },
  "interactions": ["tap","grab","dwell"] }

bounding_box_3d (draw_bounding_box):

{ "widget_type": "bounding_box_3d",
  "props": { "label": "laptop", "confidence": 0.91, "size": [0.33,0.02,0.23], "color": "#FFB74D" } }

scene_label (drop_scene_label):

{ "widget_type": "scene_label",
  "props": { "text": "Keys (last seen here)", "icon": "pin", "color": "#FF5252", "pin": true } }

Edge cases

No reliable position → fall back to a head-anchored text_label and say you couldn't pin it precisely.
Many annotations → cap to the relevant few; over-labeling clutters reality (hand to declutter-space).
Moving target → annotations are static once placed; re-place if the object moves.
Confidence honesty → carry the real confidence so the UI can style uncertainty.
Steps/tour → sequence pins and advance as the user completes each.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.