agentsclimarketplace
Sign in

Gemini er

Skill graph-robots/open-robot-skills/tools/gemini-er

Open-vocabulary 2D object detection via the Gemini Robotics-ER API — one call returns pixel-space bounding boxes with labels and scores for a text query. Use when a workflow needs a detection box to seed segmentation (e.g. sam3.segment_box) or coarse localization without any local GPU model.From its SKILL.md

Install
npx -y skills add graph-robots/open-robot-skills --skill gemini-er

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • reads credentialsReads from 2 credential sources: `GOOGLE_API_KEY` and 1 more.
  • runs commandsInstructs the agent to run 1 command, including `uv sync --extra gemini-er`.

What its file declares

Copied from the file, not written here

The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

2.4 KB, 442 tokens by cl100k_base, as published. Nobody here has run it

gemini-er

API-backed 2D detection on Gemini Robotics-ER. Zero GPU. The canonical perception recipe (from the dev tree's perceive_gemini_er workflow script) is: gemini-er.detect → best box by score → sam3.segment_box on the full frame at that box → depth projection → OBB fit.

Install

uv sync --extra gemini-er    # google-genai  (pip: pip install -e ".[gemini-er]")
export GOOGLE_API_KEY=...    # or GEMINI_API_KEY

Config

EnvMeaningDefault
GAP_GEMINI_ER_MODELGemini model namegemini-robotics-er-1.5-preview
GOOGLE_API_KEY / GEMINI_API_KEYAPI key (SDK default resolution)—

Contract

gemini-er.detect(image, query) returns {"detections": [{"box": BoundingBox2D, "label": str, "score": float}]}:

  • box is pixel-space {x1, y1, x2, y2} (top-left → bottom-right), clamped to the image bounds. The model emits the Gemini box_2d convention ([ymin, xmin, ymax, xmax] normalized 0–1000); conversion happens here.
  • score defaults to 1.0 when the model reports none — callers select the best detection with max(..., key=score).
  • No match (or unparseable model output) → empty detections, never an error. Treat empty as "object not visible".

When to use

  • Detection boxes for open-vocabulary prompts, no local weights.
  • Prefer molmo.point_prompt when a single click point is enough, and vlm.query_yes_no for semantic checks without localization.

What ships with it: 3 files

7.9 KB alongside SKILL.md, 1 of them executable

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.