Nvidia video search and summarization video understanding

Skill autohandai/community-skills/nvidia-video-search-and-summarization-video-understanding

A collection of curated, useful, and safe skills for Autohand Code CLI Agent

Install

npx -y skills add autohandai/community-skills --skill nvidia-video-search-and-summarization-video-understanding

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

9 stars9 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Call the vss agent to run video understanding on video to answer a text question. Use when the user asks about video content, or about visual details that cannot be answered from conversation history, search hits, or metadata alone.

The file declares its own license as Apache-2.0. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

3.4 KB, 738 tokens by cl100k_base, as published. Nobody here has run it

Video QnA using VLM through VSS Agent

Use this skill when you need details about the video which requires VLM to look at the video frames — for example the agent has no usable prior answer and needs a fresh look at the pixels for a specific clip.

When to Use

The user asks what happens in the video, what objects / people / actions appear, colors, timing, safety, or other visual facts that require watching the clip.
The user asks for details that cannot be answered from existing messages, summaries, Elasticsearch/MCP results, or filenames alone—you need model inference on the video.
Follow-up questions about content details after a coarse summary or after report generation.

Do not use this skill when a database / MCP / prior tool output already answers the question, unless the user explicitly wants verification against the video.

Deployment prerequisite

This skill requires a VSS profile that serves the video_understanding tool — typically base (recommended) or lvs. Before any request:

Probe the VSS agent:

curl -sf --max-time 5 "http://${HOST_IP}:8000/docs" >/dev/null

If the probe fails, ask the user:

"No VSS profile is running on $HOST_IP. Shall I deploy base (recommended for per-clip VLM QnA) using the /deploy skill? If you prefer lvs, say so."
- If yes → hand off to /deploy -p base (or -p lvs if the user prefers). Return here once it succeeds.
- If no → stop.
(If your caller has granted explicit pre-authorization to deploy autonomously — e.g. the request says "pre-authorized to deploy prerequisites", or you are running in a non-interactive evaluation harness with that permission — skip the confirmation and invoke /deploy -p base directly. Prefer base unless the request names another profile.)
If the probe passes, proceed.

Agent workflow

Clip — Identify sensor id, filename, or URL for one video segment. If ambiguous, ask the user.
Call vss agent with the sensor id and ask for it to call video_understanding tool to answer the user's question. The sensor / file name must be included in the input message to the agent.
Return the vss agent's answer back to the user.

Query VSS agent (`/generate`)

# Set from deployment (compose / .env / host where vss-agent listens)
export VSS_AGENT_BASE_URL="http://localhost:8000"

curl -s -X POST "${VSS_AGENT_BASE_URL}/generate" \
  -H "Content-Type: application/json" \
  -d '{"input_message": "Call video_understanding tool to answer the following question about <sensor-id>: <user query>"}' | jq .

Cross-Reference

vios — VST storage/replay URLs so VIDEO_URL is valid for the VLM.
report — timestamped reports via the VSS agent (/generate); this skill is direct VLM for ad-hoc video Q&A.