agentsclimarketplace

Nvidia triton inference serving review

Skill Raishin/vanguard-frontier-agentic/skills/nvidia/nvidia-triton-inference-serving-review

Curated marketplace of AI skills, agents, and rules for cloud, zero-trust, and compliance-aware engineering - works with Claude Code, Codex, Cursor, Copilot, and more.

Install
npx -y skills add Raishin/vanguard-frontier-agentic --skill nvidia-triton-inference-serving-review

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 18 stars18 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use this skill when reviewing Triton Inference Server deployments statically — `model_repository/` layout and `config.pbtxt` files, dynamic batching configuration, ensemble and BLS pipelines, custom backend (Python, C++, ONNX, OpenVINO, vLLM) trust posture, gRPC and HTTP endpoint authentication, response cache configuration, rate-limit and metrics exposure. Trigger when the user asks whether a Triton model repository or `tritonserver` invocation follows NVIDIA's published guidance and security expectations.

SKILL.md

3.0 KB, as published. Nobody here has run it

NVIDIA Triton Inference Server Review

Purpose

Static review of Triton Inference Server deployments against NVIDIA's Triton documentation — model repository layout, dynamic batching, ensemble pipelines, custom backend trust, gRPC/HTTP authentication, model encryption at rest, response cache poisoning surface. This skill is doc-anchored: it grounds review findings in NVIDIA's published documentation rather than in a certification blueprint, because no NVIDIA certification currently covers this developer-facing surface as a standalone exam objective.

Lean operating rules

  • Prefer the user's actual model_repository/ tree and config.pbtxt files as evidence; otherwise fall back to documentation-based inference.
  • Treat custom Python or C++ backends loaded from non-pinned sources or without code review as a critical finding — in-process RCE.
  • Treat gRPC or HTTP endpoints exposed without authentication, mTLS, or a restricted-protocol gateway as a critical finding for multi-tenant deployments.
  • Treat model repository directories with world-writable permissions or a writable --model-repository mount as a high finding — silent model substitution.
  • Treat response caching enabled across tenants without per-request cache-key partitioning as a high finding — cross-tenant cache poisoning.
  • Treat ensemble or BLS pipelines that pass user-supplied tensors directly to a Python backend without input validation as a medium finding — deserialization surface.
  • Treat metrics endpoints (:8002) exposed to the public network without scraping ACLs as a medium finding — model name and shape leakage.
  • Treat dynamic batching max_queue_delay_microseconds left at default with latency SLOs in the millisecond range as a low finding — throughput-vs-latency tuning is wrong by default.
  • Always emit the exact tritonserver and perf_analyzer commands the user should run — do not execute them.

Response minimum

Return, at minimum:

  • the scoped target (model repository layout and provenance, backend trust posture, endpoint and auth posture, batching and ensemble posture, response cache and metrics posture, recommended tritonserver/perf_analyzer invocations) and evidence level,
  • findings labelled critical / high / medium / low,
  • recommended NVIDIA-tooling invocations the user should run themselves,
  • safe next actions and assumptions or blockers.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.