Validate genai spans
Skill ContextJet-ai/awesome-llm-observability/skills/validate-genai-spans
50+ curated LLM observability tools PLUS 26 Agent Skills (several with runnable, unit-tested scripts) to build, evaluate, debug, secure & monitor reliable LLM apps. Tracing, evals, guardrails, LLMOps.
npx -y skills add ContextJet-ai/awesome-llm-observability --skill validate-genai-spansAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
What its author says it does
Copied from the file, not written here
Use this to check that your LLM tracing actually emits complete, spec-compliant spans, so cost/latency/model dashboards downstream are not full of holes. Trigger on "are my traces complete", "validate my instrumentation", "my spans are missing fields", "lint my OTel GenAI spans", "test my tracing". Ships a runnable, tested validator you can drop into your instrumentation tests.
The file declares its own license as CC0-1.0. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.
SKILL.md
2.4 KB, 441 tokens by cl100k_base, as published. Nobody here has run it
Validate GenAI spans
Broken instrumentation fails silently: the app works, the dashboards just quietly miss the model, the tokens, or the cost. This skill ships a validator that asserts your spans carry the required OpenTelemetry gen_ai.* fields, so you catch gaps in a test instead of in a half-empty dashboard.
Use the bundled script
scripts/validate_span.py is pure Python, no install needed:
from validate_span import validate_gen_ai_span, is_valid_gen_ai_span
problems = validate_gen_ai_span(span_attributes) # [] means valid
validate_gen_ai_span(span_attributes, strict=True) # also flag recommended fields
assert is_valid_gen_ai_span(span_attributes) # use in an instrumentation test
It checks the required fields (gen_ai.system, gen_ai.operation.name, gen_ai.request.model) and, in strict mode, the recommended usage fields (gen_ai.usage.input_tokens/output_tokens, gen_ai.response.model) that cost analysis depends on.
How to apply it
- Add an instrumentation test: capture the span your app emits for a sample call (most SDKs have an in-memory span exporter for tests) and assert
is_valid_gen_ai_span(attrs). - Run it in CI so a change that breaks instrumentation fails the build, not production.
- Use strict mode once basic spans pass, to push toward full cost/usage coverage.
Pairs with instrument-llm-observability (which gets the spans emitted in the first place).
Validation
Run the tests: pytest skills/validate-genai-spans/tests/. They cover a valid span, each missing required field, empty/wrong-typed values, strict-mode recommended checks, and non-dict input.
Anti-patterns
- Assuming "traces show up" means they are complete (missing token fields break cost dashboards).
- Only checking spans by eye once, instead of asserting them in CI.
- Ignoring the response model id (you cannot break cost down by served model without it).