Agent observability

Skill Arnie016/codex-prompt-templates/skills/agent-observability

Color-coded Codex prompt templates and Agent Skills for plugin-orchestrated AI coding workflows, MCP safety, repo intelligence, and automation.

Install

npx -y skills add Arnie016/codex-prompt-templates --skill agent-observability

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Add lightweight local observability to Codex/agent workflows. Use when tracking local runs, traces, evals, cost, token usage, cache efficiency, model comparisons, failure rates, or claudetop-style monitoring. Skip when designing production gateway policy, staged guardrails, shadow traffic, model budgets, or trace-to-regression loops; use `$auto-skill-build-agent-reliability-loop`.

SKILL.md

2.1 KB, as published. Nobody here has run it

Agent Observability

Use lightweight local observability before adopting a full platform.

Minimal Run Ledger

Create:

.agent-runs/
  runs.jsonl
  evals.md
  costs.md

Each runs.jsonl entry should include:

{"ts":"","task":"","model":"","inputs":"","commands":[],"result":"","tokens_est":"","cost_est":"","status":""}

What To Track

task type and outcome
commands run
tests passed/failed
estimated tokens and cost
repeated failure causes
user corrections
guardrail or safety decisions

Eval Loop

Define 3-10 realistic tasks.
Run the same task across workflow variants.
Score correctness, time, tokens, and human intervention.
Keep the cheaper path only if quality is comparable.

Skip Or Escalate

When observability needs gateway policy, staged guardrails, shadow traffic, trace-to-regression feedback, or model budgets, switch to $auto-skill-build-agent-reliability-loop. Keep this skill for local ledgers and lightweight eval/cost tracking.

Use full platforms like Future AGI when you need hosted traces, simulations, gateway routing, or production guardrails. Use local ledgers for personal workflows.

For appending or summarizing .agent-runs/runs.jsonl, use $auto-skill-build-agent-run-ledger-dashboard.

For Codex Supercharge artifact grades and subagent/automation handoff status, use $auto-skill-build-supercharge-feedback-loop; it writes a separate local feedback stream under plugins/codex-supercharge/.agent-runs/.

Validation

Each tracked run has a task, status, commands/tests, and outcome.
Cost/token notes are marked as estimates unless measured.
Repeated failures are summarized into a follow-up or eval case.