agentsclimarketplace

Auto skill build mcp conformance harness

Skill Arnie016/codex-prompt-templates/skills/auto-skill-build-mcp-conformance-harness

Color-coded Codex prompt templates and Agent Skills for plugin-orchestrated AI coding workflows, MCP safety, repo intelligence, and automation.

Install
npx -y skills add Arnie016/codex-prompt-templates --skill auto-skill-build-mcp-conformance-harness

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Design a repeatable MCP conformance, health-check, and tool-surface regression harness for local or CI use. Use when the user asks for MCP CI, protocol or OAuth conformance, server doctor checks, tool-surface diffing, JUnit/XML reports, or release gates for an MCP server. Requires an MCP endpoint or stdio launch command; skip when the task is only ad-hoc manual inspection where `$mcp-server-lab` is enough.

SKILL.md

2.8 KB, as published. Nobody here has run it

MCP Conformance Harness

Generated by: Codex Supercharge maintenance automation.

Goal: turn MCP server testing into a repeatable harness with clear evidence, secret handling, and pass/fail gates.

Workflow

  1. Use $sandbox-source-intake and $mcp-safety-review before running a fresh MCP server, unknown CLI, install script, or broad-account tool.
  2. Identify the target matrix:
    • transport: HTTP or stdio
    • auth: none, static token, OAuth interactive, OAuth headless, or M2M
    • checks: doctor, tool surface, protocol, OAuth, apps UI
  3. Prefer read-only checks first: server probe, server doctor, and server export with --no-telemetry and JSON output.
  4. Add protocol, oauth, or apps conformance suites only after auth and blast radius are understood; emit JUnit/XML or JSON summary artifacts.
  5. Capture before/after server export snapshots and sort them before diffing when release tool-surface drift is the risk.
  6. Use the command recipes in references/mcp-conformance-harness.md instead of expanding long shell blocks into the skill body.
  7. Store reports as CI artifacts, but keep credentials out of artifacts. Prefer environment secrets, credentials files with mode 0600, or CI secret stores.
  8. Summarize failures by phase: transport, auth, protocol, tool schema, apps UI, or release diff.

Harness Contract

# MCP Harness Plan
Target:
Transport/auth:
Read-only checks:
Conformance suites:
Tool-surface diff:
Secrets:
Artifacts:
Pass/fail gates:
Known skips:

Skip When

  • The user only wants to manually explore one server once.
  • No MCP endpoint, stdio command, or runnable server target is available.
  • The server/tool is untrusted and has not passed source intake or safety review.
  • Running the harness would call write/destructive tools without explicit scope.

References

  • references/mcp-conformance-harness.md
  • Source inspiration only, code not executed: sources/mcpjam-inspector
  • Support skills: $mcp-server-lab, $mcp-safety-review, $auto-skill-safety-mcp-secret-guard

Validation

plugins/codex-supercharge/scripts/skill_audit.sh plugins/codex-supercharge/skills
python3 "$HOME/.codex/skills/.system/skill-creator/scripts/quick_validate.py" \
  plugins/codex-supercharge/skills/auto-skill-build-mcp-conformance-harness

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.