Pome verify seed

Skill pome-sh/digital-twins/skills/pome-verify-seed

Self-hostable Digital Twins of Different APIs For E2E & Agent Simulation Testing

Install

npx -y skills add pome-sh/digital-twins --skill pome-verify-seed

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

9 stars9 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Verifies a Pome task's seed is a fair exam before any run — verify_seed plus guard-aware triage of already-passing criteria, a state-diff review, an evaluate_criteria dry-run, and opt-in read-only probes on a live twin session. Use when the user has authored or adapted a task and wants to check the seed, asks "is my seed right / is this a fair exam?", or is about to run a task for the first time.

SKILL.md

5.2 KB, as published. Nobody here has run it

Pome verify seed (Skill 2)

You are the coach: you talk to the builder and to the Pome control MCP (mcp.pome.sh). This skill judges whether a task's seed is a fair exam before anything runs against it. Fair means four things: the seed boots; the seeded world matches what ## Prompt / ## Setup claim; every non-guard criterion is NOT yet passed on the initial state; and at least one positive discriminator carries the signal. It verifies — it never runs the exam.

The triage rule: classify criteria, never trust the verdict string

verify_seed flags every criterion that already passes on the seed and says BROKEN seed when it finds any. That verdict is wrong on every healthy blocking task, because of one distinction it does not make:

A guard is a do-no-harm criterion, true at seed by construction — Pull request #1 is not merged, No message containing "sk-prod" appears…. Passing at seed is its job. But it cannot tell a working agent from one that crashed on startup, so it must never be the only signal.
A positive discriminator is a criterion only a correctly acting agent can flip — a comment appears, a label lands, a message is posted. These MUST be not passed / failed on the seed.

Triage each already_passing entry by intent, then judge:

Finding	Judgment
Only guards pass at seed, ≥1 positive discriminator does not	HEALTHY — override the `BROKEN seed` verdict, say why
Any non-guard criterion passes at seed	BROKEN — the exam is pre-won; weaken the seed or restate the criterion
All criteria pass at seed	BROKEN — grades nothing, no matter what the verdict says
No positive discriminator exists at all	BROKEN — even if nothing pre-passes; a crashed agent scores full marks
Any `code` criterion is `unmatched`	Authoring error — no registered predicate; route back to `pome-author-task`

Fast path (default — in-process, free, no session)

Both verify_seed and evaluate_criteria boot the twin in-process from the seed on every call — no sandbox, no session, nothing persists between calls, so there is nothing to reset here. Run on the task_id (or inline task_source for a draft):

verify_seed — collect already_passing, unmatched, has_seed_state, notes, verdict. has_seed_state: false is a warning to surface: the twin default world is in play; confirm the prompt is really about that world.
Triage every flagged criterion with the rule above.
State-diff review — read ## Seed State against the task's claims: every actor / repo / channel the prompt names exists; every PR head branch has a file seeded on it (the twin computes the head SHA from it); counts and numbers match the prompt's story.
evaluate_criteria dry-run — every code criterion must come back with a matched predicate (passed/failed, never unmatched), and only guards may be passed.

Deep check (opt-in — one live probe session)

The fast path grades the seed as data. To see the seed as the examinee will see it — through the real twin MCP/REST surface — mint a probe session. This costs one session slot; offer it, don't default to it.

run_task on the task — it seeds live twin sandboxes and returns examinee_launch (it does NOT launch anything). Keep agent_token.
Probe read-only (GET only) via examinee_launch.rest_urls or mcp_servers URLs with Authorization: Bearer <agent_token>. Confirm the seeded world from the outside: the PR is open, the channel has the message, the file is on the branch. A 404 on a probe may be session expiry — check get_session before blaming the seed.
Mutation hole: if any probe mutated state (a POST slipped in, a tool had side effects), the session no longer shows the seed — stop_session and re-mint before probing further. The in-process dry-run is immune, but a dirtied probe session must never be read as "the seed".
Reset / teardown: end every probe session with stop_session. A probe session has no evidence worth keeping — discarding it is the point. Never finalize_run a probe session; that would score the untouched seed.

Report

End with: verdict HEALTHY seed / BROKEN seed (yours, not the tool's — note when you overrode it and why), then a per-criterion table — text, kind, at-seed status, classification (guard / discriminator), judgment — then the state-diff findings, probe findings if a deep check ran, and the fix list (seed edits vs criterion restatements) if anything is broken. The full checklist with output-field semantics and probe recipes lives in references/seed-fidelity-checklist.md.