Pilot analysis
ACP is a reusable democratic coordination primitive for discussion under load. Relay is the reference implementation used to validate that primitive in practice.
npx -y skills add jasontang-ai/acp --skill pilot-analysisAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Use this skill to compute and inspect ACP metrics, telemetry, and export outputs for intervention and baseline cycles without overclaiming.
SKILL.md
2.5 KB, as published. Nobody here has run it
What this skill is for
Use this skill when a pilot, benchmark, or rehearsal needs condition-comparable analysis rather than raw cycle inspection.
When to use it
- Summarizing one or more cycles.
- Comparing intervention and baseline conditions.
- Preparing pilot evidence for operator or reviewer handoff.
When not to use it
- Do not use it to manage lifecycle transitions.
- Do not use it to author participant content.
- Do not convert prototype metrics into field-efficacy claims.
Inputs expected
- One or more cycle ids or cycle JSON files
- Condition labels
- Optional export mode
- Claim boundary for the analysis
Preflight / prerequisites
- Confirm cycles are comparable.
- Confirm exports and metrics are generated by Relay.
- Separate benchmark evidence, pilot evidence, and field-efficacy claims.
Workflow
- Run one-cycle metric summaries.
- Run condition comparison when at least two cycles exist.
- Inspect missing metrics and telemetry gaps.
- Summarize only what the evidence supports.
- Call out non-claims explicitly.
Decision rules / judgment criteria
- Comparable denominators matter more than impressive numbers.
- Missing telemetry is a gap.
- Synthetic benchmark evidence supports operational feasibility, not field efficacy.
- Baseline and intervention condition counts should remain aligned where the scenario requires it.
Escalation rules
Escalate when intervention and baseline differ in participant or contribution count unexpectedly, when key metrics are missing, or when a requested claim exceeds the evidence class.
Available scripts
scripts/metrics-summary.mjs <cycle-id>scripts/compare-cycles.mjs <cycle-id> <cycle-id> [...]
Outputs
- Metrics summary
- Condition comparison
- Explicit evidence gaps
- Claim-safe analysis notes
Failure handling
If telemetry is incomplete, report incompleteness and stop short of numeric inference.
Trust / safety notes
Pilot analysis is vulnerable to overclaiming. Treat every output as a bounded evidence artifact.
Composition notes
Use inside pilot-cycle-review and foresight-submission-prep.
Examples to inspect next
Read examples/casebook.md for condition comparison and overclaiming anti-patterns.
Evaluation hooks
Run scripts/metrics-summary.mjs, scripts/compare-cycles.mjs, and npm run skills:audit.