agentsclimarketplace

Paper replicate

Skill lucaswychan/quant-paper-agent/quant_paper_skills/skills/paper-replicate

A Claude Code plugin that turns a quantitative-finance arxiv paper into a production-ready event-driven strategy

Install
npx -y skills add lucaswychan/quant-paper-agent --skill paper-replicate

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Plan, approve, implement, and audit a quantitative-paper backtest against an exact evidence-backed strategy variant from extraction/reported.json. Use for independent replications, approximate reproductions, or illustrative reconstructions. Creates an immutable run directory, requires explicit approval tied to plan/config hashes before code or data execution, enforces timing and cost contracts, and never tunes parameters to improve agreement.

SKILL.md

4.8 KB, as published. Nobody here has run it

Paper replicate

Create one auditable replication run. Never mix results from different plans in the paper root.

Run layout

papers/<versioned-id>/replications/<run-id>/
├── plan.md
├── config.json
├── approval.json
├── data/
│   ├── prices.csv
│   └── manifest.json
├── code/
│   ├── signal.py
│   ├── portfolio.py
│   └── backtest.py
└── outputs/
    ├── equity.csv
    ├── returns.csv
    ├── weights.csv
    ├── trades.csv
    ├── metrics.json
    ├── comparison.json
    ├── nav.png
    ├── drawdown.png
    └── benchmark.md

Read references/plan_template.md and references/run_schema.md before Phase A. Read references/benchmark_template.md before Phase C.

Phase A: plan and halt

  1. Read extraction/note.md, extraction/reported.json, and extraction/extraction_report.md.
  2. Select one declared replication_target.target_id; never infer a target from metric ordering.
  3. Classify the run:
    • exact_replication: equivalent point-in-time data, universe, period, construction, and metric definitions are available.
    • approximate_replication: the method is preserved but a material input differs.
    • illustrative_reconstruction: public proxies or a substantially different sample are used.
  4. Write plan.md and machine-readable config.json. Specify the observation, signal, decision, fill, and return timestamps; data vendor and universe; annualization; risk-free convention; portfolio normalization; transaction costs; slippage; borrow; and every unresolved paper choice.
  5. Run scripts/validate_run.py <paper-id> <run-id> --phase plan.
  6. Halt and surface the plan and classification. Do not fetch data or write backtest code until the user explicitly approves.
  7. After explicit approval, record it with scripts/record_approval.py. The approval hashes both plan.md and config.json; later edits invalidate it.

Phase B: implement and run

After approval:

  1. Fetch or place data under the run's data/ directory. fetch_data.py is only appropriate when config.json declares yfinance; such a run is normally approximate or illustrative.
  2. Verify data/manifest.json, including vendor, request, adjustment policy, row/column counts, dependency version, and SHA-256.
  3. Write small, testable signal.py, portfolio.py, and backtest.py files. Adapt a template only when its timing and mathematics match the approved plan.
  4. Test at minimum:
    • future-price perturbations do not change earlier signals or weights;
    • weights applied to a return were knowable before that return interval;
    • gross/net exposure and leverage constraints hold;
    • turnover and configured costs reconcile to net returns;
    • missing data and rebalance boundaries behave as planned.
  5. Run:
python3 scripts/fetch_data.py <paper-id> <run-id>
python3 scripts/run_backtest.py <paper-id> <run-id>
python3 scripts/plot_nav.py <paper-id> <run-id>

The harness reads frequency and metric conventions from config.json; do not hardcode 252 for non-daily runs.

Phase C: compare and judge

Run:

python3 scripts/compare_metrics.py <paper-id> <run-id>

This produces deterministic comparison.json for the exact metric IDs selected by the target. It does not declare success automatically. Inspect the comparison, data mismatch, timing, plots, turnover, costs, and extraction ambiguities, then author benchmark.md.

Use one verdict:

  • replicated: exact run, all primary anchors comparable, and discrepancies fall within predeclared tolerances.
  • partially replicated: mechanism or direction is reproduced but material anchors or inputs differ.
  • not replicated: primary claims fail under a faithful comparable implementation.
  • inconclusive: evidence, data, or definitions are insufficient for a defensible verdict.

Confidence depends on evidence coverage and comparability, not merely numerical closeness.

Hard rules

  • Never tune parameters, choose a favorable sample, or switch variants to improve fit.
  • Never call a proxy-data run an exact replication.
  • Never add or omit costs silently.
  • Never bypass or regenerate approval after changing the plan without returning to the user.
  • Treat a surprisingly close match as an audit trigger, not proof of correctness.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.