Chronicle reproduce arxiv

Reproduce an arxiv paper

Turn a published arxiv paper into an open Chronicle experiment set up to reproduce it. The end state is: the paper registered as a public arxiv asset and linked as an experiment input, the paper's code repository imported (managed-repo branch or bundle, via the chronicle-import-repo core), a hypothesis_report grounding the reproduction, and a primary research prompt framed around it — the whole flow runs locally in this session.

Transport — MCP-direct, plus the import core

Registration and linking are bundled MCP tools (chronicle.register_publication, chronicle.link_asset). The clone-and-import half composes chronicle-import-repo — read and follow that skill for steps it owns (evaluate, confirm, create, attach code/docs, report, prompt); this skill documents only what reproduction adds. Credentials from ~/.methodic (or CHRONICLE_API_KEY); none → send the user to the web UI to mint an API key.

Inputs

• arxiv — an arXiv id or URL, any of: 2301.12345, 2301.12345v2, math.GT/0309136, https://arxiv.org/abs/…, …/pdf/…. Required.
• repo (optional) — the code repository URL, if the user already knows it. Otherwise located in step 2 and always confirmed.
• research_prompt (optional) — defaults to reproduction framing: "Reproduce the headline result of <paper title> (arXiv:<id>); identify divergences." Confirm it with the user like any import.

Workflow

1. Register the paper. Call chronicle.register_publication with { "arxiv": "<id-or-url>" }. The resolver normalizes the id, pulls metadata from the arXiv Atom API, and dedups on (arxiv_id, version) — a known paper returns the existing record (existing: true), never a duplicate. Keep the returned asset id and the title/abstract (they seed the evaluation and the prompt).

2. Locate the code. This is agent judgment, not an API: look for a repo link in the abstract/comments ("Code: https://github.com/…"), in the paper body, README badges, the authors' pages — via Paperclip or web search where available. Always confirm the pick with the user before cloning: there are often several candidates (official release vs re-implementation vs fork), and "official" matters for a reproduction. Present what you found and which one you'd choose, and say why.

   No discoverable code → offer the paper-only import: create the experiment (a minimal placeholder config_yaml, flagged as such), link the arxiv asset (step 3's link call), write the hypothesis_report from the paper's abstract/claims, and anchor the reproduction prompt — no code step. The user can attach code later (chronicle-bundle-variation, or push + chronicle-rebind-variation-git).

3. Clone and import. git clone the confirmed repo, then run the chronicle-import-repo core against the clone — evaluate, confirm, create_experiment, attach code (managed-repo push to import/<slug> / bundle fallback), attach documents, write_report, create_research_prompt — with two reproduction-specific additions:

   • Pre-link the paper as an experiment input right after create_experiment: chronicle.link_asset with { "experiment_id": "<id>", "asset_id": "<arxiv asset id>", "link": "input" }. Arxiv assets are world-readable, so the server skips ACL propagation and the link needs only Read. As a citation type the link is also exempt from the commit freeze (it stays linkable until the experiment concludes) — but link it up front anyway; it anchors the whole reproduction.
   • Frame the research prompt around reproduction (the default above), and ground the hypothesis_report in the paper: headline claim, the result to reproduce, the metric that decides success, known divergence risks (data availability, compute, undocumented hyperparameters).

   The experiment stays open; finish exactly as chronicle-import-repo does — including its step 8: ask the user for an optional kick-off question ("anything specific you want the agents to work on now beyond the reproduction itself?" — e.g. "check whether the result survives on dataset Y") and, if given, post it via chronicle.post_direction_message so the steering agent starts on it; then hand off (experiment URL + next steps).

After the skill completes

Tell the user:

1. The paper: title, arxiv asset id, and whether it was newly registered or already known (existing: true).
2. Which repository was imported and why it was chosen (official vs re-implementation) — or that this was a paper-only import.
3. Everything chronicle-import-repo's hand-off covers (experiment id/URL, code binding, report + prompt ids, next steps).

Failure modes

• No credentials → web UI to mint an API key; agents cannot bootstrap credentials.
• register_publication can't resolve the id — malformed id or arXiv API hiccup. Surface the error; re-check the id/URL with the user.
• No discoverable code → paper-only import (step 2). Don't fabricate a repo pick; if candidates are all unofficial, say so and let the user decide.
• Ambiguous repo candidates → never guess. Present them and confirm.
• Clone fails (private repo, auth wall): the repo isn't publicly clonable; ask the user for access or a mirror, or go paper-only.
• Import-core failures (managed repo not ready, push rejected, dirty clone): handled per chronicle-import-repo's failure modes — bundle fallback, etc.

Requires

• The bundled Chronicle MCP tools (register_publication, link_asset).
• The chronicle-import-repo skill (the shared core) and its requirements — methodic-research SDK + git for the local clone/push path.
• Credentials in ~/.methodic (or CHRONICLE_API_KEY exported).
• Optional: a literature/web search MCP (e.g. Paperclip) for locating the code repo; without it, use the arXiv abstract page and README links.