Generate web skills

Generate Web Skills

Learn a website by watching the user do something once, then do a variation of it yourself — and optionally bottle that action into a standalone skill. The skill has four phases; every phase is a script in this skill's scripts/ directory, and you (the model) are the reasoning glue between them.

flowchart LR
  subgraph teach [1. Teach]
    Rec["record_session.sh<br/>user drives Chrome"] --> Har["session.har + actions.js"]
  end
  subgraph distill [2. Distill]
    Har --> Proc["process_har.py"] --> Lesson["lesson.json + LESSON.md"]
  end
  subgraph apply [3. Apply]
    Lesson --> Api["API replay<br/>(new params)"]
    Lesson --> Ui["replay_ui.sh<br/>(mp4 proof)"]
  end
  subgraph gen [4. Generate skill]
    Lesson --> GS["generate_skill.py"]
    Har --> GS
    GS --> Skill["standalone skill<br/>(embedded HAR)"]
  end

Lessons live outside the repo at ~/.web-lessons/<host>/<lesson-name>/. HARs contain live cookies and tokens — never commit, print, or upload them (see Safety).

Prerequisites

• Node.js + npx (Playwright codegen and the UI replay runner).
• Google Chrome installed (default channel is chrome; set PW_CHANNEL=chromium to use Playwright's bundled browser instead, or PW_CHANNEL=msedge).
• python3 (HAR distiller — stdlib only, no pip installs).
• ffmpeg on PATH (only for Phase 3 UI replay: webm → mp4).

First run of record_session.sh / replay_ui.sh installs the playwright npm package into scripts/node_modules automatically. replay_ui.sh also fetches Playwright's own bundled ffmpeg (needed for video capture, separate from the system ffmpeg used for the webm-to-mp4 step), and for PW_CHANNEL=chromium the bundled browser. If the machine has no network, install ahead of time from scripts/ (npm install, then npx playwright install ffmpeg [chromium]).

Workflow

Copy this checklist and track progress:

- [ ] 1. Confirm the exact action the user will demonstrate + the start URL
- [ ] 2. Record: run record_session.sh, tell the user what to do, wait for close
- [ ] 3. Distill: run process_har.py, read LESSON.md, report actions learned
- [ ] 4. Confirm the variation the user wants (new params / different flow)
- [ ] 5. Apply: API replay (data) or replay_ui.sh (mp4 proof)
- [ ] 6. Report results; never print cookies/tokens

To instead package the recording as a reusable skill, do steps 1–3, then:

- [ ] 4b. Confirm skill name + install target (personal / project / skills.sh)
- [ ] 4c. If the action needs a login, ask (per detected account) whether to add
         a per-user setup step (--list-auth-accounts, then --with-setup)
- [ ] 5b. Generate: run generate_skill.py; relay any secret-scan warnings
- [ ] 6b. Refine the auto-drafted SKILL.md triggers; report the skill path

Resolve the skill directory (wherever it was installed) once:

SKILL_DIR="<the folder this SKILL.md lives in>"   # e.g. .cursor/skills/generate-web-skills

Phase 1 — Record (teach mode)

Pin down one concrete action with the user first (source/target, dates, search terms — whatever the variation will later change), then launch:

bash "$SKILL_DIR/scripts/record_session.sh" <lesson-name> <start-url>
# e.g.
bash "$SKILL_DIR/scripts/record_session.sh" flight-availability "https://www.example-air.com"

Then tell the user, in plain words, exactly what to do in the Chrome window that opens, e.g.:

Chrome is open. Please search a one-way flight from LAX to JFK departing 2026-08-14, then close the window when the results show.

The script:

• Opens Chrome via playwright codegen with a persistent profile (~/.web-lessons/.browser-profile) so logins carry across sessions (Chrome 136+ blocks automation of the default profile, so this uses a dedicated one).
• Writes session.har (all network activity) and actions.js (the UI steps, --target javascript) into the lesson directory.
• Writes meta.json (lesson name, start URL, host, timestamp).

Wait for the user to finish and close the browser before moving on — the HAR is flushed on close. The script prints the lesson directory path; capture it as LESSON_DIR.

Phase 2 — Distill

python3 "$SKILL_DIR/scripts/process_har.py" "$LESSON_DIR"

This reads session.har (+ actions.js, meta.json), drops noise (static assets, analytics/telemetry hosts), keeps XHR/fetch/document requests, and writes:

• lesson.json — machine-readable: endpoints grouped by method + path template, example payloads, detected parameter candidates (the knobs to turn — dates, IATA-like codes, ids, pagination, search terms), and the auth surface (cookie/header names required, values redacted).
• LESSON.md — human-readable summary of the same.

Read LESSON.md and report to the user what actions were learned — the endpoints, what each seems to do, and which parameters can vary. This is the answer to "what did you learn to do". See REFERENCE.md for the schema and the parameter/auth heuristics.

Phase 3 — Apply a variation

Confirm the variation (e.g. "same search, but SFO→SEA on 2026-09-02"), then pick a mode:

Want	Mode	Why
Data for a different query (availability, prices, listings)	API replay	Fast, deterministic; no browser
Visual proof of a UI flow, or the app is JS-heavy / hard to call directly	UI replay	Produces an mp4; drives the real UI

API replay

From lesson.json, pick the endpoint that produced the result (usually the XHR/fetch whose response held the data). Reissue it with substituted parameters, reusing the session:

• Take cookies/headers from session.har for that host — read them from the file at call time; never echo them into chat or logs.
• Swap the param_candidates you want to vary (dates, origin/destination, ids).
• Use curl or python requests. Prefer this only for idempotent reads (GET/search). For anything that mutates state, see Safety.

Then interpret the response for the user (parse JSON, summarize the answer).

UI replay with mp4 proof

1. Read actions.js (the recorded steps) and write a variant module at $LESSON_DIR/variant.js that exports a single async function taking page, with your substituted inputs. Contract:

module.exports = async (page) => {
  await page.goto('https://www.example-air.com');
  await page.getByLabel('From').fill('SFO');
  await page.getByLabel('To').fill('SEA');
  await page.getByLabel('Depart').fill('2026-09-02');
  await page.getByRole('button', { name: 'Search' }).click();
  await page.waitForSelector('.results');
};

(Lift the await page.... lines from actions.js, change the inputs, and drop the browser-launch boilerplate — the runner supplies page.)

2. Run it; the runner reuses the persistent profile, records video, and converts it to mp4:

bash "$SKILL_DIR/scripts/replay_ui.sh" "$LESSON_DIR" "$LESSON_DIR/variant.js"

3. It prints proof.mp4 under $LESSON_DIR/runs/<timestamp>/. Hand that path to the review-mp4 skill to verify the flow and describe what happened, and embed/link the mp4 for the user.

Phase 4 — Generate a standalone skill

Turn the recording into a self-contained, shareable skill that performs one inferred action. The generated skill embeds a trimmed HAR (the auth + prerequisite + result chain for that action) plus its own replay scripts, so it runs without this skill or the original ~/.web-lessons/ lesson.

Confirm with the user first: the primary action to package, the skill name, and the install target (cursor-personal → ~/.cursor/skills/, cursor-project → ./.cursor/skills/, or skills-sh → a repo folder). Then:

python3 "$SKILL_DIR/scripts/generate_skill.py" "$LESSON_DIR" \
  --name flight-search --format cursor-personal
# or target an explicit path:
python3 "$SKILL_DIR/scripts/generate_skill.py" "$LESSON_DIR" \
  --output ~/.cursor/skills/flight-search

What it does (each step is its own script, see REFERENCE.md):

1. infer_flow.py picks the primary endpoint (top-ranked non-mutating, data-bearing call; override with --endpoint "GET host/path") and walks the HAR to collect every prerequisite request up to and including it → flow.json.
2. scan_secrets.py scans the HAR for credentials. It reports names and locations only — never values. If it finds CRITICAL material (payment cards, plaintext passwords/CVV/SSN, private keys), generate_skill.py stops. Tell the user what kind was found and where (never the value); re-run with --allow-critical only after they acknowledge, or ask them to re-record without entering that data.
3. trim_har.py writes the flow-scoped data/session.har (re-indexed to match flow.json).
4. It copies the replay tooling, then drafts SKILL.md, REFERENCE.md, and SECURITY.md.

Then you finish it:

• Confirm the drafted action_label matches what the user demonstrated.
• Tune the SKILL.md description triggers (site name, action verbs, example params) so the skill is discovered when the user wants that action.
• If a critical pre-call was dropped, re-run with --endpoint or hand-edit data/flow.json.
• Relay the secret-scan summary to the user, and point them at SECURITY.md before they share the skill.

Optional — per-user login (multi-account setup)

By default the generated skill embeds the recorder's own login. If the action involves signing in and you want each installer to run it as themselves (e.g. their own Linear, plus an SSO provider), offer a setup step instead. This strips the selected hosts' credentials from the shipped HAR and ships a setup.sh that captures each user's own session on first run.

A single action can need more than one logged-in service, so detect the accounts first, then ask:

# lists every host in the flow that carries auth (names/hosts only, no values)
python3 "$SKILL_DIR/scripts/generate_skill.py" "$LESSON_DIR" --list-auth-accounts

Ask the user (only when a login is part of the action) whether they want a per-user setup step, and for which of the detected accounts (all, some, or none). Then generate accordingly:

# per-user setup for ALL detected accounts:
python3 "$SKILL_DIR/scripts/generate_skill.py" "$LESSON_DIR" \
  --name linear-create-issue --format cursor-personal --with-setup
# or only specific hosts (others keep the recorder's embedded login):
python3 "$SKILL_DIR/scripts/generate_skill.py" "$LESSON_DIR" \
  --name linear-create-issue --with-setup --setup-hosts linear.app

With setup enabled the shipped data/session.har carries no login for the chosen hosts (values replaced with placeholders); data/auth_accounts.json records what setup must fill. Each user runs bash scripts/setup.sh once — a browser opens per service, they log in, and their session is saved locally to data/user-auth.<host>.har (gitignored). Replay overlays that per host and refuses a setup host until its login is captured. Default (no --with-setup) keeps the embedded login, exactly as before.

The generated skill replays with python3 scripts/replay_api.py --set knob=value (reads run autonomously; mutating flows need --confirm-mutating) or, for a UI flow, bash scripts/replay_ui.sh (mp4 proof, using the embedded cookies). The original lesson can be archived afterward.

Safety

• Secrets: session.har, the browser profile, and variant.js can contain cookies, bearer tokens, and personal data. Never commit them, never paste their contents into chat, never upload them anywhere. lesson.json/LESSON.md redact credential values but still name what's required — treat them as sensitive too. Everything stays under ~/.web-lessons/, outside any repo.
• Only replay reads autonomously. GET/search-style calls that just fetch data are fine to vary on your own. Stop and ask the user before replaying anything that mutates state — bookings, purchases, posts, deletes, form submissions, or any non-idempotent request. Recording such a flow is fine; re-firing it is not, without explicit confirmation.
• Stay on the user's own session. This skill reuses the session the user established; it does not harvest credentials or bypass auth. Respect the target site's terms.
• One lesson, one host. Don't reuse a lesson's payloads against a different site.
• Generated skills embed live credentials. A Phase 4 skill's data/session.har carries the recorded session's cookies/tokens. Its SECURITY.md spells out the rules: safe to share privately with people already trusted with that account, but never publish publicly without first rotating/expiring the credentials. Always relay the secret scan; never echo a value. Do not proceed past a CRITICAL finding without the user's acknowledgment.

Anti-patterns

• Printing or committing session.har, cookies, or tokens — they're live credentials.
• Replaying a state-changing request (booking/purchase/delete) without asking.
• Reading raw HAR entries by hand instead of running process_har.py — you'll drown in static assets and miss the parameter knobs.
• Hardcoding one endpoint's payload for a different site or a different action.
• Skipping the "tell the user exactly what to demonstrate" step — a vague recording yields a vague lesson.
• Publishing a generated skill publicly without rotating the embedded session, or ignoring a CRITICAL secret-scan finding.

Resources

• HAR anatomy, the lesson.json and flow.json schemas, parameter-detection and auth heuristics, generated-skill layout, the secret-scan severity levels, and video-conversion details: REFERENCE.md
• Verifying and describing the replay mp4: the review-mp4 skill.