Web scraping

Skill jeongsk/korean-job-hunter/skills/web-scraping

Extract structured information from websites using web_fetch for simple pages and browser automation for dynamic sites, login-gated flows, pagination, infinite scroll, or pages that require clicking/searching/filtering. Use when the user asks to scrape a site, collect listings, gather fields from many pages, monitor website changes, or turn webpage content into structured summaries/JSON/CSV.From its SKILL.md

Install

npx -y skills add jeongsk/korean-job-hunter --skill web-scraping

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

2.3 KB, as published. Nobody here has run it

Web Scraping

Extract data with the lightest reliable method first.

Choose the approach

Use web_fetch for simple public pages when the needed content is already in HTML.
Use browser when the site is dynamic, needs clicking, infinite scroll, filters, tabs, or login/session state.
Use web_search only to discover candidate pages when the target URL is unknown.

Default workflow

Identify the target site and exact fields to collect.
Test one page first.
Decide the extraction method:
- web_fetch for readable article/listing text
- browser snapshot for dynamic DOM inspection
Normalize the output into a stable schema.
If scraping multiple pages, avoid tight loops and serialize requests.
Deduplicate by URL or stable item id.
Save results in the workspace when the task is larger than a quick one-off.

Browser scraping pattern

Open the page.
Take a snapshot.
Interact only as needed: search, click filters, pagination, expand sections.
Re-snapshot after each meaningful state change.
Extract only the fields the user asked for.
Close tabs when finished.

Output guidance

Prefer one of these formats:

concise bullet summary
JSON array of objects
CSV/TSV when the user wants exportable rows

Use explicit keys, for example:

[
  {
    "title": "...",
    "url": "...",
    "source": "...",
    "date": "...",
    "summary": "..."
  }
]

Reliability rules

Do not invent missing fields.
If a site blocks access, say so and switch sources when appropriate.
For news/results pages, capture source + title + link at minimum.
For large jobs, checkpoint partial results to a workspace file.
Prefer fewer larger writes over many tiny writes.

Cleanup

Close browser tabs opened for scraping.
If you create state/output files, store them under the workspace and name them clearly.