Watercrawl scrape

Skill fernandoleyra/watercrawl-skill/skills/watercrawl-scrape

Open Source Agentic scrapping skill with 8 hyperfocused skills, to give research subagents web-search on steroids.

Install

npx -y skills add fernandoleyra/watercrawl-skill --skill watercrawl-scrape

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Scrape a single URL and return its content as clean Markdown, plain text, or raw HTML. Use when asked to fetch, scrape, read, or get the content of a webpage.

SKILL.md

2.3 KB, as published. Nobody here has run it

watercrawl-scrape

Scrape a single URL and return clean content.

Instructions

Parse arguments from $ARGUMENTS:
- url — required. The URL to scrape.
- format — optional. markdown (default), text, or html.
Fetch the page using the WebFetch tool with the provided URL.
Process the response:
- If format=markdown (default): return the content as-is from WebFetch (already Markdown). Clean up navigation artifacts: remove repeated navigation menus, cookie banners, and footer boilerplate visible as plain text blocks.
- If format=text: strip all Markdown formatting (headers, links, bold) and return plain text.
- If format=html: inform the user that raw HTML is not available via WebFetch; offer the Markdown version instead.
Check content quality:
- If the returned content is fewer than 50 words, the page may be JavaScript-rendered. Inform the user: "This page may require JavaScript to render. If you have Playwright installed (npm install -g playwright), I can try a Bash-based approach to get the full content."
- If Playwright is available, offer to use it.
Return the processed content with:
- Source URL: **Source:** <url>
- Word count: **Words:** ~N
- The content body

Playwright fallback (if requested)

Check if Playwright is available:

which playwright 2>/dev/null && echo "available" || npx playwright --version 2>/dev/null || echo "NOT_AVAILABLE"

If available, run:

node -e "
const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  await page.goto('URL_HERE', { waitUntil: 'networkidle' });
  const content = await page.content();
  console.log(content);
  await browser.close();
})();
" 2>/dev/null

Replace URL_HERE with the actual URL. Parse the returned HTML yourself.

Error handling

Network error: report the error and suggest checking the URL
403/404/5xx: report the status and suggest verifying the URL
Timeout: inform the user and suggest retrying