Watercrawl map
Open Source Agentic scrapping skill with 8 hyperfocused skills, to give research subagents web-search on steroids.
npx -y skills add fernandoleyra/watercrawl-skill --skill watercrawl-mapAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Discover all URLs on a domain by following links. Use when asked to map a site, list all pages, or discover the URL structure of a domain.
SKILL.md
1.0 KB, as published. Nobody here has run it
watercrawl-map
Discover all URLs on a domain.
Instructions
-
Parse arguments from
$ARGUMENTS:url— required.max_urls— optional. Default: 50. Maximum: 200.
-
Try sitemap first:
- Fetch
<domain>/sitemap.xmlusing WebFetch - If it returns valid XML with
<url><loc>entries, parse those URLs directly - Report:
Found sitemap with N URLs
- Fetch
-
If no sitemap, crawl for links:
- Use the crawl loop from watercrawl-crawl but collect only URLs, not content
- This is faster — no need to read full page content
-
Return a sorted URL list:
## Site Map: <domain> **URLs discovered:** N **Method:** sitemap / link crawl ### All URLs: 1. https://example.com/ 2. https://example.com/about