Anakin web data
Use when fetching content from the web with Anakin — reading a page, getting a site's URL structure, or bulk-collecting many pages. Covers scrape, map, and crawl, and routes to the right Anakin tool for a task. Triggers on scraping a URL, extracting page content as markdown, listing a site's pages, ingesting a site, building a corpus from a domain, or handling JS-heavy/SPA pages.From its SKILL.md
npx -y skills add Anakin-Inc/agent-skills --skill anakin-web-dataAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
4.8 KB, as published. Nobody here has run it
Anakin: fetching web data
Pick the right tool first
Anakin has twenty-one tools across five groups. Choosing wrong is the most common and most expensive mistake — check this table before calling anything.
| The task | Tool | Skill |
|---|---|---|
| Read one known URL | scrape | this one |
| Find what pages exist on a site | map | this one |
| Collect many pages from one site | crawl | this one |
| Find pages across the web for a query | search | anakin-research |
| Answer a question needing many sources | agentic_search | anakin-research |
| See what AI engines say about something | ai_visibility_search | anakin-research |
| Extract data from a specific popular site | wire_discover → wire_read_action | anakin-wire |
| Submit a form, add to cart, post content | wire_discover → wire_write_action | anakin-wire |
| Watch a page for changes over time | monitor_create | anakin-monitoring |
| Multi-step interaction no Wire action covers | browser_task | anakin-browser |
| Reach login-protected content | session_list → pass sessionId | anakin-browser |
Check Wire before reaching for scrape or crawl on a well-known site.
Amazon, Walmart, LinkedIn, Airbnb, Zillow and hundreds more have vetted,
pre-built extractors that return clean structured data in one call. Scraping
those sites by hand is slower, costs more, and gives you markdown you then have
to parse. See the anakin-wire skill.
Escalate in cost order. scrape → scrape with useBrowser → a Wire
action → browser_task. Each step is slower and more expensive than the last;
do not start at the end.
scrape — one URL to markdown
Default behavior returns clean markdown. That is the right default; do not add options reflexively.
generateJson: true— also run AI extraction and return typed fields. Use for product pages, listings, articles — anywhere you want structured values rather than prose. Returns the JSON as the primary output.useBrowser: true— render with a stealth headless browser. Slower and more expensive. Only for SPAs and JS-rendered content. Try the default first; if the markdown comes back empty or is missing the content you can see in a browser, retry withuseBrowser: true.country— two-letter proxy egress code, defaults to"us". Set it when a page is geo-restricted or price/inventory varies by market ("de","in", …).forceFresh: true— skip the cache. Results cache for roughly 24h, which is usually what you want. Only force fresh for genuinely live data (stock levels, prices you are about to act on).sessionId/sessionName— a saved browser session for login-protected pages. Pair withuseBrowser: true.
map — discover a site's URLs
Returns internal links, external links, and counts. Use it to understand structure before crawling, so the crawl is scoped instead of blind.
limit(default 100, max 1000) — total URLs returned.depth(default 2, max 5) — link-hops followed.limitPerLevel(default 100) — breadth cap per level.search— keyword filter on path/title. Cheap way to find the section you care about without crawling everything.includeSubdomains,includeExternalLinks— both default false.
crawl — bulk markdown from a site
For catalog ingestion or building a RAG corpus. Returns an array of pages, each with markdown and its own status — check per-page status, since a crawl can partially succeed.
maxPages(default 10, max 500) — a hard cap, and the main cost lever.depth(default 1, max 5).includePatterns/excludePatterns— glob/regex URL filters. Use these. An unscoped crawl burns credits on nav pages, tag archives, and pagination.
Working pattern
For anything beyond a single page, map first, then crawl the subset:
mapthe domain with asearchfilter to see what exists.- Derive
includePatternsfrom the URLs that matter. crawlwith those patterns and a deliberatemaxPages.
This is consistently cheaper and cleaner than crawling from the homepage and filtering afterward.
Cost and failure notes
useBrowseris the biggest cost multiplier — leave it off unless the page needs it.- Every tool returns an error envelope rather than throwing. On failure, read the message: it names the tool and the cause.
- API key comes from
ANAKIN_API_KEY. Get one at https://anakin.io/dashboard — free tier is 300 credits, no card.