Spider

Skill mj-deving/pai-skills/skills/Scraping/Spider

Curated, sanitized export of 21 agent-skill packages for Claude Code and Codex, gated by an automated publication audit (no secrets, no local paths).

Install

npx -y skills add mj-deving/pai-skills --skill Spider

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Low-cost high-scale web scraping and crawling via Spider.cloud API — Rust-based, fast, ~$0.48/1K pages. USE WHEN spider cloud, scale scraping, bulk scrape, bulk crawl, high volume scraping, scrape at scale, affordable crawling, spider.cloud.

SKILL.md

1.5 KB, as published. Nobody here has run it

Customization

Before executing, check for user customizations at: ${PAI_USER_DIR}/SKILLCUSTOMIZATIONS/Spider/

Spider.cloud — Low-Cost Scale Scraping

Rust-based web scraping API. Fastest option for high-volume scraping at ~$0.48 per 1,000 pages. Use when Crawl4AI is too slow or volume exceeds what self-hosted can handle.

When to Use

Scraping 1,000+ pages (scale)
Need speed + reliability at low cost
Crawl4AI too slow for the volume
BrightData too expensive for the use case

When NOT to Use

<100 pages (→ Jina Reader or Crawl4AI, free)
Anti-bot / Cloudflare bypass (→ BrightData)
Social media platforms (→ Apify)
Zero budget (→ Crawl4AI)

Workflow Routing

Request Pattern	Route To
Scale scrape, bulk crawl, spider.cloud	`Workflows/ScaleCrawl.md`

Prerequisites

Spider.cloud account + API key
Set SPIDER_API_KEY in ${AGENT_HOME}/.env

Pricing

~$0.48 per 1,000 pages
Free credits on sign-up
Open-source self-hosted option available (Rust)

Capabilities

JS rendering: Yes
Markdown output: Yes
Multi-page crawling: Yes (primary use case)
Speed: Fastest (Rust-based)
AI extraction: Yes
Anti-bot: Basic