Tavily crawl

Skill arch3rPro/ark-space/skills/tavily-crawl

ArkSpace is a creative workspace for orchestrating agent skills, roles, and workflows across Claude Code and Code

Install

npx -y skills add arch3rPro/ark-space --skill tavily-crawl

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use when crawling a website section through Tavily Crawl to collect content from many pages, download docs, bulk extract pages, or apply path/domain filters with extraction controls.

SKILL.md

3.7 KB, as published. Nobody here has run it

Tavily Crawl

Use Tavily Crawl when ArkSpace needs extracted content from many pages under a known site or site section.

Tavily configuration is shared with all Tavily skills through provider-manager.

Source References

Official docs: https://docs.tavily.com/documentation/agent-skills
API reference: https://docs.tavily.com/documentation/api-reference/endpoint/crawl
Official skills: https://github.com/tavily-ai/skills/tree/main/skills/tavily-crawl

Before Use

Resolve the installed ArkSpace package root before running commands. Replace <installed-arkspace-path> with the directory two levels above this loaded SKILL.md, such as /Users/<user>/.claude/plugins/cache/ark-space/ark-space/0.1.2. Use the installed package path, not a repository-relative command.

Check configuration:

python3 <installed-arkspace-path>/scripts/arkspace.py provider check tavily --capability web_crawl

Configure Tavily once:

python3 <installed-arkspace-path>/scripts/arkspace.py provider setup tavily --wizard

Missing Configuration Recovery

If the provider check reports a missing Tavily API key or missing Tavily capability:

Ask the user whether to start setup now: "Tavily is not configured. Should I start the ArkSpace setup wizard now?"
Present exactly two choices:
- "Start setup wizard" - start interactive setup with python3 <installed-arkspace-path>/scripts/arkspace.py provider setup tavily --wizard.
- "Not now" - leave Tavily unconfigured; if the user still wants results, ask whether to continue with a clearly labeled non-ArkSpace fallback.
Run the setup command for the user only when the host shell can provide interactive secret input. If the shell is non-interactive, do not run --wizard through that tool.
In Claude Code non-interactive tool sessions, tell the user the wizard needs interactive secret input and offer:
- run ! python3 <installed-arkspace-path>/scripts/arkspace.py provider setup tavily --wizard in the Claude prompt or terminal, or
- paste the API key in chat and let the agent save it with python3 <installed-arkspace-path>/scripts/arkspace.py provider setup tavily --save-secret TAVILY_API_KEY --secret-stdin.
Re-run the provider check after setup.
If setup succeeds, rerun or tell the user to rerun the original invocation, such as /ark-space:tavily-crawl <url>.
Do not return Tavily Crawl results until the provider check succeeds.
If the user declines, defers, or cannot complete setup and still wants results, ask whether to continue with a clearly labeled non-ArkSpace fallback.

Helper Script

Basic crawl:

python3 <installed-arkspace-path>/scripts/arkspace.py site crawl --provider tavily "https://docs.example.com" --output json

Conservative documentation crawl:

python3 <installed-arkspace-path>/scripts/arkspace.py site crawl --provider tavily "https://docs.example.com" \
  --max-depth 1 \
  --limit 20 \
  --select-paths "/docs/.*" \
  --output json

Semantic crawl for agent context:

python3 <installed-arkspace-path>/scripts/arkspace.py site crawl --provider tavily "https://docs.example.com" \
  --instructions "Find authentication and API key setup docs" \
  --chunks-per-source 3 \
  --output json

Routing Notes

Use tavily-map before crawling when the site structure is unknown.
Use tavily-extract instead when only one or a few exact URLs are needed.
Start with small --limit and shallow --max-depth; increase only when needed.