Firecrawl
Expert on Firecrawl — web scraping, crawling, search, and browser automation API for AI agents. Use when the user wants to scrape websites, crawl pages, search the web, extract structured data, or convert URLs to LLM-ready content. Triggers on mentions of firecrawl, web scraping, crawling, site mapping, /scrape, /search, /crawl, /map, /interact.From its SKILL.md
npx -y skills add zot24/skills --skill firecrawlAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
- 2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
4.0 KB, as published. Nobody here has run it
Firecrawl
Firecrawl turns any website into clean, LLM-ready markdown, structured data, or screenshots.
Overview
/scrape— Extract markdown/HTML/JSON from a single URL/search— Query-based web discovery with optional content extraction/crawl— Recursively gather content from entire sites/map— Discover all URLs on a website/interact— Browser automation (clicks, forms, navigation)/extract— LLM-powered structured data extraction with schemas
Install
npx -y firecrawl-cli@latest init --all --browser
This installs CLI tools, build skills, and opens browser auth for API key setup.
Quick Start
from firecrawl import FirecrawlApp
app = FirecrawlApp(api_key="fc-...")
result = app.scrape_url("https://example.com", params={"formats": ["markdown"]})
print(result["markdown"])
import FirecrawlApp from '@mendable/firecrawl-js';
const app = new FirecrawlApp({ apiKey: 'fc-...' });
const result = await app.scrapeUrl('https://example.com', { formats: ['markdown'] });
Choose Your Path
- Live web data now — Use CLI:
firecrawl search,firecrawl scrape - App integration — SDK +
FIRECRAWL_API_KEYin.env - Need API key — Run install with
--browseror visit firecrawl.dev/app - REST API directly —
POST https://api.firecrawl.dev/v2/scrapewith Bearer token
Core Concepts
Scrape vs Crawl vs Map: Scrape extracts one page. Crawl recursively follows links. Map discovers URLs without extracting content.
Output Formats: markdown (default), html, rawHtml, screenshot, links, extract (structured JSON via LLM).
LLM Extraction: Pass a JSON schema to /scrape or /extract to get structured data back.
Documentation
- Quickstart — Installation and first API call
- Scrape — Single-page extraction
- Search — Web search with content hydration
- Crawl — Recursive site crawling
- Map — URL discovery
- Interact — Browser actions on live pages
- Batch Scrape — Batch operations
- Agent — Autonomous web data gathering
- Extract — LLM structured extraction
- SDKs — Node, Python, Go, Rust, Java, Elixir
- MCP Server — MCP integration for Claude/Cursor
- Self-Hosting — Deploy your own Firecrawl
- Webhooks — Event notifications
- API Reference — Auth, rate limits, errors
- Upstream README — Full project documentation
Common Workflows
Scrape a page to markdown
curl -X POST https://api.firecrawl.dev/v2/scrape \
-H "Authorization: Bearer fc-..." \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown"]}'
Search and extract
results = app.search("latest AI news", params={"limit": 5})
for r in results["data"]:
print(r["url"], r["markdown"][:200])
Upstream Sources
- Repository: https://github.com/mendableai/firecrawl
- Documentation: https://docs.firecrawl.dev
- Docs Index: https://docs.firecrawl.dev/llms.txt
- Skills Repo: https://github.com/firecrawl/skills
Sync & Update
When user runs sync: fetch latest from upstream sources, update docs/ files.
When user runs diff: compare current vs upstream, report changes.