Fetch url
获取并提取链接正文(默认 Markdown);内置 X/Twitter URL 处理,提升受限页面的抓取成功率。From its SKILL.md
npx -y skills add aiskillstore/marketplace --skill fetch-urlAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
SKILL.md
2.2 KB, 637 tokens by cl100k_base, as published. Nobody here has run it
在当前文件所在目录运行:./scripts/fetch_url.py URL(仅支持 http / https)。
说明:必须直接当作可执行文件执行。
脚本调用方式示例(不要用 uv run python 或 python):
cd skills/fetch-url && ./scripts/fetch_url.py https://example.com --output ./page.md
错误示例:
uv run python skills/fetch-url/scripts/fetch_url.py https://example.com --output ./page.md
python skills/fetch-url/scripts/fetch_url.py https://example.com --output ./page.md
默认自动探测本地 Chromium 系浏览器路径;未探测到时需安装 Playwright 浏览器:
uv run playwright install chromium
参数:
--output:将输出写入文件(默认 stdout)。--timeout-ms:Playwright 导航超时(毫秒,默认 60000)。--browser-path:指定本地 Chromium 系浏览器路径(默认自动探测)。--output-format:输出格式(默认markdown),支持csv、html、json、markdown、raw-html、txt、xml、xmltei;raw-html直接输出渲染后的 HTML(不经 trafilatura)。--fetch-strategy:仅markdown可用,支持auto、agent、jina、browser。默认auto。
--fetch-strategy 常用值:
auto:默认选择。agent:优先用原站 Markdown 协商。jina:优先用 Jina Reader。browser:直接用本地 Playwright。
环境变量:
- 可设置
JINA_API_KEY提升 Jina Reader 限流:JINA_API_KEY=your-token ./scripts/fetch_url.py ...
示例:
./scripts/fetch_url.py https://example.com --output ./page.md --timeout-ms 60000
./scripts/fetch_url.py https://example.com --fetch-strategy jina
JINA_API_KEY=your-token ./scripts/fetch_url.py https://example.com --fetch-strategy jina
./scripts/fetch_url.py https://example.com --fetch-strategy browser
./scripts/fetch_url.py https://x.com/jack/status/20 --output-format markdown
./scripts/fetch_url.py https://x.com/jack/status/20 --output-format markdown --fetch-strategy browser
Reference:scripts/fetch_url.py
What ships with it: 2 files
81.3 KB alongside SKILL.md, 1 of them executable
scripts/
- fetch_url.pyruns22.9 KB
- skill-report.json58.4 KB