agentsclimarketplace

Fetch url

Skill aiskillstore/marketplace/skills/dcjanus/fetch-url

获取并提取链接正文(默认 Markdown);内置 X/Twitter URL 处理,提升受限页面的抓取成功率。From its SKILL.md

Install
npx -y skills add aiskillstore/marketplace --skill fetch-url

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

SKILL.md

2.2 KB, 637 tokens by cl100k_base, as published. Nobody here has run it

在当前文件所在目录运行:./scripts/fetch_url.py URL(仅支持 http / https)。
说明:必须直接当作可执行文件执行。

脚本调用方式示例(不要用 uv run python 或 python):

cd skills/fetch-url && ./scripts/fetch_url.py https://example.com --output ./page.md

错误示例:

uv run python skills/fetch-url/scripts/fetch_url.py https://example.com --output ./page.md
python skills/fetch-url/scripts/fetch_url.py https://example.com --output ./page.md

默认自动探测本地 Chromium 系浏览器路径;未探测到时需安装 Playwright 浏览器:

uv run playwright install chromium

参数:

  • --output:将输出写入文件(默认 stdout)。
  • --timeout-ms:Playwright 导航超时(毫秒,默认 60000)。
  • --browser-path:指定本地 Chromium 系浏览器路径(默认自动探测)。
  • --output-format:输出格式(默认 markdown),支持 csv、html、json、markdown、raw-html、txt、xml、xmltei;raw-html 直接输出渲染后的 HTML(不经 trafilatura)。
  • --fetch-strategy:仅 markdown 可用,支持 auto、agent、jina、browser。默认 auto。

--fetch-strategy 常用值:

  • auto:默认选择。
  • agent:优先用原站 Markdown 协商。
  • jina:优先用 Jina Reader。
  • browser:直接用本地 Playwright。

环境变量:

  • 可设置 JINA_API_KEY 提升 Jina Reader 限流:JINA_API_KEY=your-token ./scripts/fetch_url.py ...

示例:

./scripts/fetch_url.py https://example.com --output ./page.md --timeout-ms 60000
./scripts/fetch_url.py https://example.com --fetch-strategy jina
JINA_API_KEY=your-token ./scripts/fetch_url.py https://example.com --fetch-strategy jina
./scripts/fetch_url.py https://example.com --fetch-strategy browser
./scripts/fetch_url.py https://x.com/jack/status/20 --output-format markdown
./scripts/fetch_url.py https://x.com/jack/status/20 --output-format markdown --fetch-strategy browser

Reference:scripts/fetch_url.py

What ships with it: 2 files

81.3 KB alongside SKILL.md, 1 of them executable

scripts/

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.