Url
Fetch and extract text from web URLsFrom its SKILL.md
npx -y skills add axoviq-ai/synthadoc --skill urlAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- runs commandsInstructs the agent to run 2 commands, including `pip install httpx beautifulsoup4` and 1 more.
- fetches URLsInstructs the agent to fetch 1 URL, including https://example.com/article.
What its file declares
Copied from the file, not written here
The file declares its own license as AGPL-3.0-or-later. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.
SKILL.md
1.5 KB, 290 tokens by cl100k_base, as published. Nobody here has run it
URL Skill
Fetches a web URL using httpx, strips navigation/script/style tags with
BeautifulSoup, and returns clean body text. PDF URLs are extracted with
pypdf (primary) and pdfminer.six (fallback).
Setup
pip install httpx beautifulsoup4
# Optional — needed only if you ingest PDF URLs:
pip install pypdf pdfminer.six
Standalone usage
import asyncio
from synthadoc.skills.url.scripts.main import UrlSkill
skill = UrlSkill()
async def main():
result = await skill.extract("https://example.com/article")
print(result.text) # clean body text
print(result.metadata) # {"url": "https://..."}
asyncio.run(main())
DomainBlockedException is raised when the site returns HTTP 401, 403, or
429. Catch it to log and skip the domain:
from synthadoc.skills.base import DomainBlockedException
try:
result = await skill.extract(url)
except DomainBlockedException as e:
print(f"Blocked: {e.domain} (HTTP {e.status_code})")
When this skill is used
- Source starts with
https://orhttp:// - User intent contains:
fetch url,web page,website
What ships with it: 3 files
5.5 KB alongside SKILL.md, 2 of them executable
scripts/
- __init__.pyruns0 B
- main.pyruns5.5 KB
- requirements.txt90 B