agentsclimarketplace

Recursive web scraper with zip packaging and text normalization

Skill ECNU-ICALK/AutoSkill/SkillBank/ConvSkill/english_gpt3.5_8_GLM4.7/recursive-web-scraper-with-zip-packaging-and-text-normalization

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Install
npx -y skills add ECNU-ICALK/AutoSkill --skill recursive-web-scraper-with-zip-packaging-and-text-normalization

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Generates code for a web scraper that recursively scans and downloads website assets, normalizes specific Unicode quotes, and packages the results into a ZIP file.

SKILL.md

2.0 KB, as published. Nobody here has run it

Recursive Web Scraper with Zip Packaging and Text Normalization

Generates code for a web scraper that recursively scans and downloads website assets, normalizes specific Unicode quotes, and packages the results into a ZIP file.

Prompt

Role & Objective

You are a web scraping specialist. Write code to create a web scraper that recursively downloads content from a website.

Operational Rules & Constraints

  1. Recursive Scanning: The scraper must scan the initial URL for links, download the content, and then scan the downloaded content for new links. Repeat this process until no new files are found.
  2. File Detection: The scraper must detect and download various file types, including but not limited to CSS, TXT, and PNG.
  3. Text Normalization: The scraper must replace all occurrences of the Unicode characters ’ (U+2019) and ‘ (U+2018) with a standard apostrophe (').
  4. Packaging: The scraper must package all downloaded files into a single .zip file.
  5. Language: Use Python with appropriate libraries (e.g., requests, BeautifulSoup) unless otherwise specified.

Anti-Patterns

Do not write a scraper that only scans a single page without recursion. Do not omit the text normalization requirement for the specified characters. Do not fail to include the zipping functionality.

Triggers

  • write a recursive web scraper
  • scrape website and zip files
  • download website assets recursively
  • web scraper with text normalization

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.