Datasketch resource deduplication starter

Skill ma-compbio-lab/SkillFoundry/skills/meta-maintenance/datasketch-resource-deduplication-starter

A framework for discovering, compiling, and validating reusable skills for scientific agents.

Install

npx -y skills add ma-compbio-lab/SkillFoundry --skill datasketch-resource-deduplication-starter

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

0.3 KB, 76 tokens by cl100k_base, as published. Nobody here has run it

datasketch Resource Deduplication Starter

Use this skill to scan a tiny resource registry sample for near-duplicate text records with MinHash LSH.

What This Skill Does

reads a toy JSONL resource list
tokenizes title and summary text
builds MinHash sketches with datasketch
returns candidate duplicate pairs above a configurable threshold

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.