agentsclimarketplace
Sign in

Ai dataset engineering

Skill ebarti/skills/ai-dataset-engineering

Practical knowledge for building, augmenting, and processing datasets for AI/LLM training and evaluation. Covers data curation (quality, coverage, quantity, acquisition, annotation), data synthesis (rule-based, simulation, AI-powered), instruction data generation, model distillation, and data processing (inspection, deduplication, cleaning, filtering, formatting). Use this skill when: - Curating or sourcing training data - Designing a data annotation pipeline - Generating synthetic data with LLMs - Distilling a smaller model from a larger one - Processing/cleaning datasets for training or evaluation - Estimating dataset size requirementsFrom its SKILL.md

Install
npx -y skills add ebarti/skills --skill ai-dataset-engineering

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.6 KB, 204 tokens by cl100k_base, as published. Nobody here has run it

AI Dataset Engineering

Knowledge from "AI Engineering" by Chip Huyen (Chapter 8). Practical methods for working with training and evaluation datasets.

Quick Start

  1. Check guidelines.md to find which files to load for your task
  2. Load only relevant files (each topic has knowledge.md, rules.md, examples.md)
  3. Apply guidance to your work

Contents

References

CategoryPurpose
data-curationData quality, coverage, quantity, acquisition, annotation
data-synthesisRule-based synthesis, simulation, AI-powered generation, distillation
data-processingInspection, deduplication, cleaning, filtering, formatting

Workflows

TaskWorkflow
Curate, synthesize, and process a training datasetworkflows/build-training-dataset.md

Guidelines

See guidelines.md for task-based file selection.

What ships with it: 14 files

78.2 KB alongside SKILL.md

references/

workflows/

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.