agentsclimarketplace

Data catalog and discovery

Skill vaquarkhan/data-engineering-agent-skills/skills/data-catalog-and-discovery

Production-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.

Install
npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-catalog-and-discovery

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 21 stars21 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Guides agents through data catalog, discovery, and metadata quality workflows. Use when publishing datasets, improving discoverability, curating lineage metadata, or making data products easier for other teams to find and trust.

SKILL.md

2.3 KB, as published. Nobody here has run it

Data Catalog And Discovery

Overview

Use this skill when the challenge is not only building data, but making it understandable and discoverable. It helps agents treat metadata, ownership, lineage, and usage context as delivery artifacts instead of afterthoughts.

When to Use

  • publishing a new shared dataset
  • improving catalog metadata quality
  • curating lineage, tags, or ownership information
  • reducing duplicate datasets created because teams cannot find trusted ones

Do not stop at filling in a title and description. Discovery quality requires operational context too.

Workflow

  1. Define the discovery contract. Include:

    • owner
    • business description
    • technical description
    • grain
    • freshness expectation
    • intended consumers

  2. Link the asset to its lineage. Show upstream sources, transformation layers, and major downstream uses where possible.

  3. Add trust signals. Typical signals:

    • quality status
    • SLA or freshness status
    • certification or review state
    • deprecation state

  4. Tag for real discovery, not taxonomy theater.

  5. Revisit metadata when the contract changes.

Common Rationalizations

RationalizationReality
"The table name is descriptive enough."Names alone do not explain grain, trust, or ownership.
"We can catalog it after people start using it."Poor discovery usually leads to duplicate local copies first.
"Lineage is a platform problem, not a delivery problem."Producers know the business meaning and must help make lineage useful.

Red Flags

  • shared datasets have no owner or description
  • certified and experimental assets are indistinguishable
  • metadata is copied from schema names without business meaning
  • deprecation state is absent for old assets

Verification

  • Ownership, description, grain, and freshness are documented
  • Lineage or source context is attached
  • Trust signals exist for consumers
  • Discovery metadata is updated when the contract changes

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.