agentsclimarketplace

Warehouse performance and cost optimization

Skill vaquarkhan/data-engineering-agent-skills/skills/warehouse-performance-and-cost-optimization

Production-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.

Install
npx -y skills add vaquarkhan/data-engineering-agent-skills --skill warehouse-performance-and-cost-optimization

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 21 stars21 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Guides agents through warehouse performance and cost decisions. Use when optimizing BigQuery, Snowflake, Redshift, Athena, Synapse, or lakehouse query patterns, storage layout, and workload isolation.

SKILL.md

2.2 KB, as published. Nobody here has run it

Warehouse Performance And Cost Optimization

Overview

Use this skill when data is correct but too slow or too expensive. It helps agents treat performance and cost as measurable design concerns rather than guesswork.

When to Use

  • slow warehouse queries
  • runaway compute or scan cost
  • poor partitioning or clustering choices
  • overloaded workloads competing on shared compute
  • repeated complaints about expensive marts or dashboards

Do not optimize blindly. Start from observed cost or performance signals.

Workflow

  1. Identify the real bottleneck. Measure:

    • scan volume
    • slot or warehouse usage
    • partition pruning
    • join behavior
    • concurrency patterns

  2. Classify the problem. Common buckets:

    • physical design
    • SQL pattern
    • workload isolation
    • storage layout
    • refresh frequency

  3. Fix the cheapest high-impact issue first. Examples:

    • partitioning
    • clustering
    • pre-aggregation
    • materialization change
    • compute right-sizing

  4. Keep business correctness stable while optimizing.

  5. Record the trade-off. Faster is not always cheaper, and cheaper is not always acceptable.

Common Rationalizations

RationalizationReality
"We just need a bigger warehouse."More compute often hides poor layout or query design.
"Optimization can wait until later."Cost debt compounds quickly in shared platforms.
"The query only runs once a day."Expensive daily jobs can still be major recurring waste.

Red Flags

  • performance work starts with no baseline measurement
  • cost issues are blamed on the platform alone
  • optimizations change business logic without validation
  • the same expensive pattern repeats across many models

Verification

  • Baseline cost or performance metrics exist
  • The bottleneck category is identified
  • The change preserves correctness while improving performance or cost
  • The resulting trade-off is documented

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.