Data quality platforms and rule management
Skill vaquarkhan/data-engineering-agent-skills/skills/data-quality-platforms-and-rule-management
Production-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-quality-platforms-and-rule-managementAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 21 stars21 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Guides agents through data-quality operating models and tool selection. Use when designing rule portfolios, severity levels, ownership, evidence, and enforcement across dbt tests, Great Expectations, Deequ, Cuallee, Soda, warehouse-native checks, and platform monitoring workflows.
SKILL.md
3.1 KB, as published. Nobody here has run it
Data Quality Platforms And Rule Management
Overview
Use this skill when the question is not only what checks to write, but how the quality program should operate across tools, teams, and publish stages. It helps agents design rule ownership, severity, evidence, and enforcement across multiple quality frameworks.
When to Use
- selecting or combining data-quality tools
- designing rule portfolios and severity models
- aligning
dbttests,Great Expectations,Deequ,Cuallee,Soda, or warehouse-native checks - deciding what blocks publish versus what raises warnings
- improving long-term maintainability of data-quality controls
Do not assume more checks automatically improve quality. The operating model matters as much as the framework.
Workflow
-
Define the quality operating model. Clarify:
- who owns rules
- who triages failures
- what blocks publish
- what becomes an alert or trend signal
-
Group rules by purpose. Typical groups:
- contract correctness
- completeness and freshness
- distribution and anomaly checks
- reconciliation and financial controls
- governance or regulated-data controls
-
Choose tools intentionally. Decide where each type of rule belongs:
dbttests for warehouse-native model validationGreat Expectations,Deequ,Cuallee, orSodafor reusable framework-based checks- warehouse-native monitors for platform-local health signals
-
Define evidence and routing. Require:
- actionable output
- severity and ownership
- trend visibility
- links to incident and publish workflows
-
Control portfolio growth. Review overlapping, stale, noisy, or low-value checks so quality stays credible and maintainable.
Common Rationalizations
| Rationalization | Reality |
|---|---|
| "We should standardize on one tool for everything." | Different rule types often fit different execution surfaces and operating models. |
| "If a rule fails, we can decide the impact later." | Publish and incident behavior must already be defined when the rule is introduced. |
| "More rules always mean better quality." | Noisy or duplicate checks reduce trust and slow triage. |
Red Flags
- rule ownership and triage paths are unclear
- tools are chosen by preference instead of workload fit
- blocking and non-blocking checks are mixed without severity logic
- evidence is hard to review or trend over time
- stale, overlapping, or low-signal rules accumulate unchecked
Verification
- The quality operating model defines ownership and severity clearly
- Rule groups map to real delivery risks and publish stages
- Tool choices fit execution surfaces and maintenance needs
- Evidence and routing are reviewable and actionable
- Portfolio sprawl is controlled over time