agentsclimarketplace

Healthcare data quality profiling

Skill rbr7/MedClawMini/skills/healthcare-data-quality-profiling

A focused, production-minded library of 197 clinical-AI and healthcare data-science skills for the OpenClaw agent platform featuring data quality, clinical NLP, big-data ML, explainable AI, drug safety, and regulatory.

Install
npx -y skills add rbr7/MedClawMini --skill healthcare-data-quality-profiling

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Profile and score the data quality of healthcare datasets (claims, eligibility/membership, provider rosters, EHR extracts). Measures completeness, validity, consistency, uniqueness, conformity, and timeliness; detects schema drift, outliers, and referential-integrity breaks; emits a versioned data-quality scorecard and a machine-readable rules suite. Use when assessing the trustworthiness of a healthcare data feed, building data-quality gates in an ingestion pipeline, or quantifying "bad data" before it reaches downstream models.

The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

5.8 KB, as published. Nobody here has run it

Healthcare Data Quality Profiling

Overview

Bad data is the dominant cost driver in US healthcare operations. This skill turns a raw healthcare feed into a quantified, explainable data-quality (DQ) scorecard so that data issues are caught before they corrupt analytics, payment, or ML pipelines. It is built around the six classic DQ dimensions and produces both a human-readable report and a reusable, machine-checkable rule suite (Great Expectations / pandera) that can be wired into CI or a Spark ingestion job.

When to Use This Skill

  • Onboarding a new claims, eligibility/membership, provider, or EHR feed and you need to know whether it can be trusted.
  • Standing up an automated data-quality gate that blocks or quarantines bad batches.
  • Quantifying the financial/operational impact of "bad data" for a stakeholder.
  • Diagnosing why a downstream model degraded (often upstream schema drift or a completeness cliff).
  • Producing audit-ready DQ evidence for a payer, provider, or regulator.

The Six DQ Dimensions (how each is measured)

  1. Completeness null / blank / sentinel rate per field; conditional completeness (e.g., discharge_date required when claim_type = inpatient).
  2. Validity values conform to type, range, regex, or code set (NPI is 10 digits and passes the Luhn check; ICD-10 matches the official code list; gender in allowed set).
  3. Consistency cross-field and cross-table logic holds (service_date <= paid_date; member_age consistent with date_of_birth; sum of line items equals claim total).
  4. Uniqueness primary keys are unique; duplicate-record rate (hand off true entity matching to patient-record-entity-resolution).
  5. Conformity formats and units are standardized (dates ISO-8601, currency scale, provider IDs normalized).
  6. Timeliness data freshness / lag vs. SLA; late-arriving partitions.

Workflow

  1. Ingest & sample the dataset (CSV/Parquet/DB). For very large feeds, profile a stratified sample, then promote the rule suite to the full data in Spark.
  2. Auto-profile every column (dtype, cardinality, null rate, min/max, top values, distribution) using the bundled scripts/profile_dataset.py.
  3. Apply a healthcare rule pack NPI Luhn validation, ICD-10/CPT/HCPCS membership, date-order logic, age/DOB consistency, currency sanity, member-ID format.
  4. Detect schema drift by diffing the current profile against a stored baseline (new/removed columns, dtype changes, null-rate shifts, distribution shifts via PSI).
  5. Score each dimension 0–100, weight into an overall DQ index, and grade A–F.
  6. Emit artifacts: a Markdown/HTML scorecard, a failed_records.parquet quarantine set with the failing rule per row, and a Great Expectations suite for reuse.

Example

# scripts/profile_dataset.py (excerpt)  pandas + Great Expectations
import pandas as pd, great_expectations as gx

df = pd.read_parquet("claims_2026Q1.parquet")
ctx = gx.get_context()
batch = ctx.sources.pandas_default.read_dataframe(df)

# Healthcare validity rules
batch.expect_column_values_to_match_regex("npi", r"^\d{10}$")
batch.expect_column_values_to_be_in_set("claim_status", ["PAID","DENIED","PENDING","REVERSED"])
batch.expect_column_values_to_not_be_null("member_id")
batch.expect_column_pair_values_a_to_be_greater_than_b("paid_date", "service_date")

results = batch.validate()
score = 100 * results.statistics["successful_expectations"] / results.statistics["evaluated_expectations"]
print(f"Validity score: {score:.1f}")

# Population Stability Index for distribution / schema drift
import numpy as np
def psi(expected, actual, bins=10):
    q = np.quantile(expected, np.linspace(0,1,bins+1))
    e = np.histogram(expected, q)[0]/len(expected) + 1e-6
    a = np.histogram(actual,   q)[0]/len(actual)  + 1e-6
    return np.sum((a-e)*np.log(a/e))     # >0.25 => material drift

Outputs

  • dq_scorecard.md / .html per-dimension scores, overall index, grade, top offending fields, and trend vs. baseline.
  • expectations_suite.json reusable Great Expectations / pandera rules.
  • failed_records.parquet quarantined rows annotated with the violated rule.
  • schema_drift.json added/removed/changed columns and PSI per numeric field.

Healthcare Context

Tuned for payer/provider data: NPI, member/subscriber IDs, ICD-10-CM, CPT/HCPCS, revenue codes, place-of-service, and eligibility spans. Pairs naturally with patient-record-entity-resolution (dedup), claims-anomaly-detection (statistical outliers), and medical-ontology-code-mapping (code normalization). Designed to scale from a laptop sample to a spark-healthcare-data-pipeline job over billions of records.

References

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.