Healthcare data interop

Healthcare data interop (validated, de-identified)

Build healthcare-data pipelines and prove the data is well-formed and stripped of PHI — correctness is gated by scripts that run (FHIR/HL7 validation, PHI scan, DICOM de-id + verify), not by assertion.

Core principle

Validity and de-identification are measured, not claimed. The loop is: scan/validate → fix the root cause → re-validate, until the gate is green; then a human reviews what the automation cannot decide.

Be honest about scope (this is the rule that keeps the skill correct):

• De-identification reduces, but does not eliminate, re-identification risk. HIPAA gives two paths — Safe Harbor (remove the 18 identifiers) and Expert Determination (a qualified statistician certifies low risk). GDPR distinguishes true anonymization (irreversible, out of scope of GDPR) from pseudonymization (reversible, still personal data). State which you are doing. → references/02-deidentification.md
• The scripts handle header/structured PHI only. They do not touch burned-in pixel PHI in images, and the regex scan does not reliably catch names or free-text PHI.
• Structural FHIR validation ≠ profile conformance. A green structural run does not prove US Core / IG conformance; that needs a real validator.

This skill is engineering assistance, not a clinical or regulatory sign-off.

When to use vs. not

• Use for: validating FHIR R4 / HL7 v2 messages; de-identifying DICOM header PHI; scanning text/JSON for Safe-Harbor identifiers; designing an ingest→validate→de-id→transform→store pipeline; mapping legacy data to FHIR.
• Not for: clinical decision-making, billing/coding adjudication, or any output treated as a compliance certification. Not a substitute for a HAPI/Inferno conformance run or a privacy/legal review.

Inputs to gather first

1. Standards & versions — DICOM, HL7 v2.x (which version), and/or FHIR R4/R5; any profile (US Core, a national profile, IHE actor). → references/01-standards.md
2. De-id basis — HIPAA Safe Harbor vs Expert Determination, or GDPR anonymization vs pseudonymization; whether dates/UIDs must be retained. → references/02-deidentification.md
3. Data shape & PHI surface — files vs streams; do images carry burned-in pixel PHI? Are there free-text fields the regex scan can't clear?
4. Pipeline stage — what to build: ingest, validate, de-id, transform/map, or store. → references/04-pipeline-patterns.md

Workflow

Load each reference when you reach its step.

1. Pin the standards & profiles. Confirm DICOM/HL7 v2/FHIR versions and any profile, and that structural checks ≠ full conformance. → references/01-standards.md

2. Set up the scripts. They are stdlib-only except DICOM de-id (optional pydicom). → references/05-running-the-scripts.md

   python3 --version           # 3.12+
   pip install -r scripts/requirements.txt   # only needed for DICOM de-id
   

3. Validate structure of FHIR and/or HL7 v2 inputs; fix errors; re-run until exit 0. → references/03-validation-and-conformance.md

   python3 scripts/validate_fhir.py path/to/fhir.json    --out-dir fhir-report
   python3 scripts/validate_hl7v2.py path/to/msg.hl7     --out-dir hl7-report
   

4. Scan for PHI before anything leaves a trusted boundary; a non-zero exit means stop and review. → references/02-deidentification.md

   python3 scripts/phi_scan.py path/to/data --out-dir phi-report
   

5. De-identify DICOM header PHI and verify by re-reading the output. → references/02-deidentification.md

   cp scripts/deid.config.example.json scripts/deid.config.json   # optional overrides
   python3 scripts/deidentify_dicom.py path/to/dicom --out-dir deid-out \
       --config scripts/deid.config.json --verify
   

6. Build the pipeline stage by stage (ingest → validate → de-id → transform → store), gating each handoff on the checks above. → references/04-pipeline-patterns.md

7. Complete the human review the automation can't: burned-in pixel PHI, free-text/name PHI, profile conformance via a real validator, and the re-identification-risk judgement. → references/02-deidentification.md

What's in this skill

• scripts/validate_fhir.py — structural FHIR R4 JSON check (resourceType, required elements, reference shape, coding presence); file or dir; report + exit code. STDLIB only.
• scripts/validate_hl7v2.py — pipe-delimited HL7 v2 parser; validates MSH + basic required fields; report + exit code. STDLIB only.
• scripts/phi_scan.py — regex scan for the regex-detectable Safe-Harbor identifiers (SSN, phone, email, dates, URLs, IPs, MRN/account, IBAN, ZIP); report + exit code. STDLIB only.
• scripts/deidentify_dicom.py — removes/blanks DICOM PS3.15 Basic-Profile PHI tags via pydicom; --verify re-reads and asserts none remain; clean message if pydicom missing.
• scripts/deid.config.example.json — tag→action overrides for the DICOM de-id.
• scripts/examples/ — valid + invalid samples used by the self-test.
• references/01–05 — standards, de-identification, validation/conformance, pipeline patterns, running the scripts.

Definition of done

• validate_fhir.py exits 0 on all FHIR inputs (errors fixed at the source, not suppressed).
• validate_hl7v2.py exits 0; MSH and required segment fields present.
• phi_scan.py exits 0 on data crossing a trust boundary — or every finding is reviewed and justified.
• deidentify_dicom.py --verify reports no residual configured PHI tags.
• Burned-in pixel PHI and free-text/name PHI reviewed by a human (the scripts cannot clear these).
• De-id basis stated (Safe Harbor / Expert Determination / GDPR anonymization vs pseudonymization) and matches what was actually done.
• Where conformance is claimed, a real validator (HAPI $validate, Inferno, HL7 conformance profile) was run — not just the structural check.

Guardrails — avoid these mistakes

• Don't claim "de-identified" from a clean PHI scan. State "no structured Safe-Harbor identifiers detected; free-text and pixel PHI reviewed separately." Overclaiming is the cardinal error here.
• Don't claim "FHIR conformant" from the structural check. It proves shape, not profile/value-set/invariant conformance — run a real validator for that.
• Never put real PHI in logs, reports, test fixtures, or prompts. Use synthetic data; the examples/ are already synthetic.
• Burned-in pixel PHI is invisible to header de-id — flag images for OCR / pixel review separately.
• Don't suppress a validation error to go green — fix the data or the mapping. Configure deid.config.json for real tag needs, never to skip PHI.
• Pseudonymization is not anonymization — a re-identification key still exists; treat the output as personal data under GDPR.
• Mind the minimum-necessary principle — only move the PHI a stage actually needs.