agentsclimarketplace

Redact pii for tracing

Skill ContextJet-ai/awesome-llm-observability/skills/redact-pii-for-tracing

50+ curated LLM observability tools PLUS 26 Agent Skills (several with runnable, unit-tested scripts) to build, evaluate, debug, secure & monitor reliable LLM apps. Tracing, evals, guardrails, LLMOps.

Install
npx -y skills add ContextJet-ai/awesome-llm-observability --skill redact-pii-for-tracing

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.

What its author says it does

Copied from the file, not written here

Use this when adding LLM observability to an app that handles sensitive data (finance, healthcare, PII) and you must NOT ship raw prompts/PII to a third-party tracing backend. Trigger on "redact traces", "PII in observability", "can we self-host tracing for compliance", "GDPR/HIPAA/SOC2 and LLM logging", or instrumenting a regulated app. Get observability without creating a data-leak.

The file declares its own license as CC0-1.0. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

3.4 KB, as published. Nobody here has run it

PII-safe / compliant LLM tracing

Observability captures prompts and completions - which, for a finance or healthcare app, means you may be shipping account numbers, SSNs, or PHI to a third-party SaaS. That's a compliance incident waiting to happen. This skill adds tracing without the leak.

Decision order (most-compliant first)

  1. Self-host the backend. For regulated data, prefer an OSS platform you host: Langfuse (MIT), Arize Phoenix, Comet Opik, Helicone. Data never leaves your VPC.
  2. If using a SaaS backend, redact before export. Strip/mask PII in the span-processor pipeline so raw values never leave the process.
  3. Or don't capture content at all. Many SDKs let you record metadata only (tokens, latency, model, span structure) and omit prompt/completion text. You lose content-level debugging but keep full cost/perf/shape observability.

Redaction pipeline (SaaS path)

Insert redaction between span creation and export so it always runs:

  1. Detect PII with a real detector - Microsoft Presidio, LLM Guard, or a domain ruleset (account/card/IBAN/SSN patterns for finance). Don't rely on ad-hoc regex alone.
  2. Transform - mask (****1234), hash (for join-ability without exposure), or drop the field. Choose per field: costs need amounts, but not the account holder.
  3. Apply on the OTel span processor / SDK hook (e.g. an on_end span processor, or the platform's masking callback) so no code path bypasses it.
  4. Redact both directions - user input and model output (models echo PII back).

Governance to layer on

  • Retention - set TTLs on trace storage; regulated data shouldn't live forever.
  • Access control - restrict who can read traces; content view separate from metrics view.
  • Audit - log who accessed traces (observability of your observability).
  • Data-processing agreements - if any content leaves your boundary, ensure the vendor DPA covers it.

Verify

  • Feed a request containing synthetic PII (fake SSN/card/account). Confirm the exported trace shows masked values, not raw ones - check the backend, not just local logs.
  • Confirm redaction runs on the export path (disable the backend and it should still redact, proving it's not backend-side).
  • Confirm output/echoed PII is also masked.

Anti-patterns

  • Turning on "log full prompts" in a finance app and pointing it at a SaaS backend. (Direct leak.)
  • Redacting only inputs, not model outputs.
  • Regex-only PII detection for high-stakes data (misses formats, context-dependent PII).
  • Redacting client-side after the SDK already sent the span - redact before export.

Authored by ContextJet.ai - we build secure, observable AI for finance and regulated industries.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.