Gds investigator

GDS Investigator

A GDS investigator traces anomalies to their root cause — determining not just THAT an entity is anomalous, but WHICH dimension drives it, WHEN the deviation started, WHAT caused it, and HOW it differs from normal peers.

The goal is findings with evidence chains, not observations. Every finding should connect detection to root cause to recommended action. A report with 20 detections and 0 root causes is a failure.

Detection recipes (event rates, Simpson's paradox, temporal bursts, drift) are in the companion skill gds-detective. Read it alongside this one.

For concrete tool output examples, see references/examples.md.

---

Core principle

Find the best pattern. Go deep. Validate against ground truth if available. 20 calls on the best pattern beats 5 calls on 4 patterns.

Your report should include a recall/precision table when ground truth exists. Without it, the investigation is incomplete.

Prerequisites check

Before network-dependent investigation steps (counterparties, paths, chains), verify the pattern has edge data:

edge_stats(pattern_id) -> row_count, unique_from, unique_to, avg_degree

If edge_stats returns null, the pattern has no edge table — skip all edge-dependent tools (find_counterparties, find_geometric_path, discover_chains, extract_chains, find_chains_for_entity) and note "no edge data available" in the report.

Sphere health gates

Pattern-level auditors in dim_quality_warnings — beyond dead_dim and sparse_dim (per-dim signals), sphere_overview also surfaces dominant_dim_mass (pattern-level: one dim accounts for ≥70% of tail variance — sphere is effectively a one-dim detector on this pattern), negative_space (gaussian-declared dims that empirically sit at zero — the gaussian z-score is wrong for them), non_normal_dim (per-dim: a kind='gaussian' dim has Shapiro-Wilk / KS p < 0.01 at build time — the z-score percentile semantics are unstable; recommend log1p / sqrt / rank transform or kind re-declaration), and heteroscedasticity (pattern-level: Brown-Forsythe Levene p < 0.01 on delta_norm partitioned by group_by_property — the global θ assumption is statistically violated, so a single global threshold produces unequal false-positive rates per group). When dominant_dim_mass fires, cross-check by inspecting reliability_flags.single_dim_driven incidence on find_anomalies top-N for that pattern — pattern-level dominance ⟹ high per-polygon incidence agreement. When negative_space fires, treat the affected dim's contribution to delta_norm as suspect and prefer kind='bernoulli' / kind='poisson' re-declaration over per-entity drill-down. When non_normal_dim fires (and negative_space does not), the dim's z-score is a poor anomaly scorer because the empirical distribution is heavy-tailed; raise a calibration ticket — variance-stabilizing transform in the column source script + rebuild fixes it. When heteroscedasticity fires, the pattern already carries per-group calibration via group_by_property (the warning is confirmation it was statistically warranted), but cross-group anomaly score comparison is suspect; use aggregate(group_by=<prop_col>) to inspect per-group anomaly rates before drawing conclusions across the whole pattern.

Label-aware dim audit (audit_pattern_dims) — when the sphere carries a label_audit: block in sphere.yaml (label_aware_available: true on get_sphere_info), call audit_pattern_dims(pattern_id, top_k=10) to surface label-discriminating dims. The tool returns per-dim raw mu/sigma plus class-conditioned mu_pos / sigma_pos / mu_neg / sigma_neg, cohens_d_pos_neg, and the per-dim component of the Fisher LDA direction vector. Each dim carries recommended_action ∈ {keep, split, drop_low_separation, investigate_drift, kind_mismatch_review}. Use to: (a) confirm which dims are actually driving the positive class — high |direction_component| ⟹ informative axis; (b) flag dims with high cohens_d_pos_neg but low |direction_component| as candidates for split (compound signal — the raw separation is real but the dim conflates two regimes); (c) flag drop_low_separation candidates pre-rebuild (|cohens_d| < 0.1 ⟹ dim contributes noise to delta vector). Spheres without label_audit: return a fallback shape with raw stats and recommended_action: keep — still useful for surfacing per-dim mu/sigma ranking without label semantics.

---

Investigating with ground truth

If the entity line has a labeled outcome column (e.g., status, outcome, label), this is the highest priority approach:

get_line_profile(line_id, label_property)  -> distribution of labels
search_entities(line_id, label_property, "bad_value") -> get "bad" entity keys

Batch recall check (preferred — 1 call instead of 2N):
  check_anomaly_batch(bad_keys, pattern_id)
  -> returns is_anomaly + delta_rank_pct per key + recall_if_all_bad

Run this for each pattern that covers these entities — including patterns
on OTHER entity lines that share the same primary keys.
Check get_sphere_info for all lines with matching key columns.

Fallback (if check_anomaly_batch not available):
  For each "bad" key: goto(key, line_id) -> get_polygon(pattern_id) -> check is_anomaly

Focus on the pattern with highest recall.

contrast_populations(best_pattern, {"keys": bad_keys})
-> look at MAX single-dimension |d|, not average

get_line_profile for TOP 2-3 numeric properties only (from contrast step),
with group_by=label. Do not profile every column — pick the ones with highest |d|.

Report per-pattern recall table:
| Pattern | Anomalies | "Bad" caught | Recall | Precision |

Checking for missed entities:

If any "bad" keys are missed by the best single pattern:

composite_risk_batch(missed_keys, line_id)

Threshold: combined_p < 0.10 (relaxed vs typical 0.05).

Lower-level p-value composition (when detectors outside cross_pattern_profile matter — external rules engines, typology scores, chain-coherent run p-values):

combine_anomaly_pvalues([(pattern_id, p_value), ...], method="hmp")
-> Wilson HMP combination (default; robust under positive dependence)
   or method="fisher" for classical independence.
-> Returns combined_p + per-pattern reliability_flags pass-through.
classify_detector_consensus([(pattern_id, p_value), ...])
-> Labels the set: unanimous_anomaly / majority_anomaly / split / unanimous_normal.
   "split" verdict is itself a signal — entity at the boundary of two typologies.

Investigating without ground truth

anomaly_summary on each pattern -> which has strongest signal?
find_anomalies(best_pattern, top_n=10) -> top suspects
  Optional: find_anomalies(best_pattern, top_n=10, min_confidence=0.7)
    -> filter to entities where bootstrap confidence >= 0.7
    -> reduces the FP investigation burden when population <= 50K
Check BOTH ends: if top anomalies all have negative deltas (below mean),
  use attract_boundary(alias, pattern, direction="in") to find entities
  at the positive extreme. Anomaly = extreme in EITHER direction.
For top 3 suspects: entity 360 (below)
Form hypotheses about what drives anomalies, test them

---

FDR control and diverse selection

When tracing root causes, use fdr_alpha=0.05 on find_anomalies to ensure the initial suspect list has controlled false discovery rate — chasing false positives through the full root-cause chain wastes the entire investigation budget. Use select="diverse" when requesting K>10 results to surface anomalies driven by different dimensions rather than clustering on one dominant failure mode; this ensures the investigation covers distinct root-cause categories. Both parameters also apply to attract_boundary, find_hubs, and find_drifting_entities.

Adaptive FDR (Storey). When the investigation needs to expand the candidate list without relaxing α, try fdr_method="storey" together with p_value_method="chi2" — the Storey LSL estimator detects when the population has a real null mass and shrinks q-values accordingly, typically recovering 10–15% more suspects at the same false-discovery guarantee. The two parameters must be set together: Storey with the default rank p-values has no effect (rank p-values are uniform by construction). If the pattern is heavily compressed (every entity looks sub-null) or saturated (every entity already exceeds the null), Storey collapses to BH — keep the default in those regimes.

Prioritise deteriorating drift. When drift_direction is "deteriorating", the entity is structurally moving away from the null centre — investigation priority is higher than for equally-displaced entities with "normalizing" direction, which are already self-correcting. Use this to rank multiple drifting entities by risk rather than raw displacement.

Decompose drift into intrinsic vs extrinsic. decompose_drift(entity, pattern) (default args) splits the entity's drift between its first and last temporal slice into intrinsic (entity-driven shape change, σ from oldest retained calibration) and extrinsic (residual, population recalibration). The aggregate intrinsic_fraction answers: "Of the squared change, what proportion was the entity itself?" Use when find_drifting_entities flags an entity and you need to discriminate "this entity actually moved" from "the population calibrated around it". An entity at intrinsic_fraction ≈ 0.05 didn't really change — recalibration shifted the coordinate system around it. An entity at intrinsic_fraction ≈ 0.95 is the real signal. find_drifting_entities already attaches the same three scalars per entry for batch triage.

---

Root cause chain

One-call root-cause tracing. trace_root_cause(primary_key, pattern_id) returns a bounded DAG of evidence in one shot — root witness dimensions from explain_anomaly, an edge-counterparty branch (sorted by anomaly, not by transaction volume), a neighbour-contamination branch (with explicit anomalous_cp_keys and revisits_root clique flag), and a hub branch — replacing the manual explain_anomaly → find_counterparties → contagion_score → π7 hub chain below. Use it as the default first step when the user asks "why is this entity anomalous"; fall back to the manual chain only when truncated=true signals the bounded tree missed context you actually need.

Parameter cheatsheet. Default call trace_root_cause(pk, pid) fits 90 % of investigations. Tune when:

• edge_counterparty_top_n=2..5 — expand multiple anomalous counterparties as separate subtrees. Needed when contagion shows 5+ anomalous cps and you want all of them recursively expanded, not just the single most anomalous.
• max_depth=3..4 — reach grand-counterparties for deep mule chains; usually depth=2 is enough because contagion branch already summarizes the neighbourhood.
• branches_enabled=["neighbor_contamination"] — skip edge_counterparty + hub computation for a fast "is this entity network-contaminated?" query; avoids the full adjacency+π7 scans and is orders of magnitude faster than the default call. Also ["hub"] for hub-only or ["edge_counterparty"] for pure chain traversal.
• contagion_min_counterparties=5 — raise above default 3 on noisy sub-populations where even 3-cp contagion can be statistical noise; small-N contagion=1.0 is a well-known fake alert.
• hub_pop_limit=<larger> — raise above the default on medium-sized anchor patterns where hub membership IS informative and the π7 scan cost is acceptable; the default biases toward skipping hub for large populations.

Evidence interpretation — the clique signals.

• revisits_root on a contagion branch = the root entity appears in that node's anomalous counterparty list. Confirmed geometric clique — root and this cp are structurally tied and both anomalous. Raise severity one notch beyond raw contagion score.
• previously_seen_as_cp_of: [X, Y] in evidence = within the current session, other trace_root_cause calls (on X and Y) reported this entity as their anomalous counterparty. Strong network-centre signal even when the current trace looks isolated. Agent workflow: investigate entity, then investigate its cps, then re-check the original entity — inter-call ledger surfaces clique without diffing multiple trees.
• anomalous_cp_keys = the up-to-10 anomalous counterparty primary keys of this node. No extra find_counterparties call needed to drill down.
• truncated: true = more evidence candidates were dropped than fit the max_branches / max_total_nodes caps. Rerun with higher caps or narrower branches_enabled to recover the dropped signal.

Session cache. trace_root_cause caches contagion_score and find_counterparties per (pattern_version, entity_key) at the navigator instance level. Repeat traces on the same or overlapping entities in one session hit the cache and return near-instantly; the first cold trace on a large anchor pattern is the one that pays the adjacency-scan cost. Capped with LRU eviction to bound memory. Pattern rebuild invalidates automatically via version key.

Sample tree — structure template + concrete example:

Template — how the fields nest:

root: <entity_key> (severity=extreme)
  top_dimensions: [{dim, kind, bregman, pct_of_total}, ...]
  children:
    - edge_counterparty: <cp_key_A> (severity=moderate)
        via_dim: <witness_dim_from_root>
        witness_counterparty_delta_rank_pct: <rank>
        children:
          - neighbor_contamination: <cp_key_A> (severity=high)
              anomalous_cp_keys: [<root_key>, <peer_1>, ...]   ← root in list
              revisits_root: [<root_key>]                      ← CLIQUE signal
    - neighbor_contamination: <entity_key> (severity=low)
        anomalous_cp_keys: [<cp_key_A>, <cp_key_B>]
        previously_seen_as_cp_of: [<other_entity>]             ← INTER-CALL signal

Concrete example — what an actual confirmed-clique trace looks like:

{
  "root": {
    "entity_key": "ACC-ROOT",
    "role": "root",
    "severity": "extreme",
    "evidence": {
      "top_dimensions": [
        {"dim": "amount_out_std", "kind": "gaussian", "bregman": 67.4, "pct_of_total": 0.28},
        {"dim": "sum_in", "kind": "gaussian", "bregman": 42.6, "pct_of_total": 0.18}
      ],
      "delta_norm": 25.16,
      "conformal_p": 0.000004
    },
    "children": [
      {
        "entity_key": "ACC-MULE",
        "role": "edge_counterparty",
        "severity": "moderate",
        "evidence": {
          "via_dim": "amount_out_std",
          "witness_counterparty_delta_rank_pct": 98.09
        },
        "children": [
          {
            "entity_key": "ACC-MULE",
            "role": "neighbor_contamination",
            "severity": "high",
            "evidence": {
              "contagion_score": 0.625,
              "total_counterparties": 8,
              "anomalous_counterparties": 5,
              "anomalous_cp_keys": ["ACC-ROOT", "ACC-PEER1", "ACC-PEER2", "ACC-PEER3", "ACC-PEER4"],
              "revisits_root": ["ACC-ROOT"]
            }
          }
        ]
      },
      {
        "entity_key": "ACC-ROOT",
        "role": "neighbor_contamination",
        "severity": "low",
        "evidence": {
          "contagion_score": 0.2,
          "total_counterparties": 10,
          "anomalous_counterparties": 2,
          "anomalous_cp_keys": ["ACC-MULE", "ACC-OTHER"],
          "previously_seen_as_cp_of": ["ACC-EARLIER-SUSPECT"]
        }
      }
    ]
  },
  "summary": "Entity ACC-ROOT is extreme in account_pattern; primary witness: amount_out_std; branches found: edge_counterparty, neighbor_contamination; 3 nodes.",
  "hop_count": 2,
  "branches_explored": 3,
  "truncated": false
}

Read order for pattern-matching: root.severity first → revisits_root (direct clique) → previously_seen_as_cp_of (inter-call clique) → anomalous_cp_keys (queue for deep-dive) → truncated (more evidence dropped — rerun with larger caps if needed).

Triage playbook:

• revisits_root present on any contagion node → confirmed geometric clique; escalate immediately.
• previously_seen_as_cp_of non-empty → shared-counterparty signal across multiple suspects; high-value graph-centre node.
• contagion_score ≥ 0.5 + revisits_root empty + root severity extreme → structural anomaly NOT contagion-driven; investigate root's own properties, not network.
• All child severities "low" + root "extreme" → isolated primary-source anomaly, not network mule.

Edge-level anomaly — edge_potential

edge_potential and attract_edge_potential score the relationship itself, not the endpoints. Score formula: distance between endpoint delta vectors × rarity prior (1/pair_tx_count, capped). High score means two geometrically divergent entities are connected by a rare pair — the classic layering signature.

When to use.

• After trace_root_cause surfaces an edge_counterparty branch — check the edge_potential field already attached in evidence. Score above sphere-specific threshold (typically > p95 of population) reinforces the counterparty signal.
• Standalone ranking: find_high_potential_edges(pattern_id, top_n=20, min_pair_count=1) surfaces the most suspicious relationships in the whole pattern without committing to a specific suspect.
• Entity-scoped: find_high_potential_edges(pattern_id, top_n=10, from_key=suspect) ranks the suspect's own edges — useful as a drill-down from trace_root_cause.

Example trace_root_cause edge_counterparty evidence with edge_potential:

{
  "role": "edge_counterparty",
  "entity_key": "ACC-MULE",
  "severity": "moderate",
  "evidence": {
    "via_dim": "amount_out_std",
    "witness_counterparty_delta_rank_pct": 98.09,
    "edge_potential": {
      "score": 12.5,
      "delta_distance": 5.1,
      "pair_tx_count": 1,
      "effective_weight": 1.0
    }
  }
}

Reading order: if both witness_counterparty_delta_rank_pct > 95 AND edge_potential.score is in the top-percentile of the pattern, you have a confirmed single-tx structurally-anomalous edge — treat as highest priority.

Structural motifs — score_motif and find_high_potential_motifs

score_motif extends the edge_potential paradigm from one edge to k edges of a named structural pattern. Scoring is product-of-edge_potential across the motif's edges — a motif of rare edges is rare. Eight motif types in the closed vocabulary:

• cycle_2 (default window 24h): bidirectional A↔B round-trip. Covers Flash-Burst Round-Trip and Bidirectional Burst typologies.
• cycle_3 (default window 72h): directed triad A→B→C→A with strict temporal ordering. Covers Round-Tripping 3-Party, Long-Cycle, and Multi-Round-Tripping typologies.
• fan_out (default window 168h): hub → k distinct targets (min k=3). Covers Offshore Hub and Concentrator (source side) typologies.
• fan_in (default window 168h): k distinct sources → sink (min k=3). Mirror of fan_out. Covers Parallel Layering (destination side) and Concentrator / Sink typologies.
• chain_k (default window 168h, open directed chain of parametric length 3 ≤ k ≤ 8): A→B→…→Z with no cycle closure, no node revisit, strict monotone timestamps, total span ≤ window. Covers Multi-Stage Layering and Multi-Jurisdiction Latency Chain typologies. Tune k to the layering depth under investigation.
• structuring (default window 1h, amount-gated): open A→B→C→D with hop1 amount ≥ amt1_min, hops 2 and 3 ≤ amt2_max. Classic deposit-split-and-wire pattern for reporting-threshold evasion.
• split_recombine (default window 24h, min_k default 3, direction="forward"|"backward"): diamond scatter-gather S → {M₁,…,Mₖ} → D with stacked-bipartite temporal order (all split-hops precede all recombine-hops within the window). Forward mode anchors the seed as the source; backward mode anchors the seed as the sink. Covers scatter-gather smurfing, parallel layering, and concentrator/sink (backward mode) typologies — amount-free counterpart to structuring.
• bipartite_burst (default window 24h, min_k default 3, min_m default 3): complete K_{k,m} bipartite subgraph in a tight time window — k distinct sources each transact with every one of m distinct sinks. Greedy single-core enumeration: tries seed-as-source first, falls back to seed-as-sink. Covers coordinated mule-ring and parallel-collusion typologies; complements fan_out + fan_in by requiring completeness on both sides rather than density at a single anchor.

When to use.

• After trace_root_cause — the edge_counterparty branch now carries motif_potential automatically when the suspect seeds a motif that passes through the counterparty. Read the block alongside edge_potential and witness_counterparty_delta_rank_pct; a confirmed signal on all three means structural + per-edge + witness-dimension agreement.
• Global screening: find_high_potential_motifs(pattern_id, motif_type="cycle_3", top_n=20) surfaces the most suspicious triads in the whole pattern. First call per (pattern, motif_type, window, …, k) is cold (30–90s on >500k-entity patterns) — subsequent calls hit the LRU cache.
• Entity drill-down: score_motif(suspect, motif_type="cycle_2", pattern_id) checks whether the suspect is the seed of a high-score round-trip without committing to a specific counterparty.

motif_potential block in trace_root_cause.edge_counterparty.evidence:

{
  "motif_potential": {
    "motif_type": "cycle_2",
    "score": 64.0,
    "time_window_hours": 24,
    "counterparty": "ACC-MULE"
  }
}

When motif_type is cycle_3, the block includes ring: [seed, B, C]. When fan_out or fan_in, it includes k (distinct neighbours in the window). When chain_k, it includes path (list of k keys) and k. When split_recombine, it includes source, sink, intermediaries (the M nodes), k, and direction. When bipartite_burst, it includes sources, sinks, k, m, and seed_role ("source" or "sink").

explain_anomaly tells you WHICH dimension is anomalous, with per-dim Bregman contributions when dimension kind tags are available. That is an observation, not a finding.

Before opening a case on any anomaly, check reliability_flags on the returned polygon (surfaced by find_anomalies, explain_anomaly, composite_risk, combine_anomaly_pvalues, and investigate_entity). Two flags fire independently:

• single_dim_driven=true — one dim contributes >70 % of total anomaly attribution. Likely a data-quality artefact (saturated counter, outlier on one property) rather than a multi-dim fraud signal. Sanity- check the dominant_dim value before escalating.
• low_confidence_bucket=true — bootstrap-derived anomaly_confidence is below 0.5. The anomaly flag is fragile to population resampling — the entity is on the borderline. Treat as a soft hit; corroborate with one more detector (a chain pattern, a witness-cohort overlap, a counterparty signal) before opening a case.

reliability_flags.dominant_dim always agrees with the top entry of explain_anomaly.top_dimensions for the same polygon — both surfaces route through the same per-dim contribution primitive. If they disagree on a real call, that's a bug to report.

Pre-case certainty gate — assess_anomaly_certainty. Where reliability_flags reports the two soft-hit flags individually, the assess_anomaly_certainty(primary_key, pattern_id) composer rolls them up with FDR-alpha stability, calibration staleness, and cross-pattern consistency into a single certainty_verdict — the one-call gate to run before opening an investigation on a single-detector hit.

assess_anomaly_certainty(primary_key, pattern_id)
-> certainty_verdict: "high" | "moderate" | "low" | "contested"
-> certainty_score, conformal_p, signed_confidence,
   stability_across_alphas, reliability_flags
   (single_dim_driven, near_data_boundary, calibration_stale),
   cross_pattern_consistency, recommended_next_steps

certainty_verdict is certainty about the CLASSIFICATION, not about anomaly status — a confidently-normal entity scores "high" too. Gate routing:

• high — stable across the swept FDR alphas, not single-dim-driven, not near the data boundary, calibration fresh. Open the case and run the full root-cause chain.
• moderate — partial stability; investigate but keep the FP possibility open. Corroborate with one more detector before classifying CONFIRMED.
• low — fragile (one-alpha hit or boundary-adjacent). Treat as a soft hit; do not open a formal case on this alone.
• contested — the calibrated conformal_p disagrees with the stored anomaly flag, or the entity is both single-dim-driven and near-boundary. Highest FP risk: resolve the conflict (re-read explain_anomaly, inspect near_data_boundary, check cross_pattern_consistency) before deciding.

Use this as the gate that feeds the CONFIRMED / SUSPECTED / FALSE POSITIVE classification later in the report: high + multi-source ⟹ CONFIRMED candidate; contested ⟹ FALSE POSITIVE candidate until the conflict resolves. Pair with find_diverse_explanations when the verdict is contested because single_dim_driven fired.

Multi-hypothesis investigation: when explain_anomaly shows reliability_flags.single_dim_driven=True, call find_diverse_explanations(primary_key, pattern_id, n_hypotheses=3) to surface alternative hypotheses beyond the dominant dim. Top-anomalous entities often degrade to 1-2 hypotheses with degraded_reason="insufficient_diverse_mass" — that's correct semantic, signals the anomaly is genuinely single-dim. When the result returns 2+ hypotheses with diversity_score > 0.8, each is an independent investigation path worth probing.

Compliance + anomaly cross-check — find_conformance_violations(pattern_id) returns entities that broke declarative rules in sphere.yaml. Independent from delta_norm anomaly flags; compose via investigate_entity on top violators to test "does the rule break also reflect a geometric anomaly?" If yes — confirmed compliance issue with structural backing. If no — rule break alone, may indicate policy gap or genuine compliance violation without geometric signature.

The root cause chain goes deeper:

explain_anomaly -> dominant dimension identified
dive_solid -> WHEN did this dimension change?
  If dive_solid returns no data, skip — report temporal analysis unavailable.
  If edge table available: degree_velocity(key, pattern_id) -> did connection rate also change?
get_event_polygons or find_counterparties -> WHAT happened in that period?
  entity_flow(key, pattern_id) -> net flow summary per counterparty (source/sink/mule role)
  contagion_score(key, pattern_id) -> what fraction of neighbors are anomalous?
compare_entities with a normal peer -> HOW does this entity differ?
find_geometric_path(from_key, to_key, pattern_id) -> HOW are two anomalous entities connected?
  Use when two entities share anomaly dimensions — traces the geometric path between them.
  scoring="geometric" (default) ranks by delta coherence; "anomaly" ranks by anomaly density along path.
find_novel_entities(pattern_id) -> WHO deviates most from neighborhood expectation?
  High novelty_score = entity doesn't behave like its neighbors. Requires edge table.
find_graph_geometry_tension(primary_key, pattern_id, line_id) -> behavioural-vs-edge 2x2 cross-tab.
  hidden_cluster = similar-but-disconnected (lookalike cohort never seen together);
  suspicious_links = connected-but-distant (out-of-peer-group). On AML-class data the signal
  sits in suspicious-links count; hidden_cluster saturates at k_geometric.
find_topological_anomalies(pattern_id) -> WHO sits in a local H_1 cycle in delta space.
  Ranks by raw h1_max_persistence (not the auxiliary normalised topo_score). Empirical lift
  is mid-rank — composition input for HMP / passive_scan, NOT a top-N drill-down replacement.
simulate_edge_removal(primary_key, pattern_id, line_id, top_n=5) -> WHICH edges made this entity anomalous.
  Per-edge counterfactual: for each candidate edge in the entity's adjacency, simulate removal and
  rank by drop in delta_norm. Returns (edge_id, drop_pct, dominant_dim_label) per edge. v0 covers
  relations + prop_columns dim classes; edge_dim_aggregations deferred. Investigator-drilldown only,
  not a population scoring axis.
simulate_counterparty_removal(primary_key, pattern_id, line_id, top_n=5) -> WHICH counterparty
  made this entity anomalous. Counterparty-level counterfactual: aggregates all edges to one
  counterparty per call; ranks counterparties by aggregate delta_norm drop. SAR-friendlier than
  per-edge ("removing transactions with ACC-X drops delta_norm by 38 %") when the edge ranking
  is too granular.
select_minimal_joint_edge_removal(primary_key, pattern_id, line_id, target="flip", k=8)
  -> SMALLEST edge set that flips the anomaly verdict. Greedy edge-set search; returns
  edge_ids[] + delta_norm_after + flipped. The minimal joint set is the literal "evidence
  of anomaly" — what you cite in the SAR narrative as the cause.
simulate_dimension_change(primary_key, pattern_id, line_id, set_dimension, top_n=5) -> would this
  entity still be anomalous if dim X were at value V. What-if dimension override: override one or
  more raw shape-vector dims, recompute delta_norm under the pattern's calibration, and report
  delta_norm_before/after, the anomaly-flag flip, and the new top witness dims. Companion to
  simulate_edge_removal for non-edge dimensions. set_dimension is {dim_label: new_value} in raw
  shape-vector units — call explain_anomaly first to pick dim_labels and read current values.
find_witness_cohort(key, pattern_id) -> peers with similar anomaly profile

Graph confirmation chain (when edge table exists): After explain_anomaly, check graph support to distinguish isolated anomalies from patterns:

1. contagion_score(key, pattern_id) — what fraction of neighbors are anomalous?
2. find_witness_cohort(key, pattern_id) — which peers share the anomaly profile?
3. find_novel_entities(pattern_id, top_n=10) — who deviates most from neighborhood?

High contagion + large witness cohort = confirmed pattern (not isolated outlier).

As-of graph reconstruction: for incident forensics, all six edge-table graph primitives (contagion_score, contagion_score_batch, entity_flow, degree_velocity, propagate_influence, find_counterparties) accept an optional timestamp_cutoff parameter (Unix seconds). When set, only edges with timestamp <= cutoff are considered. Use this to answer "what did the neighborhood look like at time T?" — e.g. pass the incident timestamp to contagion_score to see contamination state on that day, not today.

Always report the repair set from explain_anomaly. Format: "Repair: zero {repair_size} dims ({labels}) -> residual {residual_norm} (below theta={theta_norm})."

---

Entity 360 (deep-dive on one entity)

cross_pattern_profile(key, line_id)     -> multi-source risk overview
goto(key, line_id) -> get_polygon(pattern_id) -> anomaly dimensions
explain_anomaly(key, pattern_id)        -> per-dim Bregman contributions + kind tags
  If bregman_contribution present: prefer it over abs_delta for root-cause ranking.
  Each dim shows kind (gaussian/poisson/bernoulli) and pct_of_total.
  Focus on dims with highest pct_of_total, not just highest abs_delta.
  Interpret by kind: poisson dim unusual = count structure anomaly;
    gaussian dim extreme = magnitude anomaly; bernoulli dim = binary flag fired.
anomaly_confidence (from get_polygon or find_anomalies result):
  >= 0.8  -> stable anomaly: high-confidence finding, proceed with full investigation
  0.3-0.8 -> borderline: investigate but maintain FP possibility in assessment
  < 0.3   -> likely false positive: verify with additional sources before escalating
  (confidence is absent for populations > 50K, group_by_property, or use_mahalanobis patterns)
dive_solid(key, pattern_id)             -> temporal history
find_similar_entities(key, pattern_id)  -> geometric neighbors
  dim_mask=[<dims from anomaly_dimensions>] -> focus on driving dims
  metric="cosine" -> shape similarity ignoring magnitude
find_counterparties(key, event_line, from_col, to_col, pattern_id) -> network + amount aggregates
entity_flow(key, pattern_id)           -> net flow per counterparty (source/sink/mule)
anomalous_edges(key, counterparty, pattern_id) -> event-level scoring of specific transactions
discover_chains(key, pattern_id)        -> runtime chain discovery (no pre-built chains needed)
  Works directly on the edge table via temporal BFS. Use when chain lines
  are unavailable or you want chains from a specific entity without full extract.
  Tune min_hops (default 2) and direction ("forward"/"backward"/"both").

# When the sphere DOES have a chain anchor pattern (built from chain_lines:),
# the chain-coherent investigative loop offers a stronger composition:
find_chains_with_coherent_anomaly(chain_pattern_id, anchor_pattern_id, min_hops=3)
  Population sweep — flag chains where ≥min_hops consecutive entities are
  individually anomalous AND share the same dominant delta dim. Distinct
  axis from find_anomalies on the chain pattern (which scores chain shape).
anomaly_propagation_in_chain(chain_id, chain_pattern_id, anchor_pattern_id)
  Drill into a single flagged chain — returns hop-by-hop anomaly trace
  (is_anomaly + delta_norm + top_dim + delta_rank_pct per hop).
classify_chain_typology(chain_id, ...)
  Five-axis label per chain: shape / peak_position / position_in_chain /
  extension_signals / dominant_top_dim. Triage tag without re-reading hops.
extend_chain(chain_id, ..., direction="forward"|"backward")
  Suggest extension entities at the boundary of the chain's anomalous run
  using the chain reverse index. Anomalous candidates are "where to look
  next" investigation targets.
chain_witness_intersection(chain_id, chain_pattern, member_pattern,
                           min_jaccard=0.5, top_k_witness=5)
  Intersect the top witness dims of the chain's members.
  coordinated=True (mean pairwise Jaccard >= min_jaccard) means every
  member is anomalous for the same structural reason — a single
  geometric diagnosis for the chain.
chain_drift_trajectory(chain_id, chain_pattern, member_pattern, n_windows=4)
  Per-member regime (normalizing / deteriorating / neutral) over
  time-bucketed delta_norm, plus a chain-level rollup
  (mixed when members disagree) + numeric drift score. Spots chains
  jointly drifting toward anomaly before any single hop crosses θ.
find_chains_for_entity(entity_key, chain_pattern_id)
  Reverse lookup — list every chain a given entity participates in
  (deduplicated; cyclic / self-revisiting chains surface once). The
  deep-dive accessor for chain extension candidates returned by
  extend_chain. See gds-fraud-investigator R9 for the full
  flag→trace→label→cross-check→extend→deep-dive workflow.

source_count >= 2 = investigate thoroughly. source_count == 1 = likely FP.

---

Hypothesis testing

For every major finding, form an explicit hypothesis and test it:

### Hypothesis: [statement]
**Test:** [tool call + what you're checking]
**Result:** [what the tool returned]
**Verdict:** Confirmed / Rejected / Partially confirmed

Example: "Entity X drives network-wide anomaly spread." Test: propagate_influence([X], pattern_id, max_depth=3) — if 50+ affected entities with decaying influence scores, confirmed.

Minimum 3 cycles per investigation. Rejected hypotheses are valuable.

---

Population contamination

If top-K anomalies are dominated by entities that LACK the ground truth property:

1. The pattern population is too broad
2. Check: what % of anomalies have the GT property? If <30%, ranking is contaminated
3. Filter recall analysis to only entities that HAVE the property
4. Recommend: "Build a filtered pattern on the relevant subpopulation"

Cross-pattern lead-lag

When investigating drift across multiple anchor patterns over the SAME entity line (e.g. account_behavior_pattern × account_stress_pattern), use find_lead_lag(pattern_a, pattern_b) to detect the temporal ordering of population-level shifts.

Default workflow:

find_lead_lag(
    pattern_a="account_behavior_pattern",
    pattern_b="account_stress_pattern",
    cohort="fixed",            # panel-clean centroid signal
    fdr_method="storey",       # default — recovers power on rich-signal regimes
)

Read the response in this order:

1. degenerate_signal — if true, either centroid drift series has zero variance (population is constant per epoch). Treat as no signal and stop.
2. agreement — "strong" means centroid lag and volatility lag both agree on direction and magnitude (most trustworthy); "weak" means both channels show movement but disagree (population is heterogeneous); "divergent" means no coherent lead-lag — do not report a lag claim.
3. is_significant — True when abs(correlation) > max_corr_threshold (peak Bonferroni-adjusted threshold). Reliable headline gate.
4. lag — peak lag in epochs (positive = pattern_a leads pattern_b). Multiply by the temporal window (sphere config) for wall-clock units.
5. top_dim_pairs — even when is_significant=False, the top entries ranked by ascending q_value (then |corr|) surface the strongest leading dim pairs for hypothesis generation. The displayed pairs may not be FDR-significant on small N because Bonferroni-over-lags + BH over D_A * D_B pairs is intentionally conservative.

Drill-down: pass entity_key=... to replace the population centroid with that entity's own delta trajectory. Useful for case storytelling ("for customer X, behavior_change preceded stress_change by 2 epochs"), but per-entity reliability is "low" at most realistic N.

Verbose mode: verbose=True returns the full D_A × D_B matrix in per_dim_pairs (sorted by ascending q-value); use when you need to scan the full breakdown rather than the top 10.

Limit reminder: find_lead_lag requires both patterns to share the underlying entity space — cohort="fixed" raises empty-cohort otherwise. On disjoint entity spaces (e.g. accounts vs chains in AML) cohort="all" typically lands in degenerate_signal=true.

False positive assessment

For every anomaly finding, consider whether it is a true positive or false positive:

• High delta_norm can mean "large legitimate entity" — check business properties
• find_similar_entities(key, pattern, filter_expr="is_anomaly = false", top_n=20) — if 10+ normal entities have same shape, it is likely a false positive
• Use metric="cosine" when comparing anomaly profile shape regardless of severity — "same type of anomaly, different scale"
• Use dim_mask to focus similarity on dimensions from anomaly_dimensions output — finds entities similar only in the dimensions that drive the anomaly, ignoring irrelevant ones
• Use find_anomalies(metric="Linf") to catch single-dimension spikes that L2 norm dilutes — entities with one extreme dimension but normal on others
• Use find_anomalies(metric="bregman") to rank by distribution-aware Bregman divergence — better ranking on mixed-type patterns (counts + amounts + binary flags). Particularly effective when dimension_kinds shows a mix of poisson/gaussian/bernoulli
• Dormant/inactive entities being anomalous is expected, not a finding

Classify every finding as:

• CONFIRMED — 3+ tools confirm, multi-source, root cause identified
• SUSPECTED — 1-2 tools confirm, plausible but not proven
• FALSE POSITIVE — explained by artifact, structure, or domain expectation

At investigation end, compute and state the overall FP rate: FP rate = (# FALSE POSITIVE findings) / (# total findings). Include this in the report summary.

Reading contrast_populations

• Look at the TOP dimension by |d|, not the average
• |d| > 0.8 = large effect, |d| > 1.5 = very large
• Positive d = group_a higher, negative = group_b higher

---

Investigation Memory

Maintain three lists throughout the investigation session:

• checked[] — entities where full investigation is complete (goto + get_polygon + explain_anomaly + counterparties done). Never re-investigate.
• leads[] — entities flagged by tools but not yet investigated. Each lead carries a lead_score (see Decision Scoring). Sources: find_anomalies, passive_scan, find_witness_cohort, propagate_influence, investigation_coverage.unexplored_anomalous.
• dead_ends[] — entities investigated and found uninteresting (delta_rank_pct < 70, no contagion, no temporal signal). Never revisit.

Protocol:

1. Before investigating any entity: check checked[] and dead_ends[]. Skip if present.
2. After each entity investigation: move from leads[] to checked[].
3. After each tool call that returns entity lists: score new entities, add to leads[] (deduplicating against all three lists).
4. Call investigation_coverage(pk, pattern_id, explored_keys=checked) after every deep-dive. If coverage_pct < 0.5 and unexplored_anomalous is non-empty, add those to leads[].
5. When delegating to another skill, pass checked[] as context.

---

Failure Guards

Proactive limits to prevent runaway investigations:

Guard	Threshold	Action
Depth limit	3 hops from seed entity	Stop expanding, summarize findings
Strength gate	delta_rank_pct < 70	Skip entity UNLESS contagion_score > 0.3 or in witness cohort
Contagion gate	contagion_score < 0.2	Do NOT proceed to network expansion — entity is isolated
Consecutive call limit	3 calls to same tool on same entity	Move to next lead
Stale lead expiry	Lead untouched for 10+ tool calls	Demote below fresh leads
Force-switch	5 consecutive calls with no new anomalous entities	STOP current thread, switch to highest-scoring lead

---

Decision Scoring

Rank leads by composite score to decide what to investigate next:

lead_score = 0.35 × anomaly_strength
           + 0.25 × graph_support
           + 0.25 × temporal_signal
           + 0.15 × novelty_bonus

Component	Source	Value
anomaly_strength	delta_rank_pct / 100	0.0–1.0
graph_support	contagion_score	0.0–1.0 (0 if unchecked)
temporal_signal	appears in find_drifting_entities or detect_trajectory_anomaly	0.0 or 1.0
novelty_bonus	appears in find_novel_entities or find_witness_cohort	0.0 or 1.0

Protocol:

1. Always investigate the highest-scoring lead next.
2. After each investigation, update scores of remaining leads (new contagion info may change graph_support).
3. Report queue state: "Next: <entity> (score X.XX) | Queue: N leads remaining"

---

Anti-patterns

Anti-pattern	Fix
is_anomaly tells you THAT, not WHY	Check anomaly_dimensions via explain_anomaly; use bregman_contribution + kind when available
find_similar_entities finds shape twins, not transaction partners	Use find_counterparties for network relationships
Using find_chains_for_entity when chain lines do not exist	Use discover_chains — works on edge table directly, no pre-built chains needed
Calling edge-dependent tools without checking edge availability	Run edge_stats(pattern_id) first — null means no edges
goto() then get_polygon() in parallel	Always sequential — goto sets position, get_polygon reads it
Binary geometry has identical deltas for all anomalies	Use aggregate counts instead
"High burst" without root cause	dive_solid + find_counterparties to trace the why
Closing investigation without checking network coverage	investigation_coverage(key, pattern_id, explored_keys) — confirms explored vs unexplored counterparties

---

Output format

## FINDING: [one sentence]
## EVIDENCE: [tool -> result -> interpretation]
## CONFIDENCE: HIGH / MEDIUM / LOW
## FP RISK: None / Low / Medium / High
## REMEDIATION: [concrete next step or repair recommendation]

HIGH = 3+ tools confirm + multi-source + root cause identified. MEDIUM = 2 tools confirm + single-source but strong signal. LOW = 1 tool confirms + no multi-source + root cause unclear.

Every finding must include a concrete remediation step — not just "investigate further" but a specific action the data owner can take (e.g., "correct record X", "filter dimension Y above threshold Z", "segment population by property W before analysis").

---

When things don't work

• Tool returns empty results — try different parameters, wider sample, or a different pattern. Adjust top_n, limit, or filters.
• Tool errors — check pattern_id and version, verify sphere is open. If the error persists, report it rather than retrying in a loop.
• No anomalies found — not every sphere has every anomaly type. Report "no signal detected" as a valid finding.
• check_anomaly_batch returns all normal for "bad" keys — the pattern may not capture the relevant signal. Try other patterns or composite_risk.
• dive_solid returns no temporal slices — temporal data may not exist for this pattern. Skip temporal root cause and note it in the report.
• contrast_populations shows small effect sizes — the anomaly may be spread across many dimensions rather than concentrated in one. Check individual dimensions rather than relying on the top-1.

---

Skill delegation

Need	Skill
Event rates, Simpson's, temporal bursts, drift recipes	gds-detective
Cross-pattern, neighbor, trajectory, segment scans	gds-scanner
Drift interpretation, regime change handling	gds-monitor
Orientation, profiling, clustering	gds-explorer

Full investigation examples: references/examples.md

Find hidden influencers — entities defining what "normal" means

Standard anomaly detection asks "how far is this entity from normal?". The inverse question is "how much does this entity SHAPE what 'normal' means?". A hidden influencer has high impact on coordinate system calibration but LOW anomaly score — invisible to anomaly scans, yet removing it shifts μ/σ enough to flip other entities' classifications.

mcp__hypertopos__find_calibration_influencers(
    pattern_id="<pattern>",
    classify="hidden",
    top_n=10,
)

Returns entries with total_impact (high), delta_norm (low), classification="hidden", top_dim_contributions[] (which dims drive the impact). For coordinated detection: collect 2-5 candidates and pass as one group to find_group_influence — reinforcing_factor > 1.5 confirms coordinated pull.

Common operational triggers:

• Data quality audit — hidden influencer may be a duplicated record warping the entire coordinate system. Find → fix data → rebuild → previously-masked anomalies surface.
• Adversarial AML — coordinated account injection to shift coordinates and mask fraud is the canonical hidden-influencer signature.
• Explaining anomaly flux — when compare_calibrations shows population shifted between epochs, hidden influencer analysis explains WHY.

verbose=True adds cascading_flip_count per entry — count of OTHER entities that flip is_anomaly classification after this entity's removal. Use when the candidate has high impact but you need a quantitative "blast radius" before recommending exclusion.

Anomaly by absence — find_density_gaps

find_anomalies surfaces entities at unusual positions in the geometry. find_density_gaps answers the inverse: which combinations of dim values should be populated under the independence null but are not? Useful when the investigation shifts from "who is anomalous" to "what's structurally missing".

result = mcp__hypertopos__find_density_gaps(
    pattern_id="account_pattern", top_n=5,
)

for gap in result["gaps"]:
    print(
        f'{gap["dim_i"]} ∈ [{gap["delta_range_i"][0]:.2g}, {gap["delta_range_i"][1]:.2g}] '
        f'AND {gap["dim_j"]} ∈ [{gap["delta_range_j"][0]:.2g}, {gap["delta_range_j"][1]:.2g}] '
        f'— observed {gap["observed"]}, expected {gap["expected"]:.1f}, '
        f'q={gap["q_value"]:.2e}'
    )

Range space note: delta_range_* is in z-score (delta) space, NOT raw property units. Dim labels with the _d_ prefix mean "delta of <property>"; e.g. _d_tx_count ∈ [-0.6, -0.4] is "tx_count z-scored against the population sits between -0.6 σ and -0.4 σ". Raw-property-range mapping is a follow-up; today inverse-transform from points table is your job if you want raw units in the narrative.

When the gap is real: independence + uniform-marginal expectation says the bin should hold ~N entities, observed is far below, q ≤ α after BH. Interpretation: there's structural avoidance of this combination — possibly business logic, regulatory cutoff, or a class of entities that exits when both features cross specific thresholds together.

When the gap is a binning artefact (FP): filter find_anomalies(filter=delta_range_i AND delta_range_j) and check the count. If the count is non-zero the cell was mis-binned at the histogram edge; deprioritise the cell. The probe report and tests cover the boundary cases, but real-data gaps benefit from one-shot manual TP-verify before acting on them.

Excluded dims: bernoulli, degenerate, and very sparse (<30 finite values) dims are auto-excluded and reported in excluded_dims with reason. If an expected dim is missing from the gap report, look here first.