Domain validator

Domain Validator

Agent output is a hypothesis. Domain validation is the test.

An agent that produces output without validation is a system that produces hallucinations at scale. The domain validator is the check that separates "the agent said so" from "it is true."

When to use

• After an agent produces output that feeds a downstream system or human decision
• When an agent has reasoned over domain-specific data (financial figures, medical records, legal clauses, system configurations)
• Before persisting agent-generated content to a database or document store
• When an agent output will be presented to an end user as factual

Procedure

1. Declare the domain rules — before running any validation, the domain rules must be explicit:

   • What are the invariants? (e.g. "a date range must have start < end", "a price must be positive", "a configuration must reference an existing resource")
   • What are the allowed value ranges or enumerations?
   • What is the ground truth source? (database record, API response, regulatory document, schema definition)

2. Extract the claims — identify the specific assertions in the agent output that are subject to validation. Not every word in the output is a claim; focus on structured data, named values, and factual assertions.

3. Validate each claim against the domain rules:

   • Structural validation: does the output conform to the expected schema or format?
   • Range and constraint validation: are values within allowed bounds?
   • Referential integrity: do referenced entities exist in the ground truth source?
   • Logical consistency: are the claims internally consistent? (e.g. no contradictory figures)
   • Freshness: is the ground truth source current, or could it be stale?

4. Classify findings:

   • PASS: claim is valid against all domain rules
   • WARN: claim is plausible but cannot be fully verified (e.g. ground truth unavailable)
   • FAIL: claim violates a domain rule or contradicts ground truth

5. Produce a validation report — for each claim: status (PASS/WARN/FAIL), the rule checked, and the evidence.

6. Gate downstream use — FAIL findings block downstream use of the output. WARN findings require explicit human acknowledgement before proceeding. PASS findings may proceed automatically.

Outputs

• Validation report: claim | status | rule checked | evidence
• Overall verdict: PASS / WARN / FAIL
• List of FAIL and WARN findings for human review

Guardrails

• Domain rules must be declared before validation runs. Validating against implicit rules produces false confidence.
• WARN is not PASS. A WARN finding means uncertainty, not safety.
• Ground truth must be identified. If there is no ground truth source, the output cannot be validated — flag this explicitly rather than assuming it is correct.
• Validation is not proofreading. Grammar and style are not domain rules. Focus on factual and structural correctness.

Anti-rationalization table

Excuse	Counter
"The model is reliable enough"	Reliability is a statistical claim. Domain validation is a deterministic check. Run it.
"We'll catch errors in review"	Human review misses structured errors that automated validation catches. Both are needed.
"The domain rules aren't defined yet"	Then the output cannot be trusted yet. Define the rules before relying on the output.