Zk llm design

/zk-llm-design

Overview

Designing a zero-knowledge LLM system is mostly about answering four questions in order: who do you not trust, what are you proving, which scheme fits the proof, and is it actually feasible at your model size. Most teams skip straight to scheme selection and end up with a system that proves the wrong thing or cannot run.

Core principle: the threat model picks the proof; the proof picks the scheme; the model size picks whether the scheme is feasible. Reverse that order and you waste months.

When to Use

• User typed /zk-llm-design or asked about ZK-ML, verifiable inference, trustless inference, model integrity proofs, or private inference.
• User wants to give a third party a guarantee about an LLM's output without revealing the model, the input, or both.
• User is comparing Halo2, Plonk, STARKs, EZKL, or FHE for an LLM use case.

Do NOT use for: general LLM serving, RAG, fine-tuning, or "we want to run our model securely" without a defined trust requirement (that is just operations security, not ZK).

The Process

Step 1. Build the threat model

Ask the user, in order:

1. Who is the verifier? End user, regulator, auditor, paying customer, on-chain contract?
2. Who is the prover? Model host, user themselves, a third party?
3. What does the verifier not trust the prover to do honestly? Run the right model? Run on the right input? Not see the input? Not see the output? Not lie about the output?

Write the answers down explicitly. If the verifier and prover are the same party, the user does not need ZK. Stop and tell them.

Step 2. Define what is being proved

Pick one or more from this menu, with a one-sentence statement of the proof for each:

Proof	Statement	Common use
Model integrity	"I ran model M (whose hash is H) and got output Y from input X."	Auditable inference, regulated industries
Input integrity	"Output Y comes from an input X that satisfies predicate P, without revealing X."	Private user data, KYC-style claims
Weight privacy	"I ran some model with public commitment C on input X and got Y, without revealing the weights."	Proprietary model serving
Output property	"The output Y satisfies property P (e.g. classification = malicious)."	Trust-minimized moderation
Training provenance	"Model M was trained on a dataset whose root commitment is R, with procedure described by S."	Provenance, copyright defense

If the user wants more than two of these in one system, push back. Composing them grows prover cost super-linearly.

Step 3. Pick the scheme

Use this table as a starting point, then verify with current docs (the field moves; numbers below are rough as of mid-2026):

Scheme	Strength	Weakness	Fit for LLMs
Halo2 (KZG / IPA)	Mature tooling (EZKL builds on it), recursion friendly, smallish proofs	Trusted setup (KZG) or large proofs (IPA), prover memory heavy	Best general-purpose pick today for inference proofs up to ~1B params with quantization
Plonk family (gnark, plonky2)	Universal setup, good ergonomics	Prover memory still high for large models	Comparable to Halo2; ecosystem-driven choice
STARKs (Risc0, SP1)	No trusted setup, post-quantum, executes general programs	Larger proofs, slower verification on-chain	Good when you want to prove arbitrary code (e.g. tokenization + inference) without circuit work
FHE (Concrete-ML, TFHE-rs)	Computes on encrypted inputs, no proof needed for input privacy	Massive overhead (10^4 to 10^6x); not a proof system, complementary	Only when input privacy is the entire requirement and you can tolerate latency
TEE-attested (SGX, Nitro, H100 confidential)	Practical at any model size today	Trust assumption on hardware vendor; not "zero knowledge" in the cryptographic sense	The honest answer for many production needs in 2026; mention when ZK is overkill

If the user's answer to step 2 was just "model integrity" and the verifier is willing to trust hardware attestation, recommend TEE first. ZK is the right answer when the verifier cannot or will not trust hardware vendors.

Step 4. Feasibility check

Quick gate: estimate prover cost.

• Up to ~100M params, INT8 quantized: Halo2 / EZKL is feasible on a single high-RAM machine, proofs in minutes.
• 100M to 1B params: EZKL with aggressive quantization, GPU-accelerated proving, expect tens of minutes to hours per inference.
• Above 1B params: today, only proof of small per-token operations (e.g. last-layer logit checks), not full inference. Tell the user honestly.
• GPT-class (10B+): not currently feasible end-to-end. Either reduce scope (prove only the safety classifier on top), use TEE attestation, or wait.

If the model is too large for the chosen scheme, do not paper over it. Recommend scope reduction.

Step 5. Output

Produce a one-page design doc:

# ZK-LLM design: [project name]

## Threat model
- Verifier: ...
- Prover: ...
- Distrust statement: ...

## Proof statement
[Exact one-sentence statement of what the prover proves]

## Scheme
[Choice + 2-sentence justification]

## Feasibility
- Model size: ...
- Quantization: ...
- Estimated prover cost per inference: ...
- Estimated proof size and verification cost: ...

## Open risks
[2 to 4 bullets covering the most likely ways this goes wrong]

## Smallest viable v0
[The narrowest version of this that ships in 4 to 8 weeks]

End with a recommendation: ship the smallest viable v0 first, or wait for the field to catch up.

Common failure modes

Failure	Fix
Picked a scheme before defining the proof	Restart at step 1. The threat model picks the proof; the proof picks the scheme.
Conflated FHE with ZK	They solve different problems. FHE hides inputs; ZK proves computation. Compose them only if you genuinely need both.
Over-scoped the proof	Two proof types maximum. Composing more grows cost super-linearly.
Ignored TEE as an option	If the verifier will trust hardware, TEE ships in weeks instead of months. ZK is for when they will not.
Quoted prover cost numbers without verifying	The field moves quarterly. Always check current EZKL / Risc0 / SP1 benchmarks before promising numbers.
Promised end-to-end ZK on a 10B+ model	Not feasible today. Reduce scope or pick a different mechanism.

When to chain to /research

If the user is committing real engineering time, suggest running /research against the chosen scheme + model-size combination first. Specifically: search for the most recent EZKL or Risc0 benchmarks at the user's model size. Numbers in this skill go stale fast.