Traigent setup quickstart

Traigent Quickstart

When to Use

Use this skill when:

• Setting up Traigent for the first time in a new project
• Installing the SDK and configuring the environment
• Creating a first @traigent.optimize decorated function
• Building an evaluation dataset in JSONL format
• Verifying that the installation works correctly
• Running optimization in mock mode for development

Optimization Economics — Read This Before Sizing a Run

Do not default to recommending zero spend. The canonical Traigent posture on spending, the five characterization questions with their exact options, the tailoring rules (including the three-option paging rule), the explanation duty, and the local survey draft contract all live in one file that ships inside this skill: references/economics-characterization.v0.md. Read it from this skill's own directory before you propose, size, or decline a run — it is deliberately not restated here. It is generated from docs/shared/economics-characterization.v0.md in the traigent-skills repo, which is where any edit goes; the copy shipped here is byte-identical.

Characterize, never compute a budget locally. Collect the characterization and relay it; the Traigent service authors the budget, exactly as it authors the run-plan and the next-step decision — budget authorship belongs to the service. Do not compute, adjust, or recommend a budget locally: no budget arithmetic in markdown, no floor/cap table, no "roughly $X/day" of your own. The reference describes what the service computes; it is not a local calculator, and when the service returns no economics result, say so plainly and continue with no budget number at all rather than inventing one.

This skill's part: characterize the value the user's agent creates and relay it, so the service can size a bounded first experiment to it.

Mandatory whenever you relay any of it: show the options, recommend exactly one, and explain why in the user's own numbers — their agent, their volumes, their error costs. The explanation is a product requirement, not decoration.

Safety is unchanged and unweakened: mock/dry-run first, explicit user approval before any paid run, an explicit spend cap, and the recorded stop rule. The service sets how much to invest; it never affects whether approval is required — it always is.

Cold Start — First Value, One Step at a Time

When there's no prior run to look at, do not open with menus, methodology, or the advanced sections below. Detect cold start — the user is new to Traigent, has never viewed a run, and has no dataset or decorated function ready — and walk them to their first visible result one action at a time: one question or one command, wait, then the next. Never dump the whole pipeline at once.

The linear path:

1. Find the agent to optimize. Ask for (or detect from the project) the one function that calls an LLM. Just that — don't discuss knobs yet.
2. Find or make the dataset. If they already have labeled examples, hand off to traigent-dataset-curate for the one canonical dataset contract. If they have nothing ready, use the bundled fallback below — no dataset needed to see value.
3. Mock dry-run first. Always run keyless mock mode before anything paid (the "Your First Optimization" example below is exactly this). Show the ranked table so they see the loop work at zero cost and zero egress.
4. Confirm the real run explicitly. A real (paid) run happens only after the user sees the mock result and says go — and after the cost-gate approval. Never jump from mock straight to spend.
5. View it in the portal. Once a real run launches, watch the run start, watch the rows appear on the portal, and have the user open and inspect the run.

Where the plan comes from — honesty rule. When it's time to decide what to tune, ask the Traigent service for the run plan via the traigent-analyze-guidance skill (its traigent plan CLI / get_optimization_plan MCP tool). Present the one plan the service returns — not a menu you invented. There is no onboarding/phase parameter in the SDK or CLI today, so don't pass one or imply the client picks a phase. If the service can't return a plan, say so plainly and fall back to a generic, conservative knob family — model + temperature only, explicitly labeled as a generic fallback. Never encode task-specific ordering (scout/pivot/routing) in these docs; the service owns that. Tasks like text2SQL, RAG, classification, and extraction may be named as things the service plans for — but the plan logic stays server-side.

No agent or dataset yet? Start from references/first-value-fallback.md — a complete mock-first first-value path that needs neither, then gates any real-provider spend behind explicit approval and a cost_limit cap.

Once the user has seen a first result, hand off to the lifecycle skills rather than duplicating them here: traigent-dataset-curate (real data), traigent-setup-decorator (a real decorator), and traigent-analyze-guidance (what to do after a run). Do not surface advanced playbooks during cold start.

Installation

This is the Python SDK skill. Building in JavaScript/TypeScript instead? Use the traigent-js skill — it covers the native JS/TS optimizer (the @traigent/sdk package) with a different install and API. The rest of this page assumes Python.

Basic Install

The fast path is the core floored install — sufficient for the entire keyless mock quickstart because litellm ships in core. Install optional extras only after the mock run succeeds.

pip install "traigent>=0.19"
python - <<'PY'
import importlib.metadata as md
import traigent
v = md.version("traigent")
if v == "0.0.1" or not hasattr(traigent, "optimize"):
    raise SystemExit(
        f"Bad Traigent install: traigent {v}. "
        'You likely got the PyPI placeholder — reinstall with: python -m pip install --upgrade "traigent>=0.19"'
    )
print(f"traigent {v} OK")
PY

Warning: pip printing traigent 0.0.1 does not provide the extra ... is FATAL — you installed the placeholder package; reinstall with python -m pip install --upgrade "traigent>=0.19".

Literal First Run (execution-only agents)

Run this entire block in the foreground and wait for it to finish. Do not split it, background it, or continue after a failed command. The final expected line is TRAIGENT-DRY-RUN-OK.

#!/usr/bin/env bash
set -euo pipefail

# Mock/offline env is set in bash, BEFORE python imports anything.
export TRAIGENT_OFFLINE_MODE=true
export LITELLM_LOCAL_MODEL_COST_MAP=True

python -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade --timeout 60 --retries 5 "traigent>=0.19"
python - <<'PY'
import importlib.metadata as md
import traigent
v = md.version("traigent")
if v == "0.0.1" or not hasattr(traigent, "optimize"):
    raise SystemExit(
        f"Bad Traigent install: traigent {v}. "
        'You likely got the PyPI placeholder — reinstall with: python -m pip install --upgrade "traigent>=0.19"'
    )
print(f"traigent {v} OK")
PY

cat > ticket_eval.jsonl <<'JSONL'
{"input": "I was charged twice for my subscription", "output": "billing"}
{"input": "Please update the email address on my account", "output": "account"}
{"input": "The API returns a 500 error on POST requests", "output": "technical"}
{"input": "What are your business hours?", "output": "general"}
{"input": "My invoice has the wrong tax ID", "output": "billing"}
{"input": "I cannot reset my password", "output": "account"}
JSONL

cat > ticket_classifier.py <<'PY'
import traigent, litellm
from traigent.testing import enable_mock_mode_for_quickstart

enable_mock_mode_for_quickstart()


def mock_demo_accuracy(output, expected, config=None, **_):
    # Mock-only demo scorer; delete this for real runs.
    cfg = config or traigent.get_config() or {}
    base = 0.88 if cfg.get("model") == "gpt-4o" else 0.68
    return max(0.0, base - 0.04 * float(cfg.get("temperature", 0.0)))


@traigent.optimize(
    eval_dataset="ticket_eval.jsonl",
    objectives=["accuracy"],
    configuration_space={
        "model": ["gpt-4o-mini", "gpt-4o"],
        "temperature": [0.0, 0.7],
    },
    metric_functions={"accuracy": mock_demo_accuracy},
    offline=True,
)
def classify_ticket(query: str) -> str:
    config = traigent.get_config()
    response = litellm.completion(
        model=config["model"],
        temperature=config["temperature"],
        messages=[
            {"role": "system", "content": "Classify the ticket as: billing, technical, account, or general."},
            {"role": "user", "content": query},
        ],
    )
    return response.choices[0].message.content


results = classify_ticket.optimize_sync(max_trials=4, algorithm="grid")
print(f"Stop reason: {getattr(results, 'stop_reason', None)}")
print(f"Best config: {results.best_config}")
assert results.trials, "no trials ran"
assert not getattr(results, "failed_trials", []), f"failed trials: {results.failed_trials}"
assert results.best_config is not None, "no best config selected"
print("TRAIGENT-DRY-RUN-OK")
PY

python ticket_classifier.py

This block is generated from references/literal-quickstart.sh — edit that file, not the block.

Use a virtual environment (recommended)

Install into a project virtualenv — it's standard Python practice and it's the friction-free path here. A fresh venv is enough; you do not need --system-site-packages.

python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
python -m pip install --upgrade "traigent>=0.19"

pip install "traigent>=0.19" resolves from PyPI and pulls litellm (a core dependency) along with it, so the keyless mock path — which intercepts litellm.completion(...) — works immediately, with no extra install. Only the LangChain / OpenAI / Anthropic adapter clients live in the integrations extra (below).

Why a venv instead of system Python? On modern Debian/Ubuntu/Fedora the system interpreter is marked externally managed (PEP 668), so a bare pip install into it is refused with error: externally-managed-environment. The venv above avoids that entirely.

With Optional Extras

Install these after the keyless mock run succeeds.

# Recommended extras after the mock run succeeds
pip install "traigent[recommended]>=0.19"

# Framework integrations (LangChain, OpenAI, Anthropic, MLflow, W&B)
pip install "traigent[integrations]>=0.19"

# Analytics (numpy, pandas, matplotlib)
pip install "traigent[analytics]>=0.19"

# All optional features
pip install "traigent[all]>=0.19"

# Enterprise bundle (all production features)
pip install "traigent[enterprise]>=0.19"

See references/installation-extras.md for the full table of extras and their contents.

Requirements

• Python >= 3.11

Get Your Traigent API Key

Backend-connected features (the default cloud smart optimizer, dataset synthesis, analytics dashboards, the CI gate, and portal result history) all require TRAIGENT_API_KEY. There are two ways to obtain it:

Portal key (experiments-scoped)

1. Sign up at the Traigent portal and create a project.
2. In your project settings, go to API Keys and click Create key.
3. This issues a user-type key scoped to experiments:read experiments:write — sufficient for SDK optimizations and analytics.

export TRAIGENT_API_KEY="uk_..."   # portal keys use the uk_ prefix

CLI device-authorization key (project-scoped)

The CLI device-flow issues a project-scoped sk_-prefixed key with broader permissions (quota, dataset management, full project access). Use this when you need project-level operations beyond experiments.

Run traigent auth login in your terminal — it opens a browser for OAuth device authorization. The key is saved to your encrypted local credential store at ~/.traigent/secure_credentials.enc (secured with TRAIGENT_MASTER_PASSWORD); the legacy plaintext ~/.traigent/credentials.json is no longer read unless TRAIGENT_ALLOW_PLAINTEXT_CREDENTIALS=true is set for a one-time migration. Then export it:

export TRAIGENT_API_KEY="sk_..."

Which key to use? The portal experiments-scoped key is sufficient for most optimization workflows. Use the device-flow key for quota management, cross-project access, or when the CLI reports permission errors.

For the standard path, set TRAIGENT_API_KEY once, omit algorithm and offline, and let Traigent use the default cloud smart optimizer with portal result sync. Use algorithm="grid" or "random" only when you explicitly want local search; use offline=True only when zero egress is required.

Prereq for real (non-offline) runs: set TRAIGENT_API_KEY. The SDK defaults to the cloud backend (https://portal.traigent.ai) when TRAIGENT_BACKEND_URL is unset — no env var is needed for the standard cloud path. Set TRAIGENT_BACKEND_URL only to target a dev or self-hosted backend (e.g. http://localhost:5000). Exception: the traigent next-steps and traigent plan CLI commands resolve --backend-url as flag → TRAIGENT_BACKEND_URL / the URL stored by traigent auth login → a local default (http://localhost:5000), so for cloud use log in first or pass the flag/env var explicitly. plan/next-steps honor the stored auth-login URL since 0.20.0 (see version-matrix: backend-url) — on SDK <= 0.19.x these two commands ignored the stored traigent auth login URL and always defaulted to localhost unless the flag or env var was passed (Traigent/Traigent#1721). Portal-issued API keys use the uk_... prefix.

export TRAIGENT_API_KEY="uk_..."                              # portal-issued key
export TRAIGENT_BACKEND_URL="https://portal.traigent.ai"     # optional: cloud is already the default

Key Hygiene — Reference Names, Never Values

Observed across multiple coding-agent CLI families (Codex, Claude, Gemini), not one tool's quirk — treat every rule below as universal regardless of which agent is driving the session.

• Never print, echo, or log a key value. echo $TRAIGENT_API_KEY, a bare env/printenv dump, and set -x/bash -x wrapped around any key-touching command are all leaks — each writes the raw secret to stdout, which most agent harnesses capture into the visible transcript and often into a log file too.
• Reference keys only by env-var name (TRAIGENT_API_KEY, OPENAI_API_KEY, ...) in commands, code, and chat — never paste or reconstruct the value itself.
• Treat transcripts and logs as shareable artifacts. Assume anything printed to the terminal or written to a log may be read, copied, or shared later; keep secrets out of both.

Environment Setup

Development Mode (Recommended for Getting Started)

Mock mode is the keyless dev path for provider calls — LLM calls are intercepted and return canned responses. Activate it in code:

from traigent.testing import enable_mock_mode_for_quickstart

enable_mock_mode_for_quickstart()

<!-- PROTECTED -->

• enable_mock_mode_for_quickstart() is the recommended activation path. It is hard-blocked when ENVIRONMENT=production and emits a once-per-process WARNING so a test that accidentally runs in a deployed system is loud and visible.

<!-- /PROTECTED -->

• Mock scope: only LiteLLM (litellm.completion) and LangChain (ChatOpenAI, ChatAnthropic, etc.) calls are intercepted. Raw openai.OpenAI() / anthropic.Anthropic() clients are not intercepted — a function using a raw client will make real, billable calls in mock mode. Use LiteLLM in examples that must run keyless.
• No separate install needed for mock: litellm ships with the SDK core (pip install "traigent>=0.19" pulls it), so litellm.completion(...) is interceptable the moment Traigent is installed — you do not need to pip install litellm yourself. (LangChain adapters do require pip install "traigent[integrations]>=0.19".)
• Mock ≠ offline. Mock stops LLM cost (calls are intercepted) — it does not stop backend egress. With TRAIGENT_API_KEY set and the default offline=False, a "mock dry-run" is still sent to the Traigent backend and appears on your portal as a mock-data experiment (and counts against quota). For a fully local, private dry-run, also pass offline=True (or run with no key). enable_mock_mode_for_quickstart() alone does not make a run local.
• Real metrics read 0.0 under mock. Every intercepted call returns the same canned text, so exact/execution-match scorers score a uniform 0.0 across trials — expected in mock, not a broken pipeline (that is exactly why the example below wires a mock-only demo scorer).

Legacy Env-Var Path

<!-- PROTECTED -->

The previous quickstart docs taught export TRAIGENT_MOCK_LLM=true. That env var still works in non-production environments for backward compatibility with existing fixtures, but it is hard-blocked when ENVIRONMENT=production (an OSError is raised at SDK import). Prefer the in-code API for new code.

<!-- /PROTECTED -->

Using a .env File

Traigent supports .env files via python-dotenv (included in the integrations extra). Create a .env file in your project root:

TRAIGENT_API_KEY=uk_...   # portal key; use your sk_... key here if you used the CLI device flow
OPENAI_API_KEY=sk-...
TRAIGENT_DEBUG=1

Recommended: have the user paste keys into .env, never into the chat

When a user says "I have my key", do not ask them to type the secret into the conversation — it would be captured in the agent transcript, logs, and context. Open the .env file for them to paste into directly. This is both more secure (the raw key never touches the chat) and better UX (they see exactly where it goes). Procedure:

1. Create the file from the project template if one exists (cp .env.example .env), otherwise create a minimal .env with key names pre-filled and values blank, so the user only pastes after each =:

   TRAIGENT_API_KEY=
   # provider key — fill the one(s) this project uses:
   OPENAI_API_KEY=
   # ANTHROPIC_API_KEY=
   # Bedrock: AWS_ACCESS_KEY_ID= / AWS_SECRET_ACCESS_KEY= / AWS_REGION=
   

2. Always show the user the absolute path (e.g. /home/me/proj/.env). This is the guaranteed fallback — they can open it in their own editor no matter what happens next.

3. Best-effort: pop the file open in a standalone editor window, launched detached. Pick the launcher by OS; never wrap it in timeout:

   • Linux: setsid -f gnome-text-editor "$ENV" — or the first of kate / gedit / xed / mousepad that exists; last resort xdg-open "$ENV".
   • macOS: open -t "$ENV" (opens the default text editor in its own window).
   • Windows: start "" notepad "%ENV%" (Notepad is always present), or Start-Process notepad "$env:ENV" in PowerShell.

   Pitfalls that look like success but aren't (verified the hard way):

   • Don't open via the user's IDE (code <file> / cursor <file>): it can spawn a nested instance that crashes, and it hijacks whichever IDE window is focused — so .env can pop up inside an unrelated project.
   • Don't trust the launcher's exit code as "opened" — a crashed window can still exit 0. Verify the editor process is actually alive (e.g. pgrep) and ask the user to confirm the window appeared.

4. Pick the provider key by detecting the vendor from the project (its openai / anthropic / litellm / Bedrock imports or config). If the vendor is ambiguous, undetectable, or the project uses multiple providers (e.g. OpenAI and Bedrock), ask the user which provider(s) and label the matching key(s) in .env.

5. Wait for the user to paste and save. Confirm .env is in .gitignore.

6. Fallback: if no standalone editor opens (or the user says no window appeared), have them open the printed path manually; only as a last resort use a terminal export VAR=... (less private than the file).

Never echo, log, or read back the key value. .env must be git-ignored — never commit real keys.

Production Mode

For production, set your provider API keys and don't call enable_mock_mode_for_quickstart():

export OPENAI_API_KEY=sk-...
# or
export ANTHROPIC_API_KEY=sk-ant-...

Before your first real run, verify your model IDs are live. Provider catalogs change — a delisted or renamed ID causes a 404 or a degraded/unpriced trial. Preflight with traigent models --provider <p> --check <model_id> (see the CLI Quick Reference below) to catch typos and unknown ID shapes — but know its limit: the check validates against a shipped model snapshot with pattern-based fallback, so a retired-but-well-formed ID still passes; it does not detect provider-side delisting. To confirm an ID is actually live, query the provider's catalog directly (e.g. curl -s https://openrouter.ai/api/v1/models for OpenRouter). The traigent-setup-integrations skill covers multi-provider model verification.

See references/environment-variables.md for all available environment variables.

Your First Optimization

Always dry-run first. Before a real (paid) run, run in mock mode, review the cost estimate, and get explicit approval. See the traigent lifecycle skill for the mandatory dry-run-first / cost-approval workflow.

Real LLM runs require cost approval. A real (non-mock) optimization is blocked by a cost gate. To confirm you accept the cost, set TRAIGENT_COST_APPROVED=true in the environment (the verified path); some SDK versions also accept cost_approved=True in the @traigent.optimize() decorator. The SDK prints an estimate before any trial executes; the estimate may be high (fallback pricing is conservative), but the gate is a safety confirmation — nothing runs until you approve.

Tiny Real Cost and KPI Probe

After the mock dry-run passes and before any full run, run one tiny real optimization: 1-2 dataset examples, minimal trials, and the cheapest candidate model. Check both surfaces before scaling up: results.total_cost must be neither None nor 0.0 with real calls (both mean cost is not wired), and each trial's metrics must contain the declared objectives with non-degenerate values (not all 0.0/all 1.0). If either surface fails, wire it first — see traigent-optimize-run → Cost Wiring Probe for the fix ladder (custom model pricing env vars, per-trial cost metrics, TRAIGENT_STRICT_COST_ACCOUNTING).

Objective naming rule: Default: at least one objective labeled accuracy (built-in objective or your metric_functions key). If accuracy doesn't apply to this problem, name the primary quality KPI after the product concept, for example valid_schema, and note why accuracy was skipped.

Here is a complete working example. This function classifies customer queries using an LLM, and Traigent will find the best model and temperature combination.

Note on mock scope: enable_mock_mode_for_quickstart() intercepts LiteLLM and LangChain calls. Raw openai.OpenAI() / anthropic.Anthropic() client calls are not intercepted — use litellm.completion() for a fully keyless dry-run (see references/installation-extras.md for traigent[integrations]).

Why the example includes a mock_demo_accuracy scorer: in mock mode every LLM call returns the same canned string, so a real accuracy metric scores every trial 0.0 — a discouraging all-zeros table that looks broken. The demo scorer below ignores the (canned) output and ranks trials by their config, so the keyless dry-run produces a meaningful table — the same approach the bundled traigent quickstart command uses. It is mock-only: delete it for a real run, where Traigent scores actual model output against your dataset labels. (Prefer not to keep a placeholder scorer in your own code? Just run traigent quickstart for the same ranked demo without writing one.)

import asyncio
import os
from pathlib import Path

# Set no-egress flags before importing Traigent or LiteLLM.
os.environ["TRAIGENT_OFFLINE_MODE"] = "true"
os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] = "True"

import litellm  # pip install "traigent>=0.19"
import traigent
from traigent import Choices
from traigent.testing import enable_mock_mode_for_quickstart

# Step 1: dry-run in mock mode — no API keys required, no cost
enable_mock_mode_for_quickstart()

Path("eval_queries.jsonl").write_text(
    "\n".join(
        [
            '{"input": "I was charged twice for my subscription", "output": "billing"}',
            '{"input": "The API returns a 500 error on POST requests", "output": "technical"}',
            '{"input": "What are your business hours?", "output": "general"}',
        ]
    )
    + "\n",
    encoding="utf-8",
)


def mock_demo_accuracy(output, expected, config=None, **_):
    """Mock-only demo scorer — DELETE this for a real (paid) run.

    In mock mode every ``litellm.completion`` call returns the same canned
    string, so a real accuracy metric would score every trial 0.0 (a
    misleading all-zeros table). This placeholder ignores ``output`` and ranks
    trials by their config so the keyless dry-run produces a meaningful table —
    the same trick the bundled ``traigent quickstart`` demo uses. On a real run,
    remove it and let Traigent score actual model output against your labels.
    """
    cfg = config or traigent.get_config() or {}
    base = 0.85 if cfg.get("model") == "gpt-4o" else 0.65
    return max(0.0, base - 0.05 * float(cfg.get("temperature", 0.5)))


@traigent.optimize(
    eval_dataset="eval_queries.jsonl",
    objectives=["accuracy"],
    algorithm="random",
    offline=True,
    model=Choices(["gpt-4o-mini", "gpt-4o"]),
    temperature=Choices([0.0, 0.5, 1.0]),
    metric_functions={"accuracy": mock_demo_accuracy},  # mock-only; delete for a real run
)
def classify_query(query: str) -> str:
    config = traigent.get_config()
    # Use litellm so mock mode intercepts the call (raw openai client is NOT intercepted)
    response = litellm.completion(
        model=config["model"],
        temperature=config["temperature"],
        messages=[
            {"role": "system", "content": "Classify the query as: billing, technical, or general."},
            {"role": "user", "content": query},
        ],
    )
    return response.choices[0].message.content


async def main():
    # Step 1: dry-run (mock) — confirm the setup works and review estimated cost
    results = await classify_query.optimize(max_trials=6)

    # Inspect results
    print(f"Best config: {results.best_config}")
    print(f"Best score:  {results.best_score}")
    print(f"Trials run:  {len(results.trials)}")

    # Apply the best configuration for production use
    classify_query.apply_best_config(results)

    # Now calling the function uses the best config
    answer = classify_query("I can't log in to my account")
    print(f"Classification: {answer}")


asyncio.run(main())

In a notebook (Jupyter/IPython/Colab)? asyncio.run() raises RuntimeError: asyncio.run() cannot be called from a running event loop there — the notebook already runs one. Use await main() directly in a cell, or the synchronous alternative below.

Synchronous Alternative

If you prefer synchronous execution:

results = classify_query.optimize_sync(max_trials=6)  # uses the offline random dry-run settings above

expected is a scoring label — do not put it in your agent function's signature. The evaluator calls your function with the example's input fields only (plus any config-injected params). It then passes the function's output and the dataset's expected / output field to your scoring_function or metric_functions. A function that declares expected as a parameter will fail every trial with TypeError: missing required argument: 'expected'.

# WRONG — fails every trial
def classify_query(query: str, expected: str) -> str: ...

# CORRECT — function takes input fields only; scorer receives (output, expected)
def classify_query(query: str) -> str: ...
def score(output: str, expected: str) -> float: ...

Key Concepts

1. @traigent.optimize(...) -- Decorator that wraps your function for optimization. Define what parameters to tune in the decorator arguments.
2. traigent.get_config() -- Call inside your function to retrieve the current trial's configuration. Works during optimization trials and after apply_best_config().
3. func.optimize(max_trials=N) -- Run the optimization loop asynchronously. Returns an OptimizationResult.
4. func.apply_best_config(results) -- Lock in the best configuration found so that subsequent calls use it.

You've run your first optimization — now make it robust. The decorator above is intentionally a local dry-run recipe: a small model + temperature space, algorithm="random", and offline=True. For a real optimization, graduate to the more robust traigent-setup-decorator skill — custom evaluators / metric_functions, injection mode, execution policy, and weighted objectives — then launch with traigent-optimize-run, which adds what a real run needs beyond the basic .optimize() call: cost limits (cap a paid sweep before it overruns), algorithm choice ("auto" for connected real runs; "grid"/"random" for explicit local/offline search; named smart selectors like bayesian/optuna on authenticated connected runs, SDK 0.20.1+), parallel trials, and quota-aware run sizing. That decorator-setup → run-optimization pair is the recommended path from "first run" to a production optimization.

Dataset Format

Traigent uses JSONL (JSON Lines) files for evaluation datasets. Each line must have an input field and an output field.

Example: eval_queries.jsonl

{"input": "I was charged twice for my subscription", "output": "billing"}
{"input": "The API returns a 500 error on POST requests", "output": "technical"}
{"input": "What are your business hours?", "output": "general"}

• input -- The value passed to your function during evaluation.
• output -- The expected/ground-truth result used for scoring.

You can include additional fields for metadata, but input and output are required.

One canonical contract. This flat form is the simplest case of the single Traigent dataset contract: input (or input_data) → your function's args, the gold key → the expected value, and every other top-level key → example.metadata[...] (how a per-example side field like a db_path reaches a scorer). Values may be nested dicts, and the gold key has accepted aliases (output, expected, expected_output, answer, target, label). See traigent-dataset-curate for the full contract and traigent-eval-build for an execution-scored recipe that reads a metadata field.

Dataset path sandbox. The SDK enforces a TRAIGENT_DATASET_ROOT / CWD sandbox — an eval-dataset path outside the sandbox root (default: the current working directory) is rejected at load with a ConfigurationError ("Dataset path must reside under …"). Keep the JSONL under the directory you run from, or set TRAIGENT_DATASET_ROOT to the directory containing your datasets.

Tips for Good Datasets

• Include at least 10-20 examples for meaningful optimization.
• Cover edge cases and diverse inputs.
• Ensure ground-truth output values are consistent and well-defined.
• Reserve a holdout slice before tuning; never tune and validate on the same rows.
• For evaluation dataset creation beyond this minimal JSONL, use traigent-dataset-curate.

Verify Installation

Check SDK info

traigent info

This prints the installed version, Python version, available integrations, and optimization defaults.

Verify from Python

import traigent
print(traigent.get_version_info())

Validate an evaluation dataset

traigent validate eval_queries.jsonl

CLI Quick Reference

Start here (keyless, no API key required):

traigent quickstart      # bundled working demo — mock mode, zero config
traigent onboard         # guided first-run setup wizard

Command	Description
traigent quickstart	Run the bundled mock-mode demo (keyless, zero-setup, always works)
traigent onboard	Guided setup for Traigent in this project (API key, project, env)
traigent models	List/validate model IDs before a run, e.g. traigent models --provider anthropic --check claude-3-haiku-20240307 (ID-shape preflight against a shipped snapshot; confirm liveness via the provider's catalog)
traigent recommend	Evidence-backed TVAR recommendations for your agent/task type
traigent recommend-eval	Metric and evaluator recommendations for your task type
traigent generate-config	Scaffold a full @traigent.optimize() config for your function
traigent detect-tvars	Detect tuned-variable candidates in existing Python files
traigent info	Show SDK version, environment, and integrations
traigent algorithms	List available optimization algorithms
traigent validate	Validate dataset files and configuration

Next Steps

• Configure your decorator for a real optimization -- The example above is the minimal decorator. To make it optimization-ready -- custom evaluators / metric_functions, injection mode, execution (algorithm/offline), and weighted objectives -- use the traigent-setup-decorator skill, then launch with traigent-optimize-run. This decorator-setup → run-optimization pair is the standard two-step cycle for going from "first run" to a real optimization.
• Dry-run before a real run -- See the traigent lifecycle skill for the mandatory dry-run-first / cost-approval workflow before any paid execution.
• Mind your plan quota -- Cloud optimization is metered by optimization_samples (~max_trials × dataset_size per run) and optimization_trials, separate from dollar cost. Check usage and size large runs to fit; see the traigent-optimize-run skill ("Quota & Run Sizing").
• Define parameter search spaces -- See the traigent-optimize-config-space skill for Range, IntRange, Choices, LogRange, factory presets, and constraints.
• Choose an optimization algorithm -- For connected real runs, omit algorithm or use "auto"; use "grid"/"random" for explicit local/offline search. traigent algorithms lists auto + smart names since 0.20.1 (see version-matrix: algorithms-cli) — the full public selector surface with a local/connected availability column (on 0.20.0 the CLI omitted auto and the smart names: Traigent/Traigent#1751). Named smart selectors execute on connected runs since 0.20.1 (see version-matrix: smart-selector-exec): bayesian/tpe/optuna/optuna_tpe/optuna_random bind to the typed backend Optuna strategy, while unsupported smart names such as nsga2/cmaes fail fast with a capability message (Traigent/Traigent#1752, #1758; on 0.20.0 no smart name executed end-to-end). With offline=True every smart name still raises ConfigurationError at decoration time (verified on 0.21.0). The traigent-optimize-run skill owns the full selector contract.
• Add multiple objectives -- Use objectives=["accuracy", "cost", "latency"] for multi-objective optimization.
• Use framework integrations -- Install traigent[integrations] for LangChain, OpenAI, and Anthropic adapters.
• Verify model IDs before a real run -- Catalogs change; run traigent models --provider <p> --check <id> to catch typos and unknown ID shapes, then confirm the ID is actually live against the provider's catalog — the CLI check validates a shipped snapshot with pattern fallback and cannot detect provider-side delisting, so a retired/renamed ID can still pass and cause a 404 or a degraded, unpriced trial. See traigent-setup-integrations.

<!-- Reserved: managed longitudinal-guidance region. Step-level edits must not write here. --> <!-- SLOW_UPDATE --> <!-- /SLOW_UPDATE --> <!-- INTERACTION_POLICY v1 (synced — do not edit inline; edit docs/shared/interaction-policy.v1.md) -->

Traigent Interaction Policy

Track an interaction profile and adapt to it. Persona (stable): control=delegate|guided|inspect, expertise=se|ds|unknown. Mood (this session): pace=execute|balanced|explore. Default when unknown: guided,se,balanced. Infer from explicit user statements first, then recent behavior; an explicit correction wins immediately. Never store or send this profile anywhere by default.

Fetch the live profile (when available)

At session or skill start, if a configured Traigent client is available, seed the profile from the backend with the skill name:

policy = None
try: policy = await client.get_interaction_policy(skill="<this skill>")
except Exception: pass

Treat the returned profile as the STARTING seed: its control/expertise/pace axes plus question_budget, options_max, and jargon_level replace the static defaults below. Explicit user corrections in-conversation ALWAYS override the seed. If the call is unavailable or fallback_policy="static_v1", simply use the static defaults below; the SDK already fails soft.

• Always be concise.
• Match terminology to expertise. For se: plain engineering words; define each Traigent or statistics term once in plain language (no Bayesian / variance-decomposition / Pareto jargon unless asked). For ds: compact optimization and statistical terms are fine.
• Presenting options: show at most 3, mark exactly one Recommended, and give one short persona-appropriate trade-off per option.
• Autonomy. For delegate or execute: pick the recommended reversible action and proceed, asking only at hard gates. For guided: offer options with a recommendation at the key decisions. For inspect or explore: give brief rationale or evidence before asking, and ask before branch choices.
• Hard gates — always confirm regardless of persona: paid or provider model calls, sending data or private content off the machine, destructive edits, decisions the Traigent service is meant to return, and any missing fact the step truly requires.
• Always end by recommending the next Traigent skill or action to take.
• Never weaken Traigent safety: dry-run before any paid run; get explicit approval before real cost or before any data leaves the machine; treat service-returned plans and next steps as authoritative. Never put the persona profile or any private content into telemetry, run metadata, experiment names, logs, or provenance files.

<!-- /INTERACTION_POLICY v1 -->