Kookr terminal backend

Kookr Terminal Backend

Kookr runs agents (Claude Code, Codex CLI) in persistent terminal sessions so the agent survives Kookr restart. Two backends coexist in the code:

• dtach (LocalDtachBackend at src/adapters/local-dtach-backend.ts) — current default as of v7 Main B.b (#343, 2026-04-21). Byte-transparent. Unix-socket-based. Selected when KOOKR_BACKEND !== 'tmux'.
• tmux (TmuxTerminalManager at src/adapters/tmux-terminal-manager.ts) — legacy. Still constructed at startup. Kept for the KOOKR_BACKEND=tmux rollback path. Scheduled for deletion in V8 (docs/rfc/rfc-v8-tmux-removal.md).

Two interface shapes (know which one you're holding)

TerminalManager (legacy, tmux-shaped)
├── createSession(name, command, options)
├── sendKeys(name, keys)           // text + Enter
├── pasteText(name, text)          // bracketed paste (Codex only)
├── sendKeystroke(name, key)       // single key, no Enter
├── capturePane(name, lines?)      // rendered screen text
├── isAlive(name)
├── killSession(name)
├── listSessions(prefix?)
└── setSessionOption(name, opt, val)  // tmux-only

TerminalBackend (new, backend-agnostic)
├── createSession(spec: SessionSpec): Promise<SessionHandle>
├── attachSession(id): Promise<SessionHandle>
├── listSessions(): Promise<SessionId[]>
├── isAlive(id)
└── killSession(id)

SessionHandle (returned by createSession/attachSession)
├── write(data: Uint8Array)    // byte-transparent
├── resize(cols, rows)
├── onData(cb)                 // subscribe to child PTY bytes
├── onExit(cb)
└── dispose()                  // detach; child survives

Backend selection at startup

src/server/start.ts:42:

const BACKEND_KIND: 'tmux' | 'dtach' = process.env.KOOKR_BACKEND === 'tmux' ? 'tmux' : 'dtach';

• BACKEND_KIND === 'dtach' → LocalDtachBackend constructed + passed to adapters as options.backend.
• TmuxTerminalManager is always constructed (start.ts:180) and passed as terminal, regardless of BACKEND_KIND. This is intentional — the adapters' legacy method paths still thread through it.

The critical seam: adapter partial wiring

ClaudeCodeAdapter / CodexCliAdapter receive both terminal and backend:

// launch() — uses backend when set:
if (this.backend) {
  await this.backend.createSession({...});   // dtach path
} else {
  await this.terminal.createSession(...);    // tmux path
}

// sendInput / sendKeystroke / stop / captureDisplay — ALWAYS use terminal:
await this.terminal.sendKeys(tmuxName, text);

Consequence: when KOOKR_BACKEND=dtach, sessions live in dtach, but per-session ops still call tmux send-keys -t kookr-XXXX → throws "no such session" → breaker increments.

V8 fixes this by porting per-session ops onto the backend (new backend.write(id, bytes) + writeSequence(id, payloads[])).

The CircuitBreaker (currently named 'tmux')

src/server/index.ts:220:

const tmuxBreaker = new CircuitBreaker({ name: 'tmux', failureThreshold: 5, failureWindowMs: 60_000, resetTimeoutMs: 15_000 });
const wrappedTerminal = new CircuitBreakerTerminalManager(terminal, tmuxBreaker);

• Only wraps TerminalManager calls (the legacy path).
• Does not wrap dtach backend calls.
• The "tmux" label is correct re: implementation but misleading re: operator intent — failures it catches are always adapter-legacy-path calls failing for missing tmux sessions.

Dashboard symptom: breaker showing open state with 3+ failures when KOOKR_BACKEND=dtach is the default = not a tmux bug, it's the adapter-partial-wiring bug. RFC v8 deletes the breaker entirely in favor of backend-emitted structured BackendError events.

WebSocket routing: three bridge kinds

src/server/index.ts:745-786 — when a browser xterm.js connects to /ws/terminal/<session>:

1. FakeTerminalBridge — E2E / demo mode (useFakeTerminalBridge).
2. SessionBridge (new, byte-transparent) — if terminalBackend?.isAlive(id) returns true. Uses backend.attachSession() + ring buffer.
3. TerminalBridge (legacy, tmux) — fallback. Spawns tmux attach -t <id> via node-pty. Used when dtach doesn't know the session (e.g., pre-V7 tmux-era session, or a stale session ID from task store).

Sessions with kind=terminal bridge opens in the log are the tell-tale sign of a task whose session ID points to a session that dtach does not have.

dtach on-disk layout

• Sockets: /tmp/kookr-dtach/<uid>/<instanceId>/<sessionId>.sock (e.g. /tmp/kookr-dtach/1000/port-4800/kookr-c1c8bcc4.sock).
• Manifest: /tmp/kookr-dtach/<uid>/<instanceId>/manifest.json — array of {sessionId, pid, startedAt, status, sock}. Written atomically via temp+rename. LocalDtachBackend owns this exclusively.
• Per-instance isolation: instanceId = port-${PORT}. kookr-prod (port 4800) and dev (4801) never collide.
• Session ID cap 40 chars — keeps socket paths under Linux UDS 107-byte limit.

Reconciliation

src/server/reconciliation.ts:

• Takes a TerminalManager (NOT wrappedTerminal — not breaker-guarded). Calls terminal.listSessions('kookr-').
• Problem for dtach: tmux's list returns [] when there's no tmux server running; this silently makes ALL task sessions look dead to reconcile.
• Fixed partially in PR #346 (commit a42ccfd — "reconcile queries dtach backend") but reconcile still calls TerminalManager.
• V8 changes the signature to reconcile(taskStore, backend: TerminalBackend).
• reconcile() only COMPUTES the orphan set (sessions the backend reports live that no task accounts for — including a session whose OWN record already carries lastStatus: 'completed'/'aborted', since the loop continues past those without marking them accounted-for). It never acts on it.

Session reaping (issue #1720)

src/server/session-reaper.ts (SessionReaperService) is the separate "act on it" step run after every reconcile() call (boot + periodic liveness tick, wired from index.ts / lifecycle-timers.ts):

• Cross-references every backend-reported live session against the TaskStore itself (not reconcile's output) and classifies each with the pure src/core/session-reap-policy.ts logic into unowned (leak class 1 — the original orphan report) or terminal-task-leak (leak class 2 — task already completed/terminated/cancelled, or its session record already terminal, but the process tree is still resident; e.g. completeTask's fire-and-forget adapter.stop() silently failed).
• Reaps via the backend's existing killSession (TERM → grace → KILL + socket removal) once past an age/grace threshold (KOOKR_REAP_ORPHAN_AGE_MS default 24h for unowned sessions — never race a mid-launch session per #1537 item 2 — or the shorter KOOKR_REAP_ORPHAN_AGE_UNDER_PRESSURE_MS default 2h when dtach count ≥ soft bound 20, issue #2081; KOOKR_REAP_TERMINAL_TASK_GRACE_MS default 60s for owned leaks, since the owning task is already known to be done). Master flag KOOKR_REAP_ORPHAN_SESSIONS (default on). Each sweep logs effectiveOrphanAgeMs and surfaces it on /api/health → sessionReaper.
• Boot-only runStaleAttacherSweep (backed by src/adapters/dtach-attach-reaper.ts) kills leaked dtach -a attach-client processes from dead server generations (leak class 3), scoped to LocalDtachBackend.getInstanceDir() so it never touches another port's sessions or a user's own terminal.
• Every reap is logged to audit.jsonl and surfaced cheaply (cached counters, no re-scan) on GET /api/health's sessionReaper block.

Gotchas

• tmuxName parameter name is load-bearing in logs but has nothing to do with tmux post-v7. Leaky abstraction. Planned for rename in a follow-up PR.
• session.tmuxSession field in persisted tasks.json — same story. Renaming requires a schema-migration step.
• TmuxTerminalManager swallows some errors, throws others. listSessions, killSession, isAlive, setSessionOption swallow "no such session." sendKeys, capturePane, pasteText, sendKeystroke throw it. Breaker sees only the throwing paths.
• TerminalBridge spawns tmux attach via node-pty directly — does NOT go through the breaker. So bridge open failures don't count toward breaker.
• Cutover detection runs on every dtach startup (start.ts:124-164) — runs tmux list-sessions, warns about stale kookr-* sessions. Noise after V7 has baked; V8 removes this.

Where to make changes

Task	File
Change how sessions are spawned	ClaudeCodeAdapter.launch / CodexCliAdapter.launch
Change per-session I/O	Same adapters' sendInput/sendKeystroke/stop/captureDisplay
Change backend selection / startup	src/server/start.ts:42, 108-167, 180
Change WS routing	src/server/index.ts:733-794
Change reconciliation	src/server/reconciliation.ts + src/server/index.ts:409, 419-421
Change breaker	src/server/index.ts:220, 224, 237 + src/adapters/circuit-breaker-terminal-manager.ts

References

• V7 RFC: docs/rfc/rfc-claude-code-terminal-parity.md
• V8 RFC: docs/rfc/rfc-v8-tmux-removal.md
• ADR-014 local-dtach-backend.
• ADR-007 managed-terminal-sessions (to be superseded).