Plugin testing

Plugin Testing Checklist

Verify plugin spec requirements with actionable test cases for the reflection and TTS plugins.

Plugin Specifications

Reflection Plugin (reflection-3.ts)

Purpose

Evaluates task completion when the agent goes idle. If the task is incomplete, sends feedback to continue work.

Spec Requirements

ID	Requirement	Description
R1	Uses RECENT human input	Extract the most recent human message as the task (not the first)
R2	Returns feedback only if INCOMPLETE	Only call promptAsync() when verdict.complete === false
R3	No feedback if COMPLETE	Complete tasks show toast only, no prompt (prevents infinite loop)
R4	No console.log	No logging to avoid breaking CLI output
R5	Stores in .reflection/	Save reflection data (task, result, tools, prompt, verdict, timestamp) to .reflection/ directory
R6	Skip judge sessions	Never reflect on judge sessions (contain "TASK VERIFICATION")
R7	Skip aborted sessions	Never reflect on sessions cancelled by user (Esc key)
R8	Attempt limiting	Max 3 reflection attempts per session before giving up
R9	Reset on new input	Reset attempt counter when user provides new input
R10	Concurrent protection	Prevent multiple simultaneous reflections on same session

Data Storage Format (.reflection/)

{
  "task": "string - the most recent human message",
  "result": "string - the assistant's response (truncated to 2000 chars)",
  "tools": "string - last 10 tool calls",
  "prompt": "string - the full judge prompt sent",
  "verdict": {
    "complete": "boolean",
    "feedback": "string"
  },
  "timestamp": "ISO 8601 timestamp"
}

---

TTS Plugin (tts.ts)

Purpose

Reads the agent's final response aloud when a session completes.

Spec Requirements

ID	Requirement	Description
T1	Default engine is Coqui	loadConfig() defaults to engine: "coqui"
T2	Stores in .tts/	Save TTS data (originalText, cleanedText, spokenText, engine, timestamp) to .tts/ directory
T3	Skip judge sessions	Never speak judge session responses
T4	Skip incomplete sessions	Only speak when session is complete
T5	Speech lock	Prevent multiple agents from speaking simultaneously
T6	Text cleaning	Remove code blocks, markdown, URLs before speaking
T7	Text truncation	Truncate to 1000 chars max
T8	Engine fallback	Fall back to OS TTS if configured engine fails
T9	Multiple engines	Support coqui, chatterbox, and os engines
T10	Server mode	Keep TTS model loaded for fast subsequent requests

Data Storage Format (.tts/)

{
  "originalText": "string - raw assistant response",
  "cleanedText": "string - after removing code/markdown",
  "spokenText": "string - final text sent to TTS (may be truncated)",
  "engine": "string - coqui|chatterbox|os",
  "timestamp": "ISO 8601 timestamp"
}

---

Testing Checklist

Pre-requisites

• Plugins deployed to ~/.config/opencode/plugin/
• OpenCode restarted after deployment
• TTS config exists at ~/.config/opencode/tts.json

Reflection Plugin Tests

R1: Uses RECENT human input

• Unit test exists: Check extractTaskAndResult() uses last human message
• Code review: Line 137 uses task = part.text (overwrites, not assigns once)

R2: Returns feedback only if INCOMPLETE

• Code review: Lines 288-304 only call promptAsync() when verdict.complete === false

R3: No feedback if COMPLETE

• Code review: Lines 282-286 only call showToast(), no promptAsync()

R4: No console.log

• Code search: grep -n "console.log\|log(" reflection-3.ts returns no matches

R5: Stores in .reflection/

• Code review: saveReflectionData() function exists (lines 35-49)
• Code review: reflectionDir = join(directory, ".reflection") (line 27)
• Code review: All required fields saved (task, result, tools, prompt, verdict, timestamp)
• E2E test: After running a task, .reflection/ directory contains JSON file

R6: Skip judge sessions

• Code review: isJudgeSession() function exists (lines 72-81)
• Code review: Judge sessions marked as processed (line 234)

R7: Skip aborted sessions

• Code review: wasSessionAborted() function exists (lines 83-109)
• Code review: abortedSessions Set tracks aborted sessions
• Code review: Fast path check at line 330

R8: Attempt limiting

• Code review: MAX_ATTEMPTS = 3 (line 12)
• Code review: Attempt check at lines 218-223

R9: Reset on new input

• Code review: Lines 206-212 reset attempts on new human message

R10: Concurrent protection

• Code review: activeReflections Set exists (line 23)
• Code review: Early return at lines 182-184

TTS Plugin Tests

T1: Default engine is Coqui

• Code review: loadConfig() returns engine: "coqui" (line 116)
• Unit test: npm test includes test for default engine

T2: Stores in .tts/

• Code review: saveTTSData() function exists (lines 1213-1226)
• Code review: ttsDir = join(directory, ".tts") (line 1205)
• Code review: All required fields saved
• E2E test: After TTS triggered, .tts/ directory contains JSON file

T3: Skip judge sessions

• Code review: isJudgeSession() check at line 1338

T4: Skip incomplete sessions

• Code review: isSessionComplete() check at line 1339

T5: Speech lock

• Code review: waitForSpeechLock() called at line 1263
• Code review: Lock released in finally block (line 1300)

T6: Text cleaning

• Code review: cleanTextForSpeech() function removes code, markdown, URLs (lines 1242-1252)

T7: Text truncation

• Code review: MAX_SPEECH_LENGTH = 1000 (line 34)
• Code review: Truncation logic at lines 1258-1260

T8: Engine fallback

• Code review: OS TTS fallback at line 1298

T9: Multiple engines

• Code review: speakWithCoqui(), speakWithChatterbox(), speakWithOS() all exist

T10: Server mode

• Code review: serverMode option in config (lines 68, 76)
• Code review: Server startup functions exist

---

Running Tests

Unit Tests

cd /Users/engineer/workspace/opencode-reflection-plugin
npm test

E2E Tests (CRITICAL - must always run)

cd /Users/engineer/workspace/opencode-reflection-plugin
OPENCODE_E2E=1 npm run test:e2e

Manual TTS Test

npm run test:tts:manual

Verify Deployment

# Check plugins are deployed
ls -la ~/.config/opencode/plugin/

# Verify they match source
diff reflection-3.ts ~/.config/opencode/plugin/reflection.ts
diff tts.ts ~/.config/opencode/plugin/tts.ts

# Check TTS config
cat ~/.config/opencode/tts.json

Verify Data Storage (after running a task)

# Check reflection data
ls -la .reflection/
cat .reflection/*.json | head -50

# Check TTS data
ls -la .tts/
cat .tts/*.json | head -50

---

Known Issues

1. Reflection may not trigger in test environments - If tasks complete very quickly before session.idle fires, reflection may not run. This is expected behavior, not a bug.

2. TTS Coqui server startup time - First TTS request with Coqui may take 30-60 seconds while model downloads and loads. Subsequent requests are fast due to server mode.