Plugin test

/plugin-test — NCC Plugin E2E Test Suite

Usage

/plugin-test                    # Interactive — suggest a test mode based on recent changes
/plugin-test --quick            # Phase 1 only: 4 diverse projects × 4 features each
/plugin-test --full             # Phase 1 + Phase 2: full suite including monorepo + cross-feature mods
/plugin-test --install-test     # Install test: 2 projects (npx + marketplace), 5 features each, non-spec-dev focus
/plugin-test --phase1           # Phase 1 only (same as --quick)
/plugin-test --phase2           # Phase 2 only: 10-feature monorepo (solo + team)
/plugin-test --security         # Security-focused: run security-reviewer on all generated code
/plugin-test --a11y             # Accessibility-focused: run browser-tester a11y checks on UI projects
/plugin-test --codequality      # Code quality audit: deep analysis of all generated source files
/plugin-test --token             # Token optimization audit: run token-optimizer on plugin source
/plugin-test healthcare,marketplace   # Run specific projects only
/plugin-test --create              # /create pipeline test: new + existing project ideation

Test Modes

Phase 1: Quick Validation (--quick, --phase1)

4 diverse projects, each with 4 features + extra agent verification.

Project	Stack	Extra Agents
healthcare	Prisma + next-auth + zod + vitest	/security, /review
design-system	framer-motion + playwright + storybook	browser-tester, /qa --a11y
realtime-collab	drizzle + tanstack-query + zustand + msw	performance-optimizer, /loop
marketplace	Prisma + stripe + ai-sdk + @vercel/blob	/security --diff, /review, /status

Per-project: NCC install → /init → 4× spec→dev (3 tasks each) → extra agent runs → TEST_LOG.md

Reviewer scores 15 categories: 1-10: Installation, Hooks, Skills, Spec, Design, PLAN, TDD, Token, Code, Flow 11: Rules size (<200 lines each) 12: Skill selection accuracy (manifest vs package.json) 13: Extra agent execution results 14: Document sync (CONTEXT.md/design.md consistency) 15: Code quality detail (error handling, Zod, composition, Server/Client)

Install Test (--install-test)

Tests the published version of NCC using two real installation methods. Primary focus: non-spec-dev features.

Project	Install Method	Stack	Features	Non-Spec-Dev Focus
npx-healthcare	npx nextjs-claude-code@latest	Prisma + next-auth + zod + vitest	5 features	/init, /security, /log, /commit, /rule, /test, hooks, skills
mkt-marketplace	/plugin marketplace add + /plugin install	Prisma + stripe + ai-sdk + @vercel/blob	5 features	/init, /review, /status, /debug, /qa, /loop, /commit, hooks, skills

Per-project: Install → /init → 5× spec→dev (2-3 tasks each) → non-spec-dev tests → TEST_LOG.md

Reviewer scores 20 categories (base 15 + 5 install-test categories): 16: Init Completeness (PROJECT.md, ARCHITECTURE.md, STATE.md quality) 17: Security & Log Setup (SECURITY_STRATEGY.md, LOG_STRATEGY.md, audit results) 18: Git Workflow (/commit conventional format + REQ linking) 19: Debug & Loop (DEBUG.md hypothesis-driven, LOOP_NOTES.md tracking, /status read-only) 20: Installation Method Fidelity (npx: local files + manifest; marketplace: namespace + CLAUDE_PLUGIN_ROOT)

Phase 2: Large-scale Monorepo (--phase2)

2 identical CRM projects (solo vs team mode), each with:

• 10 features in dependency chain
• 5 cross-feature modifications (api-layer error format, RBAC expansion, tagging, cursor pagination, custom stages)
• Document Sync verification (CONTEXT.md, design.md, history entries)
• Solo vs Team comparison

Features: api-layer → auth-system → contact-management → deal-pipeline → email-integration → activity-feed → reporting → team-workspace → notification-center → admin-settings

/create Pipeline Test (--create)

Tests the /create ideation pipeline on both new and existing projects.

Project	Scenario	Validation
create-new	Empty project → /init → /create "AI recipe app"	VISION.md, C-REVIEW.md, DECISION.md exist in spec/create/
create-existing	Healthcare project → /create "notification system"	Reads spec/PROJECT.md context, outputs to spec/create/

Per-project: NCC install → /init → /create → verify output docs → token isolation check → spec conversion test

Reviewer checks 5 categories:

1. Document Quality: VISION.md has Problem/Demand/Target/Value sections
2. C-Level Coverage: C-REVIEW.md has all 5 assessments (CEO/CTO/CPO/CMO/CDO) with verdicts
3. Decision Clarity: DECISION.md has chosen approach + key decisions
4. Token Isolation: After /create, run /spec and verify spec-writer does NOT read spec/create/
5. Spec Conversion: If converted, output is valid /spec command format

Full Suite (--full)

Runs Phase 1, analyzes issues, then runs Phase 2. Generates consolidated REPORT with cross-phase comparison.

Security Audit (--security)

For each generated project:

• Read installed security-reviewer agent
• Scan all API routes for OWASP Top 10: injection, auth bypass, sensitive data exposure, CSRF
• Check MSW handlers for auth mock bypass patterns
• Check Zod validation coverage
• Output: SECURITY_REVIEW.md per project

Accessibility Audit (--a11y)

For UI-heavy projects (design-system or any with UI components):

• Read installed browser-tester agent
• WCAG 2.1 AA checklist per component
• Check: ARIA attributes, keyboard nav, focus management, color contrast
• Check: prefers-reduced-motion respect, semantic HTML, role attributes
• Output: ACCESSIBILITY_REVIEW.md per project

Code Quality Audit (--codequality)

Deep source code analysis across all projects:

• Error handling grade (try/catch, error classification, no silent swallowing)
• Zod validation grade (boundary validation, schema completeness)
• TypeScript strictness (no any, no as unknown, proper inference)
• Composition patterns (Context vs prop drilling, compound components)
• Server/Client discipline ('use client' minimization)
• DRY compliance (no duplicate logic across routes)
• Named constants (no magic numbers/strings)
• Output: code quality table in REPORT

Review Checklist

Standard (15 categories, /15 scoring)

#	Category	What to check
1	Installation	agents 25+, scripts 6+, skills 5+, rules 3, spec/rules 8+, CLAUDE.md
2	Settings & Hooks	env keys, SessionStart(2), PreToolUse(2), PostToolUse(3), Stop(1)
3	Skill Selection	core skills vs package.json library matching
4	Spec Quality	frontmatter (mock, testing, deps), REQ-NNN:, all required sections
5	Design Quality	Components, State, Data Flow, Technical Decisions
6	PLAN.md Quality	domain tags, model, wave, MSW tasks, Approved-at, Budget, Team Composition (team mode)
7	TDD/Mock	MSW handlers for all endpoints, mock:true enforcement, test files
8	Token Efficiency	file sizes <200 lines, artifact limits, skill budget rules
9	Code Quality	try/catch, Zod, strict TS, no stubs, Server/Client
10	Flow Completeness	STATE.md, CONTEXT.md, [x] tasks, TEST_LOG.md
11	Rules Size	all spec/rules/ <200 lines, no redundancy
12	Skill Accuracy	manifest vs detected libs, on-demand coverage
13	Extra Agents	security/a11y/performance/review/loop results
14	Document Sync	CONTEXT.md consistent after modifications, no stale decisions
15	Code Quality Detail	error handling grade, composition, naming, DRY

Install Test Extended (20 categories, /20 scoring)

All 15 standard categories plus:

#	Category	What to check
16	Init Completeness	PROJECT.md framework detection, ARCHITECTURE.md structure, STATE.md features
17	Security & Log Setup	SECURITY_STRATEGY.md, LOG_STRATEGY.md, audit findings quality
18	Git Workflow	/commit conventional format, REQ linking, no git add -A
19	Debug & Loop	DEBUG.md hypotheses, LOOP_NOTES.md tracking, /status read-only
20	Install Method Fidelity	npx: local files + manifest + doctor; marketplace: namespace + plugin paths

Report Output

Test artifacts are written to __plugin-tests__/ during execution:

• Per-project: TEST_LOG.md, REVIEW.md
• Extra agents: SECURITY_REVIEW.md, ACCESSIBILITY_REVIEW.md, PERFORMANCE_REVIEW.md, REVIEW_RESULT.md

After each test run, generate a key-summary English report and save it to reports/REPORT_YYYYMMDD_HHMM.md (inside this skill directory). The report should contain:

• Date, branch, version, scope
• Score summary table
• Key findings (bullet points, not full prose)
• Critical issues with severity
• Recommendations
• For --install-test: installation method comparison table + non-spec-dev feature results

Keep each report under 80 lines. This is the archival summary — the raw artifacts in __plugin-tests__/ have full details.

Past Reports

Reports are stored in reports/ directory. By default, read only the latest report (highest datetime in filename) to understand current plugin quality baseline. Read older reports only when the user explicitly asks for trend analysis or historical comparison.

Report	Scope	Score
REPORT_20260321_1900.md	Main branch validation (2 projects)	9.5/10 avg
REPORT_20260321_1700.md	R5: Incremental fix-and-retest (3 projects)	8.7/10 avg
REPORT_20260321_1500.md	R4: Full suite Phase 1+2 (4+2 projects, 15 categories)	12.0/15 avg
REPORT_20260321_1300.md	R3: Monorepo solo vs team (10 features × 2 modes)	Solo 6/10, Team 5/10
REPORT_20260321_1100.md	R2: 4 projects × 4 features (post-fix)	9.0/10 avg
REPORT_20260321_0900.md	R1: 4 projects × 1 feature (baseline)	7.5/10 avg

NCC Source Path

For dev/Phase 1/Phase 2 modes: node {NCC_SOURCE}/dist/index.js (tests current branch build). For install-test mode: uses published npm version (npx nextjs-claude-code@latest) and marketplace.