App verify

App Verify

Verify a running or deployed web app by operating it, not by reading its source. The highest-value bugs are the ones that pass the typechecker and return HTTP 200 but are still wrong: a form that saves nothing, a report built on a column no code writes, a stat card with a hardcoded number. Source review and unit tests routinely miss these. Driving the real app in a browser and checking the datastore underneath is how you find them.

Two modes, often run together:

• Sweep — go through every route/flow as a user, find what's broken or fake.
• Deploy-verify — confirm a build/commit/deploy actually shipped and the live app is healthy.

Stack-agnostic. The method is the same for any framework, language, and datastore. Concrete commands below use one stack (Next.js + Supabase/Postgres) as a worked example — translate each to yours: the route list comes from your router, the "database" is your authoritative store (SQL, NoSQL, an API), and the "direct read" is whatever client reaches it (psql, an ORM console, a Mongo shell, the provider's dashboard or API). Where you see a Postgres/Supabase term, read the principle, not the brand.

Security & credentials

This skill stores no secrets — it's instructions, nothing else. Two steps touch credentials, and both stay inside the operator's own machine:

• Session cookie — the route sweep reuses the already-logged-in browser session by reading its cookie (document.cookie) to make authenticated curl requests to the operator's own app. The cookie is used locally and never written to a file or sent anywhere external.
• Direct DB credential — rung 4 reads the operator's own datastore for ground truth using a credential that already lives in the project's environment (a connection string, a service/admin key, an API token — e.g. in .env). It's read from there at runtime, used only against the operator's own datastore, and never transmitted or embedded in the skill.

The skill never enters passwords into login fields — that's off-limits even when the user asks; the user authenticates. It also keeps prod and scratch/test databases straight and confirms before deletes (see Safety). Nothing in these files is a credential, so the skill is safe to read, share, and publish.

The ladder (cheapest checks first)

Run in this order. Each rung is cheaper than the next and rules out whole failure classes.

1. Route health — hit every route, record HTTP status + error-boundary markers. Catches hard crashes. Cheap; do it first. See scripts/route_sweep.sh.
2. Render + console — load the pages a browser can render; read console errors and the actual text. Catches client error boundaries and blank/broken renders that still return 200.
3. Write flows — drive the real create/submit/approve flows end to end. A network 200 is NOT success — verify the row exists and side effects fired.
4. Data-layer truth — when the UI and the datastore disagree, read the datastore directly to find the real cause: access-control rules, a swallowed query error, caching, or a dead field.
5. Deploy-verify — build gate → commit → push → deploy → health-check the live server.

Stop climbing once you've answered the question. A quick "is prod up?" needs rungs 1 + 5, not the whole ladder.

Setup

Know your target environment first. Before anything that writes, be sure which environment and datastore the app and your checks point at — dev, staging, or production. Read-only checks (route health, render, reading the store) are safe anywhere. Write flows create real rows, so run them against dev/staging by default; only test writes on production with explicit user confirmation, and then use data the user actually needs rather than throwaway junk. Check the running app's own config (its .env / environment) to see which database it's talking to — don't assume localhost means a scratch DB, and don't assume the deployed URL means prod-only.

Get the app running. Don't guess the start command. Look for the project's own config: a launch/preview config, package.json scripts (dev/start), a Makefile, Procfile, docker-compose.yml, or the framework's dev command (rails s, manage.py runserver, php artisan serve). Use the project's browser-preview tool if one exists. If a dev server is already running on its port, reuse it. NOTE: running a production build against the same working dir as a running dev server can corrupt the dev server's cache and wedge it on a loading screen (seen with next build vs next dev, but the hazard is general) — build in isolation or restart the dev server after.

Authenticate. Reuse an existing/persisted session if the browser already has one. If a login is required, fill the email but do not type the password yourself — entering passwords into login fields is off-limits even when asked; have the user log in, or drive a browser that already holds the session. Once authenticated, you can copy the session cookie out of the browser (document.cookie) to make authenticated curl requests for the fast route sweep.

Driving the browser (the executor for rungs 2-3). This skill is the playbook — what to check and why. It doesn't reimplement browser automation. Use whatever concrete driver is at hand, in this order of preference:

• An interactive browser pane / preview tool if the session has one (Claude Browser pane, the /browse skill) — best for click-through you watch live.
• The webapp-testing skill (Anthropic's Playwright toolkit — install from anthropics/skills if absent) for headless/non-interactive runs and repeatable scripts. Its scripts/with_server.py manages the dev-server lifecycle and its reconnaissance-then-action pattern (navigate → wait for networkidle → inspect DOM → act on discovered selectors) is exactly what rungs 2-3 need. Let it handle the Playwright mechanics; this skill decides what to look for and how to confirm the data underneath is right.
• Plain authenticated curl for rung 1 (route health) — no browser needed; see scripts/route_sweep.sh.

Match the driver to the rung: curl for HTTP-level route health, a real browser (pane or webapp-testing/Playwright) for render + write flows, and always the datastore directly (a direct/admin connection) for rung 4 truth — no browser tool can tell you a green-looking write actually persisted correctly.

Route sweep (rung 1-2)

Enumerate every route from whatever your stack exposes: a route/router config, a file-routed pages directory (Next.js find src/app -name page.tsx, SvelteKit, Remix), a framework route dump (rails routes, Django URLconf, php artisan route:list), a sitemap.xml, or by crawling links from the homepage. Then:

• Fast pass: authenticated curl each route, record non-200s and scan the response for error-boundary markers ("Something went wrong", "Application error", "TypeError", a stack trace; framework-specific ones too, e.g. Next.js digest, Rails "We're sorry", a Django traceback page). scripts/route_sweep.sh does this given a base URL, a cookie, and a route list. Sequential, not parallel — hammering an on-demand/dev compiler with parallel requests makes it thrash and time out.
• Render pass: for the data-heavy / chart / dashboard pages, load them in the browser (pane, or a Playwright script via the webapp-testing skill) and read console errors + page text. Wait for the page to settle (networkidle) before inspecting — a dynamic app inspected too early looks blank when it's just still fetching. A page can 200 at the HTTP layer and still throw a client-side error or render blank.

Distinguish three things that all look like "nothing there":

• Broken — an error, a stack trace, a blank where content should be. A bug.
• Empty state — real "no data yet" because the datastore genuinely has none. Not a bug (but confirm it's empty, don't assume).
• Fake/stale — a number that shows regardless of the data (a hardcoded "3", a delta that never moves). A bug, and a nasty one because it looks real. See references/bug-classes.md.

Write flows (rung 3)

For each create/submit/approve flow: open it, fill every field, submit, then verify it actually worked — don't trust the button.

• Check the network call, but know that HTTP 200 ≠ success. Server actions and RPCs commonly return { ok: false, error } as a 200 body, or a transport-200 masking an app-level failure. Read the response body and the resulting UI.
• Confirm the record exists and side effects fired — the write usually triggers more than one thing (a new order decrements stock and queues a confirmation email; an approved request creates a follow-up record and notifies someone). Check downstream, not just the form.
• Stale-after-write false alarms: a create can succeed server-side but the on-screen list hasn't re-rendered yet. Before logging "it didn't save", wait a beat and/or reload. If it appears on reload, it saved — that's a UI-refresh nit, not data loss. If it's still gone after a full reload, it's a real data-layer bug.
• Multi-section wizards: try submitting with a later section incomplete — a good form surfaces the error; a bad one silently blocks or double-submits.

Data-layer truth (rung 4)

When the UI shows something the datastore contradicts (or vice versa), stop guessing and read the datastore directly, bypassing the app. Use whatever direct-read path your stack has: a SQL client (psql, mysql), an ORM console (rails console, django shell, Prisma Studio), a Mongo/Redis shell, or the provider's dashboard/API (a Supabase/Firebase REST endpoint, an admin API). Reading the raw store — with admin/service access that bypasses per-user rules — is how you separate these culprits:

• Access control — the row exists but the logged-in user's role can't read it. Confirm by reading once with the user's own identity/token and once with admin/direct access: visible to admin, invisible to the user → an authorization rule (row-level security, a scoped query, a permissions filter), not a missing row.
• Swallowed query error — a wrong or ambiguous query (a bad join/include, an ambiguous relationship, a malformed filter) errors at the driver, and the code ignores the error (checks only the result, never the error path), so a real row surfaces as "nothing". Reproduce the exact query the app runs against the store directly; a driver error there confirms it. (Example: in PostgREST, an ambiguous embed returns error PGRST201.)
• Wrong identity / connection — a call that needs the caller's identity runs under the wrong context (an admin/service connection with no user, a background job, a pooled connection), so an identity check sees "no user" and its auth gate fails for everyone, or it returns the wrong tenant's data. (Example: a Postgres SECURITY DEFINER function reading auth.uid() called through a service-role client — auth.uid() is null.)
• Dead field — a stored column/field the UI reads but no app code ever writes (only a seed or migration set it). Grep the field name across the source: if the only writer is seed/migration code, the value is frozen at seed time forever. Fix by computing it live or keeping it in sync.
• Caching — a caching layer (framework fetch/render cache, CDN, query cache) serving a stale read after a mutation. Tell: a hard reload / cache-bypass shows the correct value.

The full catalog with real reproductions is in references/bug-classes.md — read it before a deep sweep so you know what to look for.

360° element trace

A finding is worth more with its cause and blast radius than as a bare symptom. For a field, element, or tab that's wrong, suspicious, or release-critical, trace it in four directions — this is how "the box shows 0" becomes "this field is a dead column no code writes, and every report that reads it is wrong":

• Upstream — what feeds it. The loader/query/API/computed value/column that produces this value. Where does the number actually come from?
• Transform — what happens on the way. Filtering, aggregation, status filters, rounding, formatting. A wrong total is often a wrong filter, not wrong data.
• Downstream — what it feeds. Does this value flow into other tabs, totals, reports, exports, or a later write? Editing it — where does that land? This is the blast radius: if it's wrong here, what else is wrong.
• Dependencies — what it needs to work. Columns/tables, a related record, a role/permission, a feature flag, an env var, an external service, another tab's data.

Then an empty or wrong value resolves to a cause, not a shrug: real no-data · broken query · missing permission · dead source (nothing writes it) · missing dependency. Same four-direction trace works forward too — pick a critical input (a price, a role, a config flag) and follow it to every surface it should change, to catch the ones it silently doesn't.

Do this for the release-critical surfaces (permissions, anything a transaction or a decision rides on) as a matter of course; for everything else, reach for it when a value looks off.

Deploy-verify (rung 5)

When shipping or confirming a deploy:

1. Build gate. Run the project's real production build command locally (e.g. npm run build, mvn package, go build, cargo build --release), not just the typechecker, before committing. A full build catches errors a typecheck misses — bundling, server/client boundaries, template/asset compilation. Honor whatever correctness gate the project states.
2. Commit only your changes. Check git status — don't sweep in unrelated in-flight edits from other sessions or a runtime lock file. Stage your files explicitly. If you must include someone else's coherent uncommitted change, commit it separately so authorship stays honest.
3. Push, then run the project's deploy mechanism (a deploy.sh, a CI trigger, an SSH-and-pull). Read the deploy script first so you know what it does. A set -euo pipefail deploy that fails the build won't restart the service — the old version keeps serving, which is a safe failure.
4. Health-check the live server, not just localhost: HTTP status of a known route (login page → 200), the service is active, the deployed commit matches what you pushed, and the logs are clean since the restart (ignore errors dated before the deploy).

Findings report

Report so the reader can act without re-reading the whole session. Per finding: severity · one-line defect · file:line root cause · how it fails (concrete inputs → wrong output) · verification evidence (what you saw live). Lead with the highest-severity data-correctness bugs; group the trivial/OK stuff. Mark false positives explicitly when you retract one — a walked-back finding is as useful as a confirmed one.

Safety

• Never type passwords into login fields, even when asked. Have the user authenticate.
• Test data is not free. Testing write flows creates real rows and can fire real side effects — on production a submit might send an email, charge a card, ship an order, or notify a customer. On a throwaway/scratch DB that's fine; on production it dirties live data and can reach real people. Know which environment you're pointed at before you submit anything. Prefer verifying with data the user actually needs over throwaway junk on prod.
• Deletes are destructive. Before clearing test data, list exactly what you'll remove, delete children before parents (respect foreign keys), and delete only rows you created (match by unique markers + date). Leave pre-existing data alone. Prefer the app's own archive/soft-delete where it exists.
• Confirm before outward-facing or irreversible actions — deploys, deletes, anything that touches production.

Pre-release gate

For a production release, functional + data correctness is necessary but not the whole picture. references/pre-release-checklist.md is the full surface: the rows app-verify owns (routes/flows, data correctness, empty/error states, authorization at the endpoint not just the hidden button, form validation, numeric/total correctness, no-test-data-in-prod) and the rows that belong to a specialist tool (security → cso, accessibility → the Accessibility Auditor, performance, responsive, observability) — run those and record the result rather than half-doing them here. Scale depth to the release: a hotfix needs the changed surface + smoke + deploy; a first go-live wants the whole list. "I didn't check X" is a finding, not silence.

When NOT to use this

• If the question is purely "does the code compile / pass lint / pass unit tests", that's the code-quality health check or the test runner, not this.
• If the app isn't runnable yet (mid-refactor, no server), there's nothing to drive — review source instead.
• For a diff-scoped bug review before merge, a code review fits better. This skill is for the running app.