Media memory

Why this exists, and what I'd change

Why it exists. Stateless chat forgets every screenshot, chart, recording, and doc. Asking "find me the wireframe from last month" becomes "scroll through old Slack." This skill embeds everything so it's actually searchable across sessions and projects.

Design tradeoffs.

• Local ChromaDB instead of a hosted vector DB. Everything stays in your workspace, no cloud dependency. Cost: setup friction (GEMINI_API_KEY, Python deps), and search is single-machine only.
• Description-first indexing instead of raw image embedding. Gemini describes the asset, then we embed the description. Cost: files over 18MB get a placeholder description; full transcription requires routing through the Gemini Files API manually.
• Manual ingest call by default. I didn't auto-trigger on every Claude session because that creates surprise database writes. Cost: people forget to ingest, and a month of screenshots stays unindexed.

What I'd change. Auto-ingest on file paste in Claude Code, with a quiet confirmation. Manual ingest is reliable but rarely happens in practice.

---

Media Memory

A single source of truth for every piece of media the user shares or the assistant generates. Each asset is described by a multimodal Gemini call, embedded with Gemini Embedding 2 (gemini-embedding-001), and stored in a local ChromaDB collection alongside a JSON metadata sidecar.

Commands

/media-memory ingest <path>          # log a single asset
/media-memory ingest <path> --source ai_generated --tags chart,roadmap
/media-memory search "query"         # semantic search
/media-memory search "query" --type image --from 2026-01-01 --tag finance
/media-memory status                 # counts and storage health

Prerequisites

pip install -r scripts/media-memory/requirements.txt
export GEMINI_API_KEY=...   # or GOOGLE_API_KEY

google-genai and chromadb are the only runtime deps. Vectors persist to media-memory/chroma/.

When to Auto-Trigger

Use this skill without being asked in any of these cases:

Trigger	Action
User attaches an image, audio, video, or document	ingest with --source user_uploaded
You generate an image, diagram, or media artifact	ingest with --source ai_generated and --source-detail "<one-line prompt summary>"
User mentions a past asset ("that chart", "the recording from last week")	search with relevant filters before answering
Question references visual or audio context	search first, then answer with citations

If the user explicitly says "don't log this" — skip ingestion and proceed.

---

Workflow A — Ingest

1. Confirm the file path. If the asset only exists in the chat, save it locally first (e.g. into media-memory/inbox/) and pass that path.
2. Pick the right --source:
   • user_uploaded (default for shared files)
   • ai_generated (anything you produced)
   • screenshot (UI screenshots)
   • web_url (downloaded from a URL — put the URL in --source-detail)
3. Run the ingest command. The script will:
   • Detect MIME and media type (image | audio | video | document | other)
   • Call gemini-2.5-flash to produce a natural language description, OCR/extracted text, and (for audio/video) a transcript
   • Auto-generate 4-8 semantic tags
   • Embed the combined description + extract with gemini-embedding-001
   • Persist:
     • The binary under media-memory/assets/YYYY/MM/<id><ext>
     • The full record under media-memory/metadata/<id>.json
     • The vector + flat metadata in media-memory/chroma/
     • An audit row in media-memory/index.jsonl
4. Confirm to the user with the new id, type, and tags.

Duplicates are detected by SHA-256 checksum and skipped automatically.

Example

python scripts/media-memory/media_memory.py ingest \
  ~/Downloads/product-roadmap-q2.png \
  --source ai_generated \
  --source-detail "Generated for Q2 roadmap review on 2026-04-25" \
  --tags product,roadmap,q2

---

Workflow B — Search

Use this before answering questions that touch on prior assets, designs, recordings, or screenshots.

python scripts/media-memory/media_memory.py search "the dashboard wireframe with the Money In Money Out widget" \
  --type image --from 2026-01-01 --tag product -n 5

Filters supported:

Flag	Meaning
--type	`image
--source	Exact match (e.g. user_uploaded, ai_generated)
--from	ISO date — only assets ingested on/after
--to	ISO date — only assets ingested on/before
--tag	Single tag — post-filtered against the asset's tag list
-n	Top-k results (default 5)

Returned JSON contains: id, filename, type, source, timestamp_ingested, tags, stored_path, truncated description, and similarity distance.

Citing search results

When you reference an asset back to the user, use this format:

Found in media memory: <filename> (<type>, ingested <date>) — <one-line description>. Path: <stored_path>.

If the user wants the actual file, point them at <stored_path> (it's repo-relative).

---

Metadata Schema (canonical record)

See media-memory/README.md. Fields you will most often reason over:

• type — coarse media class
• source — who created it
• description — natural language summary
• extracted_text / transcript — searchable text content
• tags — semantic indexing
• timestamp_ingested / timestamp_epoch — recency + range filters
• stored_path — where the binary lives

ChromaDB metadata stores all primitive fields; tag arrays are mirrored as tags_joined for filtering.

---

Quality Checklist (before confirming an ingest)

• File saved under media-memory/assets/YYYY/MM/
• Sidecar JSON written under media-memory/metadata/
• description is concrete (not "an image of something")
• extracted_text / transcript populated when content has text/audio
• At least 3 semantic tags generated
• Vector added to ChromaDB (script prints [ingested] id=...)
• If the file was ai-generated, source_detail summarizes the prompt or context

Edge Cases

Situation	Handling
File >18 MB	Description is skipped with a note; consider Files API upload
Unknown MIME	Stored as type=other; description still attempted
GEMINI_API_KEY not set	Script exits with a clear error; tell the user before retrying
Duplicate (same SHA-256)	Skipped; existing id returned
Empty Chroma collection on first search	Return empty list — never fabricate citations

---

Related

• Pairs with /research-scout — when scout finds relevant external assets, ingest them with --source web_url.
• Pairs with /morning-brief — screenshots from the brief can be archived for later recall.
• Pairs with /sketchnote, /frontend-design, /prototype — auto-ingest their outputs as ai_generated.