Pdf2md

PDF → Markdown OCR (operator runbook)

Audience: you, an AI agent operating this tool for a human user. Goal: OCR every PDF under a folder the user picks and write matching .md files into an output folder they pick, mirroring the directory structure.

This file is your runbook. Work through the sections in order. After every command, look at the result before continuing. If a step fails, fix it or tell the user exactly what's needed — never push ahead blindly onto their real files.

Throughout, $SKILL means the directory containing this SKILL.md. Set it once:

SKILL="/path/to/this/skill"   # the folder this SKILL.md lives in

If the skill directory is read-only in your environment, copy it to a writable working directory first and set $SKILL to the copy.

---

0. What's in this skill

Path	What it is
scripts/ocr_pdfs.py	The tool. Recursively OCRs PDFs → mirrored .md. Quality-first: orientation fix, deskew, adaptive binarization, confidence-driven escalation.
scripts/selftest.py	Self-contained readiness test — synthesizes degraded scans, OCRs them, scores against ground truth. No sample PDFs needed.
scripts/fetch_models.py	Downloads Tesseract's highest-accuracy models (~35 MB) into scripts/tessdata_best/ — not checked in; create it with this script (section 2d). The tool auto-uses it when present.
requirements.txt	Python dependencies.
README.md	Human-facing reference (flags, scaling notes).
.venv/	Python virtual environment. May already exist; if so, reuse it.

The tool only writes .md files (plus _errors.log / _review.log) under the output folder. It never modifies the input PDFs.

---

1. Your job, end to end

1. Check prerequisites — Tesseract + language data, Python venv, best-accuracy models.
2. Self-test — run selftest.py; it generates degraded sample scans itself.
3. Ask the user for the input PDF folder, the output folder, and the language(s) of the documents.
4. Validate & confirm scope — dry-run, report the file count, get the go-ahead.
5. Run the conversion.
6. Report the result summary and where the .md files landed.

---

2. Prerequisite checks

2a. Tesseract + language data

which tesseract && tesseract --list-langs

• Binary found and the needed language codes listed (e.g. eng, deu) → continue.

• Otherwise install it:

  # macOS (Homebrew; tesseract-lang carries all extra languages, few hundred MB)
  brew install tesseract tesseract-lang
  
  # Ubuntu/Debian (one package per language)
  sudo apt-get install tesseract-ocr tesseract-ocr-deu
  

  These are sizeable downloads — run them in the background and wait. If no package manager is available, tell the user what to install and pause. Then re-run the check above and confirm the needed languages are listed.

2b. Python virtual environment + dependencies

System Pythons are often "externally managed", so always install into a local .venv — never into system Python — and always invoke the tool with $SKILL/.venv/bin/python.

First, if .venv already exists, test it:

"$SKILL/.venv/bin/python" -c "import fitz, pytesseract, PIL, tqdm; print('deps ok')"

• Prints deps ok → skip the install, go to section 2c.

• Missing, or the import errors → (re)create and install:

  python3 -m venv "$SKILL/.venv"
  "$SKILL/.venv/bin/python" -m pip install --upgrade pip
  "$SKILL/.venv/bin/pip" install -r "$SKILL/requirements.txt"
  

  Then re-run the deps ok check.

2c. Best-accuracy OCR models (recommended)

Quality is the priority, so fetch Tesseract's highest-accuracy models once (~35 MB, into scripts/tessdata_best/; the tool picks them up automatically):

"$SKILL/.venv/bin/python" "$SKILL/scripts/fetch_models.py" deu eng osd

Pass the language codes the user's documents need (plus osd, used for orientation detection). If the machine is offline, continue without them — the tool falls back to the system models and still passes the self-test; just tell the user that difficult scans read slightly better once the models are fetched.

2d. German language data (needed by the self-test)

The self-test fixtures are German on purpose — umlauts and ß make a demanding test case. If deu was not in the --list-langs output, install it (section 2a) even when the user's documents are in another language.

---

3. Readiness self-test (do not skip)

Prove the whole pipeline works before going near the user's data. The self-test needs no sample PDFs — it generates pages degraded like real bad scans (noise, low contrast, skew, uneven lighting), OCRs them, and scores the output against the known ground truth:

cd "$SKILL" && .venv/bin/python scripts/selftest.py

Expected: a per-fixture similarity table where every row says ok, then SELFTEST: PASS (takes ~30–60 s; it cleans up after itself).

The self-test must pass with or without tessdata_best/. If it prints FAIL, stop and debug (see section 7) before involving the user's files — a failing shadow fixture usually means the numpy/Pillow preprocessing is broken (reinstall, section 2b).

---

4. Ask the user

Ask directly in chat and wait for the answers:

1. "Which folder contains the PDFs? I'll search it and all its subfolders."
2. "Where should I save the Markdown files? I'll mirror the folder structure there."
3. "What language(s) are the documents in?" — map the answer to Tesseract codes (deu, eng, fra, …); mixed batches join with +, e.g. deu+eng.

Notes:

• Accept absolute paths, ~, and relative paths — expand ~ before using them.
• The two folders should be different. If the user gives an output folder that sits inside the input folder, that's fine (the tool skips its own output), but prefer a separate location to keep things tidy.
• If they have no preference for output, suggest a folder named mdfiles.
• If the language they name isn't installed yet, go back to section 2a for it.

---

5. Validate, confirm scope, then run

Substitute the answers for <INPUT>, <OUTPUT>, <LANG> below (keep the quotes — paths may contain spaces).

Dry run first — this lists how many PDFs would be processed and the src → dst mapping, without doing any OCR. It also errors clearly if <INPUT> isn't a real folder:

"$SKILL/.venv/bin/python" "$SKILL/scripts/ocr_pdfs.py" "<INPUT>" --out "<OUTPUT>" --lang <LANG> --dry-run

Tell the user the count ("Found N PDFs"). OCR is CPU-bound and runs across all cores; as a rough guide, dense 300-DPI scans take ~1–2 s per page per core. If N is large (say > 500) or it'll clearly run long, confirm with the user before the real run.

Real run:

"$SKILL/.venv/bin/python" "$SKILL/scripts/ocr_pdfs.py" "<INPUT>" --out "<OUTPUT>" --lang <LANG>

• Resumable: if it's interrupted (Ctrl-C, crash, machine sleep), just run the exact same command again — files already converted are skipped.
• Re-OCR everything (e.g. after tuning settings): add --overwrite.
• Quality is the default (orientation fix, deskew, escalation on low-confidence pages, best models). Only add --fast if the user explicitly prefers speed over accuracy.

While it runs, the tool prints a live progress bar with throughput, ETA, and running ok / skip / fail / ocr_pages / conf / review counts. Time estimate for the user: clean scans ≈ one OCR pass (~2–4 s/page/core); genuinely poor pages escalate and can take several times that — that's intentional.

---

6. Report back

When it finishes, summarize for the user:

• the final counts — ok / skipped / failed,
• pages OCR'd vs. taken from an existing text layer,
• the mean OCR confidence,
• the output folder path where the .md files now live,
• if any pages stayed below the confidence floor, point them to <OUTPUT>/_review.log — it lists every uncertain page (worst first) so they can spot-check those instead of proofreading everything,
• if failed > 0, point them to <OUTPUT>/_errors.log (one path <TAB> reason per line) and mention the top couple of failures.

---

7. Troubleshooting

Symptom	Fix
tesseract not found	Section 2a — install Tesseract with the system package manager.
language data missing: <lang>	Install the language pack (brew install tesseract-lang / apt-get install tesseract-ocr-<lang>), then re-verify with tesseract --list-langs.
externally-managed-environment / pip refuses to install	You're using system Python. Use the venv: $SKILL/.venv/bin/pip … (section 2b).
ModuleNotFoundError: numpy	Dependencies out of date: "$SKILL/.venv/bin/pip" install -r "$SKILL/requirements.txt".
missing in --tessdata dir	tessdata_best/ is incomplete for the requested --lang. Fetch what's missing, e.g. python scripts/fetch_models.py fra osd, or pass --tessdata "" to fall back to system models.
Many pages in _review.log	Expected for genuinely bad scans — that's the triage list. If a whole batch is low-confidence, spot-check whether the documents match the --lang given, are handwritten (OCR won't fix that), or are photos.
Still-poor OCR on specific scans	The quality ladder already tries binarization/PSMs/450 DPI. Last resorts: --dpi 400 --dpi-hi 600, or --psm 4 (single column) / --psm 6 (uniform block) if all documents share that layout. If the text is handwritten, OCR won't recover it — tell the user honestly.
Many empty results	Those PDFs are image-only with little OCR output, or genuinely blank. Spot-check a few pages.
Run is slow	That's the quality-first design working (escalation on hard pages). If the user prefers speed: --fast, and/or --dpi-hi 0.
Basic shell utilities (mkdir, cp, …) "command not found"	Some sandboxed shells expose only a subset of binaries. Do file operations through Python instead, e.g. .venv/bin/python -c "import shutil, os; …".

---

8. Flag quick-reference

Flag	Default	Meaning
--out	mdfiles	Output root for the mirrored .md tree.
--lang	eng	Tesseract language code(s), e.g. deu or deu+eng.
--dpi	300	Base render resolution for OCR.
--dpi-hi	450	Re-render resolution for low-confidence pages (0 = off).
--fast	off	Single-pass mode: skip orientation/deskew/escalation.
--conf-accept	90	Mean word confidence at which a page is accepted immediately.
--conf-floor	75	Pages below this land in _review.log.
--tessdata	auto	Model dir; auto = use local tessdata_best/ when present.
--workers	#CPU cores	Parallel worker processes.
--min-chars	20	Embedded chars/page above which a page's text layer is used instead of OCR.
--force-ocr	off	OCR every page, ignore any embedded text.
--overwrite	off	Re-process files whose .md already exists (default is resume/skip).
--limit N	0	Process at most N PDFs (testing).
--dry-run	off	List what would run; do no OCR.

Run "$SKILL/.venv/bin/python" "$SKILL/scripts/ocr_pdfs.py" --help for the full list.

---

9. Guardrails

• The tool writes only under <OUTPUT>. Don't delete or overwrite anything outside it, and never touch the input PDFs.
• If the input path is missing or ambiguous, ask the user — don't guess.
• Don't kick off a large run without telling the user the file count first.
• Keep scripts/selftest.py and scripts/fetch_models.py in place — they are the readiness check and the model installer. (tessdata_best/ itself is disposable: re-create it any time with fetch_models.py.)
• Don't switch to --fast to "speed things up" on your own initiative; quality is the stated priority. Suggest it to the user only if they raise runtime concerns.