Pdf

PDF Processing Guide

Overview

This guide covers essential PDF processing operations using Python libraries and command-line tools. For advanced features, JavaScript libraries, and detailed examples, see REFERENCE.md. If you need to fill out a PDF form, read FORMS.md and follow its instructions.

Quick Start

from pypdf import PdfReader, PdfWriter

# Read a PDF
reader = PdfReader("document.pdf")
print(f"Pages: {len(reader.pages)}")

# Extract text
text = ""
for page in reader.pages:
    text += page.extract_text()

Python Libraries

pypdf - Basic Operations

Merge PDFs

from pypdf import PdfWriter, PdfReader

writer = PdfWriter()
for pdf_file in ["doc1.pdf", "doc2.pdf", "doc3.pdf"]:
    reader = PdfReader(pdf_file)
    for page in reader.pages:
        writer.add_page(page)

with open("merged.pdf", "wb") as output:
    writer.write(output)

Split PDF

reader = PdfReader("input.pdf")
for i, page in enumerate(reader.pages):
    writer = PdfWriter()
    writer.add_page(page)
    with open(f"page_{i+1}.pdf", "wb") as output:
        writer.write(output)

Extract Metadata

reader = PdfReader("document.pdf")
meta = reader.metadata
print(f"Title: {meta.title}")
print(f"Author: {meta.author}")
print(f"Subject: {meta.subject}")
print(f"Creator: {meta.creator}")

Rotate Pages

reader = PdfReader("input.pdf")
writer = PdfWriter()

page = reader.pages[0]
page.rotate(90)  # Rotate 90 degrees clockwise
writer.add_page(page)

with open("rotated.pdf", "wb") as output:
    writer.write(output)

pdfplumber - Text and Table Extraction

Extract Text with Layout

import pdfplumber

with pdfplumber.open("document.pdf") as pdf:
    for page in pdf.pages:
        text = page.extract_text()
        print(text)

Extract Tables

with pdfplumber.open("document.pdf") as pdf:
    for i, page in enumerate(pdf.pages):
        tables = page.extract_tables()
        for j, table in enumerate(tables):
            print(f"Table {j+1} on page {i+1}:")
            for row in table:
                print(row)

Advanced Table Extraction

import pandas as pd

with pdfplumber.open("document.pdf") as pdf:
    all_tables = []
    for page in pdf.pages:
        tables = page.extract_tables()
        for table in tables:
            if table:  # Check if table is not empty
                df = pd.DataFrame(table[1:], columns=table[0])
                all_tables.append(df)

# Combine all tables
if all_tables:
    combined_df = pd.concat(all_tables, ignore_index=True)
    combined_df.to_excel("extracted_tables.xlsx", index=False)

reportlab - Create PDFs

Basic PDF Creation

from reportlab.lib.pagesizes import letter
from reportlab.pdfgen import canvas

c = canvas.Canvas("hello.pdf", pagesize=letter)
width, height = letter

# Add text
c.drawString(100, height - 100, "Hello World!")
c.drawString(100, height - 120, "This is a PDF created with reportlab")

# Add a line
c.line(100, height - 140, 400, height - 140)

# Save
c.save()

Create PDF with Multiple Pages

from reportlab.lib.pagesizes import letter
from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, PageBreak
from reportlab.lib.styles import getSampleStyleSheet

doc = SimpleDocTemplate("report.pdf", pagesize=letter)
styles = getSampleStyleSheet()
story = []

# Add content
title = Paragraph("Report Title", styles['Title'])
story.append(title)
story.append(Spacer(1, 12))

body = Paragraph("This is the body of the report. " * 20, styles['Normal'])
story.append(body)
story.append(PageBreak())

# Page 2
story.append(Paragraph("Page 2", styles['Heading1']))
story.append(Paragraph("Content for page 2", styles['Normal']))

# Build PDF
doc.build(story)

Subscripts and Superscripts

IMPORTANT: Never use Unicode subscript/superscript characters (₀₁₂₃₄₅₆₇₈₉, ⁰¹²³⁴⁵⁶⁷⁸⁹) in ReportLab PDFs. The built-in fonts do not include these glyphs, causing them to render as solid black boxes.

Instead, use ReportLab's XML markup tags in Paragraph objects:

from reportlab.platypus import Paragraph
from reportlab.lib.styles import getSampleStyleSheet

styles = getSampleStyleSheet()

# Subscripts: use <sub> tag
chemical = Paragraph("H<sub>2</sub>O", styles['Normal'])

# Superscripts: use <super> tag
squared = Paragraph("x<super>2</super> + y<super>2</super>", styles['Normal'])

For canvas-drawn text (not Paragraph objects), manually adjust font the size and position rather than using Unicode subscripts/superscripts.

Command-Line Tools

pdftotext (poppler-utils)

# Extract text
pdftotext input.pdf output.txt

# Extract text preserving layout
pdftotext -layout input.pdf output.txt

# Extract specific pages
pdftotext -f 1 -l 5 input.pdf output.txt  # Pages 1-5

qpdf

# Merge PDFs
qpdf --empty --pages file1.pdf file2.pdf -- merged.pdf

# Split pages
qpdf input.pdf --pages . 1-5 -- pages1-5.pdf
qpdf input.pdf --pages . 6-10 -- pages6-10.pdf

# Rotate pages
qpdf input.pdf output.pdf --rotate=+90:1  # Rotate page 1 by 90 degrees

# Remove password
qpdf --password=mypassword --decrypt encrypted.pdf decrypted.pdf

pdftk (if available)

# Merge
pdftk file1.pdf file2.pdf cat output merged.pdf

# Split
pdftk input.pdf burst

# Rotate
pdftk input.pdf rotate 1east output rotated.pdf

Common Tasks

Extract Text from Scanned PDFs

# Requires: pip install pytesseract pdf2image
import pytesseract
from pdf2image import convert_from_path

# Convert PDF to images
images = convert_from_path('scanned.pdf')

# OCR each page
text = ""
for i, image in enumerate(images):
    text += f"Page {i+1}:\n"
    text += pytesseract.image_to_string(image)
    text += "\n\n"

print(text)

Add Watermark

from pypdf import PdfReader, PdfWriter

# Create watermark (or load existing)
watermark = PdfReader("watermark.pdf").pages[0]

# Apply to all pages
reader = PdfReader("document.pdf")
writer = PdfWriter()

for page in reader.pages:
    page.merge_page(watermark)
    writer.add_page(page)

with open("watermarked.pdf", "wb") as output:
    writer.write(output)

Extract Images

# Using pdfimages (poppler-utils)
pdfimages -j input.pdf output_prefix

# This extracts all images as output_prefix-000.jpg, output_prefix-001.jpg, etc.

Password Protection

from pypdf import PdfReader, PdfWriter

reader = PdfReader("input.pdf")
writer = PdfWriter()

for page in reader.pages:
    writer.add_page(page)

# Add password
writer.encrypt("userpassword", "ownerpassword")

with open("encrypted.pdf", "wb") as output:
    writer.write(output)

Quick Reference

Task	Best Tool	Command/Code
Merge PDFs	pypdf	writer.add_page(page)
Split PDFs	pypdf	One page per file
Extract text	pdfplumber	page.extract_text()
Extract tables	pdfplumber	page.extract_tables()
Create PDFs	reportlab	Canvas or Platypus
Command line merge	qpdf	qpdf --empty --pages ...
OCR scanned PDFs	pytesseract	Convert to image first
Fill PDF forms	pdf-lib or pypdf (see FORMS.md)	See FORMS.md

Next Steps

• For advanced pypdfium2 usage, see REFERENCE.md
• For JavaScript libraries (pdf-lib), see REFERENCE.md
• If you need to fill out a PDF form, follow the instructions in FORMS.md
• For troubleshooting guides, see REFERENCE.md

---

Markdown → PDF (LRN-extracted, Lightpanda renderer)

Browser Setup (Lightpanda)

LP_PORT=9222
lightpanda --remote-debugging-port $LP_PORT &
LP_PID=$!; sleep 1
trap "kill $LP_PID 2>/dev/null" EXIT

lp_navigate()    { local URL="$1"; node -e "const{chromium}=require('playwright');(async()=>{const b=await chromium.connectOverCDP('http://localhost:$LP_PORT');const p=await b.newPage();await p.goto('$URL');await b.close();})();" 2>/dev/null; }
lp_screenshot()  { local OUT="${1:-/tmp/lp.png}" W="${2:-1440}" H="${3:-900}"; node -e "const{chromium}=require('playwright');(async()=>{const b=await chromium.connectOverCDP('http://localhost:$LP_PORT');const p=await b.newPage();await p.setViewportSize({width:$W,height:$H});await p.screenshot({path:'$OUT',fullPage:true});await b.close();})();" 2>/dev/null; echo "$OUT"; }
lp_click()       { local SEL="$1"; node -e "const{chromium}=require('playwright');(async()=>{const b=await chromium.connectOverCDP('http://localhost:$LP_PORT');const p=await b.newPage();await p.click('$SEL');await b.close();})();" 2>/dev/null; }
lp_get_text()    { node -e "const{chromium}=require('playwright');(async()=>{const b=await chromium.connectOverCDP('http://localhost:$LP_PORT');const p=await b.newPage();console.log(await p.innerText('body'));await b.close();})();" 2>/dev/null; }
lp_get_html()    { node -e "const{chromium}=require('playwright');(async()=>{const b=await chromium.connectOverCDP('http://localhost:$LP_PORT');const p=await b.newPage();console.log(await p.content());await b.close();})();" 2>/dev/null; }
lp_eval()        { local JS="$1"; node -e "const{chromium}=require('playwright');(async()=>{const b=await chromium.connectOverCDP('http://localhost:$LP_PORT');const p=await b.newPage();console.log(await p.evaluate($JS));await b.close();})();" 2>/dev/null; }
lp_wait()        { sleep "${1:-1}"; }
lp_ai_vision()   { local FILE="${1:-/tmp/lp.png}"; lp_screenshot "$FILE"; echo "Screenshot saved: $FILE — pass to Claude vision for analysis."; }
lp_skill_run()   { echo "lp_skill_run: $* — map to agent-browser skill if available"; }
lp_browse()      { echo "lp_browse: $* — use lp_navigate/lp_screenshot/lp_click as appropriate"; }
lp_type()        { local SEL="$1" TXT="$2"; node -e "const{chromium}=require('playwright');(async()=>{const b=await chromium.connectOverCDP('http://localhost:$LP_PORT');const p=await b.newPage();await p.fill('$SEL','$TXT');await b.close();})();" 2>/dev/null; }

make-pdf: publication-quality PDFs from markdown

Turn .md files into PDFs that look like Faber & Faber essays: 1in margins, left-aligned body, Helvetica throughout, curly quotes and em dashes, optional cover page and clickable TOC, diagonal DRAFT watermark when you need it. Copy-paste from the PDF produces clean words, never "S a i l i n g".

On Linux, install fonts-liberation for correct rendering — Helvetica and Arial aren't present by default, and Liberation Sans is the standard metric-compatible fallback. CI and Docker builds install it automatically via Dockerfile.ci.

Emoji need a color-emoji font. macOS (Apple Color Emoji) and Windows (Segoe UI Emoji) ship one; most Linux distros and containers ship none, so emoji render as empty boxes (▯). ./setup auto-installs fonts-noto-color-emoji on Linux (apt/dnf/pacman/apk, best-effort) and the print CSS falls back through Apple / Segoe / Noto emoji families. Set GSTACK_SKIP_FONTS=1 to skip the install (CI without sudo, managed or offline machines).

Core patterns

80% case — memo/letter

One command, no flags. Gets a clean PDF with running header + page numbers

• CONFIDENTIAL footer by default.

$P generate letter.md                 # writes /tmp/letter.pdf
$P generate letter.md letter.pdf      # explicit output path

Publication mode — cover + TOC + chapter breaks

$P generate --cover --toc --author "Garry Tan" --title "On Horizons" \
  essay.md essay.pdf

Each top-level H1 in the markdown starts a new page. Disable with --no-chapter-breaks for memos that happen to have multiple H1s.

Draft-stage watermark

$P generate --watermark DRAFT memo.md draft.pdf

Diagonal 10% opacity DRAFT across every page. When the draft is final, drop the flag and regenerate.

Fast iteration via preview

$P preview essay.md

Renders HTML with the same print CSS and opens it in your browser. Refresh as you edit the markdown. Skip the PDF round trip until you're ready.

Brand-free (no CONFIDENTIAL footer)

$P generate --no-confidential memo.md memo.pdf

Common flags

Page layout:
  --margins <dim>            1in (default) | 72pt | 2.54cm | 25mm
  --page-size letter|a4|legal

Structure:
  --cover                    Cover page (title, author, date, hairline rule)
  --toc                      Clickable TOC with page numbers
  --no-chapter-breaks        Don't start a new page at every H1

Branding:
  --watermark <text>         Diagonal watermark ("DRAFT", "CONFIDENTIAL")
  --header-template <html>   Custom running header
  --footer-template <html>   Custom footer (mutex with --page-numbers)
  --no-confidential          Suppress the CONFIDENTIAL right-footer

Output:
  --page-numbers             "N of M" footer (default on)
  --tagged                   Accessible PDF (default on)
  --outline                  PDF bookmarks from headings (default on)
  --quiet                    Suppress progress on stderr
  --verbose                  Per-stage timings

Network:
  --allow-network            Fetch external images. Off by default
                             (blocks tracking pixels).

Metadata:
  --title "..."              Document title (defaults to first H1)
  --author "..."             Author for cover + PDF metadata
  --date "..."               Date for cover (defaults to today)

When to use

Watch for markdown-to-PDF intent. Any of these patterns → run $P generate:

• "Can you make this markdown a PDF"
• "Export it as a PDF"
• "Turn this letter into a PDF"
• "I need a PDF of the essay"
• "Print this as a PDF for me"

If the user has a .md file open and says "make it look nice", propose $P generate --cover --toc and ask before running.

Debugging

• Output looks empty / blank → check browse daemon is running: lp_browse status.
• Fragmented text on copy-paste → highlight.js output (Phase 4). Retry with --no-syntax once that flag exists. For now, remove fenced code blocks and regenerate.
• Paged.js timeout → probably no headings in the markdown. Drop --toc.
• External image missing → add --allow-network (understand you're giving the markdown file permission to fetch from its image URLs).
• Generated PDF too tall/wide → --page-size a4 or --margins 0.75in.

Output contract

stdout: /tmp/letter.pdf          ← just the path, one line
stderr: Rendering HTML...        ← progress spinner (unless --quiet)
        Generating PDF...
        Done in 1.5s. 43 words · 22KB · /tmp/letter.pdf

exit code: 0 success / 1 bad args / 2 render error / 3 Paged.js timeout
           / 4 browse unavailable

Capture the path: PDF=$($P generate letter.md) — then use $PDF.