Spec splitter

Spec Splitter

Two functions for specification processing:

1. Split: Break a bound project manual PDF into individual spec section PDFs — navigable files the project team can use directly
2. Extract: Pull searchable text from each section PDF into persistent .txt files — enables downstream skills (submittal-log-generator, spec-parser) to work from text without re-extracting from PDFs

Pipeline Position

Run after /project-setup identifies bound spec manuals. Produces split PDFs, spec_index.yaml, and extracted text consumed by /submittal-log-generator and /code-researcher.

Either function can run independently. For example, specs may already be split but text has not yet been extracted.

Workflow

Spec Split Progress:
- [ ] Step 1: Check current state (split? text extracted?)
- [ ] Step 2: Discover Specifications directory
- [ ] Step 3: Find ALL spec PDFs (bound manuals)
- [ ] Step 4-5: Split PDF into individual section files
- [ ] Step 6: Write spec index
- [ ] Step 7: Extract text from all sections
- [ ] Step 8: Repair degraded/poor text quality
- [ ] Step 9: Write graph entry (AgentCM only)

Step 1: Check Current State

Check what already exists:

Split PDFs present?

• Look for individual spec section PDFs with CSI section numbers in filenames (e.g., 03 30 00 - Cast-in-Place Concrete.pdf)
• Check for spec_index.yaml
• If found, report count and skip to Step 7 (text extraction)

Text already extracted?

• Check .construction/spec_text/manifest.json
• If manifest exists and covers all sections, report and skip Step 7

Step 2: Discover Specifications Directory

Determine where split spec PDFs should go. Search for an existing Specifications directory (case-insensitive):

1. 02 - Specifications/ (numbered project folder convention)
2. Specifications/
3. Any folder with "specification" in the name

Output directory resolution:

• If Specifications directory found → output to {specs_dir}/Specification Sections/
• If not found → output to Specification Sections/ in project root

Step 3: Find ALL Spec PDFs

Search the project directory for ALL PDFs that are specifications. Many projects have multiple spec PDFs:

• Multi-volume: Volume 1.pdf, Volume 2.pdf (split by CSI division range)
• Single bound manual: one large PDF with all sections
• Attachment-based: Attachment-E-Specs.pdf (government projects)

Search in:

• The Specifications directory discovered in Step 2
• The project root (some projects have no folder structure)
• Look for PDFs > 1MB with keywords: "spec", "manual", "volume", "attachment" + spec-related terms

Process EACH PDF found. All split sections go to the same output directory.

If the user specifies a single file (/spec-splitter path/to/specific-volume.pdf), process only that file.

Steps 4-6: Split and Index

Run the split script with the resolved output directory:

${CLAUDE_SKILL_DIR}/../../bin/construction-python ${CLAUDE_SKILL_DIR}/scripts/split_spec_manual.py \
  "{project_manual.pdf}" \
  --output-dir "{resolved_spec_sections_dir}"

The script:

1. Scans ALL pages for SECTION XX XX XX headers to find exact page boundaries — this is the primary method and does NOT depend on a Table of Contents
2. Scans all pages for Table of Contents entries to enrich section titles (optional, best-effort)
3. For sections without ToC titles, extracts titles directly from the section header page
4. Splits into individual PDFs named {section_number} - {SECTION TITLE}.pdf
5. Writes spec_index.yaml with section metadata

ToC edge cases handled:

• ToC located deep in the document (e.g., page 60+): Common when front matter (transmittals, addenda) precedes the project manual. The script scans all pages, not just the first few.
• No ToC at all: Section boundaries are found by scanning every page for SECTION headers. Titles are extracted directly from each section's title page. The split still succeeds — titles may be slightly less polished than ToC-enriched versions.

Step 7: Extract Text

After splitting (or if specs are already split), extract searchable text from every section:

${CLAUDE_SKILL_DIR}/../../bin/construction-python ${CLAUDE_SKILL_DIR}/scripts/extract_spec_text.py \
  --specs-dir "{resolved_spec_sections_dir}" \
  --output-dir ".construction/spec_text"

The script:

• Extracts text from each section PDF via pdfplumber
• Assesses extraction quality (GOOD / DEGRADED / POOR)
• Writes one .txt file per section to .construction/spec_text/
• Writes manifest.json with quality metadata per section
• Incremental: skips sections that already have .txt files (use --force to re-extract all)

Step 8: Text Repair — GUIDED

After extraction, check manifest.json for sections rated DEGRADED or POOR. Spec-splitter owns text quality — downstream skills (submittal-log-generator, spec-parser) expect clean, repaired text.

For DEGRADED sections — attempt repair:

1. Read the .txt file and identify failure modes from the manifest
2. Split word repair: Scan for sequences of short tokens (≤2 chars) not in known abbreviation lists (GC, CM, PE, QA, SF, LF, etc.). Attempt progressive concatenation of adjacent tokens. Validate against construction vocabulary. Merge if valid; leave as-is if not.
3. Merged word repair: Tokens >25 characters that contain multiple dictionary words — insert spaces at word boundaries
4. Garbled character repair: Replace known encoding artifacts (e.g., é → é, ligature breakage)
5. After repair, re-assess quality. If improved, overwrite the .txt file and update the manifest with "repair_attempted": true and the new quality rating.
6. If repair made things worse, discard repairs and fall back to vision.

For POOR sections — vision extraction fallback:

1. Render each page of the section PDF as an image:

   ${CLAUDE_SKILL_DIR}/../../bin/construction-python ${CLAUDE_SKILL_DIR}/../../scripts/pdf/rasterize_page.py "{section.pdf}" {page} --dpi 200 --output spec_page.png
   

2. Process each page image through vision:

   Extract all text from this construction specification page.
   Preserve paragraph structure, numbering (A, B, C, 1, 2, 3),
   and indentation hierarchy. This is CSI-formatted specification
   section [SECTION NUMBER] - [SECTION TITLE].
   

3. Concatenate extracted text in page order
4. Write the vision-extracted text to .construction/spec_text/, overwriting the POOR pdfplumber output
5. Update manifest: "extraction_method": "vision", "repair_attempted": true, new quality rating

Known abbreviation preservation list (do not merge these during repair):

• Standard: A, I, or, an, as, at, be, by, do, if, in, is, it, no, of, on, so, to, up, we
• Construction: GC, CM, PE, QA, QC, SF, LF, CY, EA, LS, GA, MIL, PSI, KSI, CFM, GPM
• Section refs: A, B, C, D (as paragraph identifiers)

Output

{Specifications dir}/Specification Sections/
  01 10 00 - SUMMARY.pdf
  03 30 00 - CAST-IN-PLACE CONCRETE.pdf
  08 71 00 - DOOR HARDWARE.pdf
  ...
  spec_index.yaml

.construction/spec_text/
  01_10_00.txt
  03_30_00.txt
  08_71_00.txt
  ...
  manifest.json

Step 9: Write Graph Entry (AgentCM only)

If .construction/ directory exists, write a graph entry:

${CLAUDE_SKILL_DIR}/../../bin/construction-python ${CLAUDE_SKILL_DIR}/../../scripts/graph/write_finding.py \
  --type "specs_split" \
  --title "Spec sections split: {N} sections from {source_pdf}" \
  --data '{"section_count": N, "source_pdf": "...", "output_dir": "...", "quality_summary": {"good": X, "degraded": Y, "poor": Z}}'

If no .construction/ directory exists, skip this step — the spec_index.yaml and manifest.json files serve as the local record.

Report to user: number of sections split, total pages, text extraction quality summary (GOOD/DEGRADED/POOR counts), and output locations.

File Safety

Never overwrite existing split spec PDFs or extracted text. The split script skips existing sections. Text extraction overwrites only with --force. The spec_index.yaml merge is additive.

---

Allowed Scripts

Allowed scripts — exhaustive list. Only execute these scripts during this skill:

• scripts/split_spec_manual.py — split bound PDF into per-section PDFs
• scripts/extract_spec_text.py — extract searchable text from section PDFs
• ../../scripts/pdf/rasterize_page.py — rasterize PDF pages for vision fallback
• ../../scripts/graph/write_finding.py — graph entry (Step 9)