Modular tool integration

Modular Tool Integration

Core Principle

Your current approach — develop each module independently, then connect them through filesystem conventions and process launching — is architecturally sound. It follows the Unix philosophy (do one thing well) and provides the best fault isolation, independent versioning, and deployment flexibility for offline Windows environments.

The question is not whether to abandon this pattern, but which integration layer to add on top of it.

Integration Architecture Levels

Level 0: Manual Handoff (No Integration)

User runs Tool A, then manually opens Tool B and points it to Tool A's output.

• Best for: Infrequent use, tools with different owners, no coordination needed
• Cost: Zero development
• User friction: High

Level 1: Filesystem Convention + Process Launch (Your Current Pattern)

Tool A knows where to find Tool B via relative path conventions. After finishing, Tool A calls os.startfile() or subprocess.Popen() to launch Tool B.

• Best for: Two-tool workflows, offline deployment, PyInstaller exes
• Cost: Low — just path detection logic
• User friction: Low (one click to confirm)
• Limitation: No state sharing, no conditional branching

Level 2: Workflow Orchestrator (Recommended Next Step)

A thin orchestrator reads a workflow definition (YAML/JSON) and executes steps in sequence. Each step is an independent module (exe, Python script, or browser automation). The orchestrator manages context passing (e.g., output folder from step 1 becomes input folder for step 2).

• Best for: Three or more tools, conditional branches, need for retry logic
• Cost: Medium — one orchestrator script (~200 lines)
• User friction: Very low (single entry point)
• Key benefit: Tools remain independent; only the glue changes

Level 3: Shared State + Event Notification

Tools write progress/state to a shared JSON file or SQLite database. Other tools poll or watch for changes. Decouples producer and consumer in time.

• Best for: Long-running pipelines, tools that may crash and resume, parallel processing
• Cost: Medium-High — need file watching or polling logic
• User friction: Low (fully automatic)
• Risk: State file corruption, race conditions

Level 4: Plugin Architecture

A host application loads modules as plugins (DLLs, Python packages, or dynamically imported scripts). All tools share a unified UI, configuration system, and logging framework.

• Best for: Mature product with dedicated team, need for deep UI integration
• Cost: High — requires host framework design
• User friction: Lowest (seamless experience)
• Risk: Tight coupling makes independent updates difficult; PyInstaller plugins are problematic

Level 5: Message Bus / Event Stream

Tools communicate via named pipes, local sockets, or a lightweight message broker (ZeroMQ, Redis on localhost).

• Best for: Real-time coordination, many-to-many relationships, distributed tools
• Cost: Very high — overkill for offline single-machine workflows
• Not recommended for: Colleague-facing offline tool suites

Decision Framework

How many tools in the workflow?
  2  -> Level 1 is sufficient
  3+ -> Consider Level 2

Do tools need to run in parallel?
  No  -> Level 1 or 2
  Yes -> Level 3 (shared state)

Do colleagues need a single entry point?
  No  -> Level 1 (each tool has its own icon)
  Yes -> Level 2 (one orchestrator launcher)

Will the workflow change frequently?
  No  -> Level 1 or hardcoded Level 2
  Yes -> Level 2 with YAML config

Do tools share significant UI/state?
  No  -> Level 2
  Yes -> Level 4 (but evaluate cost carefully)

Recommended: Level 2 Workflow Orchestrator

For your use case (offline Windows deployment, PyInstaller exes, colleague-facing), Level 2 offers the best cost-benefit ratio.

How It Works

1. A workflow.yaml defines the pipeline:

   name: 海关发票处理流程
   steps:
     - id: convert
       tool: Excel转PDF工具.exe
       inputs:
         source_dir: "{{user.excel_dir}}"
         output_dir: "{{temp.pdf_dir}}"
     - id: upload
       tool: TLA海关查验上传工具包/启动上传工具.bat
       inputs:
         pdf_dir: "{{steps.convert.output_dir}}"
       condition: "{{steps.convert.success_count}} > 0"
   

2. A thin Python orchestrator (启动工作流.exe) reads the YAML, executes each step, and passes context.

3. Each tool remains an unchanged standalone exe.

Why This Beats Level 1 for Complex Workflows

• Single entry point: Colleagues double-click one icon, not two
• Conditional execution: Skip upload if conversion failed
• Context passing: No need to ask user for the same paths twice
• Retry logic: Orchestrator can retry failed steps
• Logging: Unified log across all steps
• Extensibility: Add new steps by editing YAML, not code

When to Stay at Level 1

If your workflow is strictly linear (A always feeds B) and will never grow beyond two tools, Level 1 is actually preferable. The pop-up dialog in your PDF tool is simpler and more robust than adding an orchestrator layer.

State Passing Patterns

Pattern A: Environment Variables

Orchestrator sets env vars before launching each tool. Tool reads them.

# Orchestrator
env = os.environ.copy()
env["PDF_OUTPUT_DIR"] = pdf_dir
subprocess.Popen([tool_path], env=env)

# Tool
output_dir = os.environ.get("PDF_OUTPUT_DIR", default_path)

Pros: Simple, works with any executable Cons: Limited to string data, pollution risk

Pattern B: Temp State File

Orchestrator writes a JSON file; tool reads it.

# Orchestrator
with open("workflow_state.json", "w") as f:
    json.dump({"pdf_dir": pdf_dir, "success": True}, f)

# Tool
if os.path.exists("workflow_state.json"):
    with open("workflow_state.json") as f:
        state = json.load(f)

Pros: Rich data types, inspectable Cons: File I/O overhead, cleanup needed

Pattern C: Command-Line Arguments

Most robust for PyInstaller tools.

# Orchestrator
subprocess.Popen([tool_path, "--pdf-dir", pdf_dir])

# Tool (with argparse)
parser.add_argument("--pdf-dir", default=".")
args = parser.parse_args()

Pros: Explicit, no hidden state, easy to test Cons: Requires modifying tools to accept CLI args

Anti-Patterns to Avoid

Anti-Pattern	Why It Fails	Better Alternative
Merge all tools into one mega-exe	Dependencies conflict (e.g., Chromium + ReportLab), huge file size, hard to debug	Keep separate, use orchestrator
Direct function calls between PyInstaller exes	PyInstaller exes are not importable Python modules	Use subprocess or state files
HTA/web frontend launching exes	WScript.Shell has Chinese path issues, security warnings	Native tkinter or orchestrator
Hard-coding absolute paths	Breaks on every colleague's machine	Relative paths from sys.executable
Tight coupling via shared global variables	Makes testing and independent use impossible	Pass state explicitly

Migration Path

If you're currently at Level 1 and want to evolve:

1. Immediate (today): Keep your current two-tool popup pattern. It works.
2. Short term (next workflow): Add a workflow.yaml and a 150-line Python orchestrator. Reuse existing exes unchanged.
3. Medium term: Extract common utilities (path resolution, logging, config) into a shared Python package that both tools import.
4. Long term (only if justified): Consider a plugin host if you have 5+ tools that all need the same UI chrome.

Reference

For complete orchestrator implementation code, YAML schema, and BAT launcher patterns, see reference.md.