External llm

External LLM

Multi-turn conversation with an external LLM via any OpenAI-compatible API. Designed primarily for reasoning models (DeepSeek Reasoner, o1, o3) where multi-turn verification and pushback are valuable. Works with any OpenAI-compatible provider.

Setup

Set your API credentials as environment variables:

# Required
export LLM_API_KEY="your-key-here"

# Optional (defaults shown)
export LLM_BASE_URL="https://api.deepseek.com"    # any OpenAI-compatible endpoint
export LLM_MODEL="deepseek-reasoner"                # model to use

Common configurations

Provider	LLM_BASE_URL	LLM_MODEL	Get a key
DeepSeek	https://api.deepseek.com	deepseek-reasoner	https://platform.deepseek.com
OpenAI	https://api.openai.com/v1	o1, o3-mini, gpt-4o	https://platform.openai.com
Groq	https://api.groq.com/openai/v1	llama-3.3-70b-versatile	https://console.groq.com
Together	https://api.together.xyz/v1	meta-llama/Llama-3-70b-chat-hf	https://api.together.xyz
Ollama (local)	http://localhost:11434/v1	llama3	--

Invocation

/external-llm <message>      # Send message to external LLM
/external-llm /report        # Get final report from conversation
/external-llm /new [name]    # Start fresh session
/external-llm /sessions      # List sessions
/external-llm /transcript    # Show full conversation

How Agents Should Use This

Starting a Conversation

/external-llm /new verify-theorem3

Then send your prompt:

/external-llm "You are verifying a mathematical proof.
Be SKEPTICAL. Show explicit work, not vague assurances.

## Content to Verify
[content]

## Concerns
[list]

For each concern, show the explicit verification or flag as ISSUE."

Analyzing and Pushing Back

After each response:

1. Check for vagueness — "clearly", "obviously", "by inspection"
2. Check for unverified claims — results stated without justification
3. Check for brief "ALL CLEAR" — needs explicit verification

If any of these, send pushback:

/external-llm "You said X but didn't show work.
Please compute Y explicitly. Don't just say it works."

Getting the Report

After the conversation is complete (typically 3-15 turns):

/external-llm /report

Then synthesize the conversation into a structured report.

Session Management

Sessions are stored as JSON files in ~/.cache/external-llm/sessions/.

Command	Description
/new [name]	Start fresh session
/sessions	List all sessions with turn counts
/transcript	Show full conversation
/report	Mark report requested (agent should generate)

Implementation

Requires requests (pip install requests).

python3 << 'PYEOF'
import requests, json, os, sys
from datetime import datetime

SESSIONS_DIR = os.path.expanduser("~/.cache/external-llm/sessions")
os.makedirs(SESSIONS_DIR, exist_ok=True)

def load_session(name="default"):
    path = f"{SESSIONS_DIR}/{name}.json"
    if os.path.exists(path):
        with open(path) as f:
            return json.load(f)
    return {"name": name, "created": datetime.now().isoformat(), "messages": [], "metadata": {}}

def save_session(session):
    path = f"{SESSIONS_DIR}/{session['name']}.json"
    with open(path, 'w') as f:
        json.dump(session, f, indent=2)

def get_current_session_name():
    marker = f"{SESSIONS_DIR}/.current"
    if os.path.exists(marker):
        with open(marker) as f:
            return f.read().strip()
    return "default"

def set_current_session(name):
    with open(f"{SESSIONS_DIR}/.current", 'w') as f:
        f.write(name)

def call_llm(messages):
    api_key = os.environ.get("LLM_API_KEY")
    if not api_key:
        return "ERROR: LLM_API_KEY environment variable not set."

    base_url = os.environ.get("LLM_BASE_URL", "https://api.deepseek.com").rstrip("/")
    model = os.environ.get("LLM_MODEL", "deepseek-reasoner")

    r = requests.post(
        f"{base_url}/chat/completions",
        headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
        json={"model": model, "messages": messages, "max_tokens": 16384},
        timeout=600
    )

    if r.status_code == 200:
        msg = r.json()["choices"][0]["message"]
        content = msg.get("content", "") or ""
        reasoning = msg.get("reasoning_content", "") or ""
        if reasoning and content:
            return f"[REASONING]\n{reasoning}\n\n[OUTPUT]\n{content}"
        return reasoning or content
    return f"ERROR {r.status_code}: {r.text}"

arg = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else ""

if arg.startswith("/new"):
    parts = arg.split(maxsplit=1)
    name = parts[1] if len(parts) > 1 else "default"
    session = {"name": name, "created": datetime.now().isoformat(), "messages": [], "metadata": {}}
    save_session(session)
    set_current_session(name)
    print(f"Started new session: {name}")
    sys.exit(0)

elif arg == "/sessions":
    files = sorted([f for f in os.listdir(SESSIONS_DIR) if f.endswith('.json')])
    if not files:
        print("No sessions found.")
    for fname in files:
        with open(f"{SESSIONS_DIR}/{fname}") as f:
            s = json.load(f)
            turns = len(s['messages']) // 2
            print(f"  {s['name']}: {turns} turns")
    sys.exit(0)

elif arg == "/transcript":
    name = get_current_session_name()
    session = load_session(name)
    print(f"=== TRANSCRIPT: {name} ({len(session['messages'])//2} turns) ===\n")
    for i, msg in enumerate(session['messages']):
        role = msg['role'].upper()
        content = msg['content']
        print(f"\n[{role}]:\n{content}\n")
        print("-" * 40)
    sys.exit(0)

elif arg == "/report":
    name = get_current_session_name()
    session = load_session(name)
    turns = len(session['messages']) // 2
    print(f"Session: {name} ({turns} turns)")
    print("REPORT_REQUESTED")
    sys.exit(0)

# Regular message
current_name = get_current_session_name()
session = load_session(current_name)
session['messages'].append({"role": "user", "content": arg})

reply = call_llm(session['messages'])

if not reply.startswith("ERROR"):
    session['messages'].append({"role": "assistant", "content": reply})
    save_session(session)

print(reply)
PYEOF

Example Multi-Turn Verification

# Turn 1: Initial prompt
/external-llm /new verify-convergence
/external-llm "Verify this claim: the series converges for all p > 2.
Show the explicit bound and check the boundary case p = 2."

# LLM responds with analysis

# Turn 2: Push back on vague step
/external-llm "You claimed the tail bound works but didn't show the computation.
Write out the sum explicitly for N = 10, 100, 1000."

# LLM works through the cases

# Turn 3: Check edge case
/external-llm "What happens at p = 2 exactly? Does it diverge? How fast?"

# LLM addresses edge case

# Turn 4: Request report
/external-llm /report

Notes

• Response time: Varies by provider — reasoning models (DeepSeek, o1) take 30-120s, chat models are faster
• Max turns: Aim for 3-6 turns per topic
• Push back: Don't accept vague answers — ask for explicit work
• Session files: Inspect .json files in ~/.cache/external-llm/sessions/ to debug conversations
• reasoning_content: DeepSeek Reasoner returns this field; other providers may not. The script handles both cases.