agentsclimarketplace

Dspy miprov2 optimizer

Skill OmidZamani/dspy-skills/skills/dspy-miprov2-optimizer

Collection of Claude Skills for DSPy framework - program language models, optimize prompts, and build RAG pipelines systematically

Install
npx -y skills add OmidZamani/dspy-skills --skill dspy-miprov2-optimizer

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its author says it does

Copied from the file, not written here

Use for MIPROv2, Bayesian optimization, instruction and demo tuning, and high-performance DSPy program optimization.

SKILL.md

5.4 KB, as published. Nobody here has run it

DSPy MIPROv2 Optimizer

Goal

Jointly optimize instructions and few-shot demonstrations using Bayesian Optimization for maximum performance.

When to Use

  • You have 200+ training examples
  • You can afford longer optimization runs (40+ trials)
  • You need state-of-the-art performance
  • Both instructions and demos need tuning

Related Skills

Inputs

InputTypeDescription
programdspy.ModuleProgram to optimize
trainsetlist[dspy.Example]200+ training examples
metriccallableEvaluation function
autostr"light", "medium", or "heavy"
num_trialsintOptimization trials (40+)

Outputs

OutputTypeDescription
compiled_programdspy.ModuleFully optimized program

Workflow

Install the optional Optuna dependency before using MIPROv2:

pip install -U "dspy[optuna]>=3.2.1,<3.3"

Three-Stage Process

  1. Bootstrap - Generate candidate demonstrations
  2. Propose - Create grounded instruction candidates
  3. Search - Bayesian optimization over combinations

Phase 1: Setup

import dspy
from dspy.teleprompt import MIPROv2

lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)

Phase 2: Define Program

class RAGAgent(dspy.Module):
    def __init__(self):
        self.retrieve = dspy.Retrieve(k=3)
        self.generate = dspy.ChainOfThought("context, question -> answer")
    
    def forward(self, question):
        context = self.retrieve(question).passages
        return self.generate(context=context, question=question)

Phase 3: Optimize

from dspy.teleprompt import MIPROv2

optimizer = MIPROv2(
    metric=dspy.evaluate.answer_exact_match,
    auto="medium",  # Balanced optimization
    num_threads=24
)

compiled = optimizer.compile(RAGAgent(), trainset=trainset)

Auto Presets

PresetTrialsUse Case
"light"~10Quick iteration
"medium"~40Production optimization
"heavy"~100+Maximum performance

Production Example

import dspy
from dspy.teleprompt import MIPROv2
from dspy.evaluate import Evaluate
import json
import logging

logger = logging.getLogger(__name__)

class ReActAgent(dspy.Module):
    def __init__(self, tools):
        self.react = dspy.ReAct("question -> answer", tools=tools)
    
    def forward(self, question):
        return self.react(question=question)

def search_tool(query: str) -> list[str]:
    """Search knowledge base."""
    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
    return [r['long_text'] for r in results]

def optimize_agent(trainset, devset):
    """Full MIPROv2 optimization pipeline."""
    
    agent = ReActAgent(tools=[search_tool])
    
    # Baseline evaluation
    evaluator = Evaluate(
        devset=devset,
        metric=dspy.evaluate.answer_exact_match,
        num_threads=8
    )
    baseline = evaluator(agent)
    logger.info(f"Baseline: {baseline:.2%}")
    
    # MIPROv2 optimization
    optimizer = MIPROv2(
        metric=dspy.evaluate.answer_exact_match,
        auto="medium",
        num_threads=24,
        # Custom settings
        num_candidates=15,
        max_bootstrapped_demos=4,
        max_labeled_demos=8
    )
    
    compiled = optimizer.compile(agent, trainset=trainset)
    optimized = evaluator(compiled)
    logger.info(f"Optimized: {optimized:.2%}")
    
    # Save with metadata
    compiled.save("agent_mipro.json")
    
    metadata = {
        "baseline_score": baseline,
        "optimized_score": optimized,
        "improvement": optimized - baseline,
        "num_train": len(trainset),
        "num_dev": len(devset)
    }
    
    with open("optimization_metadata.json", "w") as f:
        json.dump(metadata, f, indent=2)
    
    return compiled, metadata

Instruction-Only Mode

from dspy.teleprompt import MIPROv2

# Disable demos for pure instruction optimization
optimizer = MIPROv2(
    metric=metric,
    auto="medium",
    max_bootstrapped_demos=0,
    max_labeled_demos=0
)

Best Practices

  1. Data quantity matters - 200+ examples for best results
  2. Use auto presets - Start with "medium", adjust based on results
  3. Parallel threads - Use num_threads=24 or higher if available
  4. Monitor costs - Track API usage during optimization
  5. Save intermediate - Bayesian search saves progress

Limitations

  • High computational cost (many LLM calls)
  • Requires substantial training data
  • Optimization time: hours for "heavy" preset
  • Memory intensive for large candidate sets

Official Documentation

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.