Groq migration deep dive

Groq Migration Deep Dive

Current State

!npm list groq-sdk openai @anthropic-ai/sdk 2>/dev/null | grep -E "groq|openai|anthropic" || echo 'No LLM SDKs found'

Overview

Migrate to Groq from OpenAI, Anthropic, or other LLM providers. Groq's OpenAI-compatible API makes migration straightforward — the primary changes are a different SDK import, different model IDs, and different response metadata. The reward is 10-50x faster inference.

The safe path is a provider-abstraction layer plus feature-flagged traffic shifting: route a small canary to Groq, benchmark quality and speed, ramp to 100%, and keep a one-flag rollback the whole way.

Migration Complexity

Source	Complexity	Key Changes
OpenAI	Low	Import, model IDs, base URL — API shape is identical
Anthropic	Medium	Different API shape, message format, streaming protocol
Local LLMs	Medium	Remove infra, add API calls
Other cloud (Bedrock, Vertex)	Medium	Remove cloud SDK, add groq-sdk

Prerequisites

• A Groq API key (GROQ_API_KEY) from console.groq.com.
• groq-sdk installed: npm install groq-sdk.
• A feature-flag mechanism (LaunchDarkly, env var, config service) exposing a groq_migration_pct value for gradual traffic shifting.
• The existing provider's key still available (OPENAI_API_KEY or equivalent) so you can run both providers side-by-side during the cutover.
• Node >=18 if you use the performance.now() benchmark helper.

Instructions

Steps 1-2 below are the essential skeleton — the two changes every migration needs. Steps 3-7 (the provider abstraction, traffic shifting, scanner, benchmark, and full compatibility matrix) are moved verbatim into the reference files linked under Examples so this file stays scannable.

Step 1: OpenAI to Groq Migration

The minimal change: swap the SDK import, client, and model ID. The response shape is identical, so downstream code (result.choices[0].message.content) is untouched.

// BEFORE: OpenAI
import OpenAI from "openai";
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const result = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello" }],
});

// AFTER: Groq (minimal changes)
import Groq from "groq-sdk";
const groq = new Groq({ apiKey: process.env.GROQ_API_KEY });
const result = await groq.chat.completions.create({
  model: "llama-3.3-70b-versatile",  // or "llama-3.1-8b-instant"
  messages: [{ role: "user", content: "Hello" }],
});

// Same response shape: result.choices[0].message.content

Step 2: Model ID Mapping

Centralize the OpenAI/Anthropic → Groq translation so a single map drives the whole codebase and unknown models fall back to a safe default.

// OpenAI → Groq model equivalents
const MODEL_MAP: Record<string, string> = {
  // OpenAI → Groq (quality equivalent)
  "gpt-4o":        "llama-3.3-70b-versatile",
  "gpt-4o-mini":   "llama-3.1-8b-instant",
  "gpt-4-turbo":   "llama-3.3-70b-versatile",
  "gpt-3.5-turbo": "llama-3.1-8b-instant",

  // Anthropic → Groq (approximate)
  "claude-3-5-sonnet": "llama-3.3-70b-versatile",
  "claude-3-haiku":    "llama-3.1-8b-instant",
};

function migrateModelId(model: string): string {
  return MODEL_MAP[model] || "llama-3.3-70b-versatile";
}

Steps 3-7: Zero-Downtime Rollout

Once Steps 1-2 compile, wrap both providers in a common interface and shift traffic gradually. The full code lives in the references:

• Step 3 — Provider abstraction layer and Step 4 — feature-flag traffic shifting: references/implementation.md.
• Step 5 — automated migration scanner (sizes the migration before you start) and the rollback plan: references/implementation.md.
• Step 6 — comparison benchmark and Step 7 — the OpenAI↔Groq compatibility matrix: references/examples.md.

Output

Running this skill's workflow produces:

• A migration assessment printout from the scanner (Step 5): OpenAI import count, the distinct gpt-* model IDs in use, any OpenAI-only features (embeddings/images/fine-tuning) that block a clean cutover, and the number of API-key references to update.
• A provider-agnostic LLMProvider layer with GroqProvider and OpenAIProvider implementations both live behind one getProvider() call.
• A benchmark table per prompt: Groq vs OpenAI latency in ms, token counts, and the measured speedup factor.
• A groq_migration_pct feature flag driving the canary → 100% ramp, with a one-flag rollback to 0%.

Error Handling

Issue	Cause	Solution
Quality regression	Different model strengths	Tune system prompts for Llama models
Missing features	Groq doesn't have embeddings/images	Keep OpenAI for those features
Rate limits	Different limits than OpenAI	Configure per-model rate limits
Cost increase	Different pricing structure	Route simple tasks to 8B model

Examples

• Full provider abstraction + traffic shifting + scanner + rollback: references/implementation.md — the complete Step 3-5 code plus the rollback procedure.
• Benchmark harness + compatibility matrix: references/examples.md — the side-by-side quality/speed benchmark and the full OpenAI↔Groq feature table.

Resources

• Groq Quickstart
• Groq Models
• Groq API Reference
• groq-sdk npm

Next Steps

For ongoing SDK version upgrades between Groq model generations, see the companion groq-upgrade-migration skill in this pack.