Model serving

Skill param087/agent-ml-skills/skills/model-serving

Production-grade Machine Learning, Data Science & MLOps skills for AI coding agents (Codex, Claude Code, Cursor, OpenCode). One npx command to install.

Install

npx -y skills add param087/agent-ml-skills --skill model-serving

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

7 stars7 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use when deploying a trained model behind an API. Covers FastAPI inference services, loading artifacts safely, request validation, batching, ONNX/quantization for speed, health checks, and monitoring.

SKILL.md

3.0 KB, as published. Nobody here has run it

Model Serving

Overview

Serving turns a saved model into a reliable, low-latency API. The concerns shift from accuracy to latency, throughput, robustness, and observability. The model artifact and its preprocessing must travel together (use the pipeline from sklearn-pipelines).

When to use

A validated model needs to be callable by other systems.
Moving from notebook to production.

Minimal FastAPI service

from fastapi import FastAPI
from pydantic import BaseModel
import joblib, numpy as np

app = FastAPI()
model = joblib.load("model.joblib")  # full pipeline: preprocessing + estimator

class Features(BaseModel):
    age: float
    income: float
    country: str
    plan: str

@app.get("/health")
def health():
    return {"status": "ok"}

@app.post("/predict")
def predict(f: Features):
    import pandas as pd
    X = pd.DataFrame([f.model_dump()])
    proba = float(model.predict_proba(X)[0, 1])
    return {"probability": proba, "label": int(proba >= 0.5)}

Run: uvicorn app:app --host 0.0.0.0 --port 8000 --workers 4.

Production checklist

Load the model once at startup, not per request.
Validate inputs with Pydantic; return 422 on bad payloads.
Health/readiness endpoints for orchestrators (k8s).
Batch requests where possible to raise throughput.
Timeouts + graceful degradation for downstream calls.
Version the model in the response (model_version) for traceability.
Pin the artifact's training env — preprocessing must match training exactly.

Speed: ONNX + quantization

# Export sklearn/torch model to ONNX, then serve with onnxruntime
import onnxruntime as ort
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
out = sess.run(None, {"input": X.astype(np.float32)})

ONNX runtime + dynamic quantization often gives 2-4x CPU speedups. For LLMs, use vLLM/TGI rather than rolling your own.

Monitoring (don't deploy blind)

Operational: latency p50/p95/p99, error rate, throughput.
ML-specific: input feature drift, prediction distribution shift, and (when labels arrive) live metric decay.
Alert on drift — a silently degrading model is worse than a down one.

Pitfalls

Training/serving skew — different preprocessing in prod than training. Ship the pipeline, not just the estimator.
Loading the model per request — kills latency.
No input validation — one bad payload takes down the worker.
pickle/joblib from untrusted sources — only load artifacts you produced.
No monitoring — you won't notice drift until users do.

Hand-off

A containerized, monitored endpoint serving the exact pipeline that model-evaluation validated and experiment-tracking registered.