Modal

Modal provides serverless containers with sub-5s cold starts. No SSH, no Kubernetes, no Docker registry. Docs · Examples · API Reference

First-Time Setup

Prerequisites

• Python 3.11+ and uv installed
• Modal package: If using this skills repo, make install handles it. Standalone: uv tool install modal

1. Create a Modal Account

Sign up at modal.com. Go to Settings → API Tokens to create a token pair (token ID + secret).

2. Authenticate

For humans — token file (simplest):

uv run modal token set --token-id <ID> --token-secret <SECRET>

Writes to ~/.modal.toml. Use --profile <name> for multiple accounts, then modal profile activate <name>.

For agents and CI — environment variables (takes precedence over token file): Add to ~/.zshrc (or ~/.bashrc) so they're available when Claude Code launches:

export MODAL_TOKEN_ID="ak-..."
export MODAL_TOKEN_SECRET="as-..."

Then source ~/.zshrc or restart your terminal.

3. Verify

uv run --script SKILL_DIR/scripts/modal_helper.py check-auth

CLI (always use uv run modal)

uv run modal run script.py                # run @app.local_entrypoint()
uv run modal run script.py --detach       # run in background (survives terminal close)
uv run modal serve script.py              # dev server with hot reload (web endpoints)
uv run modal deploy script.py             # persistent production deployment
uv run modal shell --gpu A100             # interactive GPU shell
uv run modal volume create my-data        # create volume
uv run modal volume put my-data ./local /remote
uv run modal volume get my-data /remote ./local

GPU Options & Pricing

GPU	VRAM	$/hr	Max per container	Best for
T4	16 GB	$0.59	8	Light inference, prototyping
L4	24 GB	$0.80	8	Lightweight inference
A10G	24 GB	$1.10	4	Inference, moderate training
L40S	48 GB	$1.95	8	Vision, mid training
A100 (40 GB)	40 GB	$2.10	8	Training, large inference
A100 (80 GB)	80 GB	$2.50	8	7B-70B models
H100	80 GB	$3.95	8	Large-scale training, best ecosystem support
H200	141 GB	$4.54	8 (1,128 GB total)	Large models, 671B fits on 8x
B200	192 GB	$6.25	8 (1,440 GB total)	Fastest; ~2.5x faster TTFT vs H200

Multi-GPU: gpu="A100:4". GPU fallbacks: gpu=["h100", "a100", "any"]. CPU-only: omit gpu=. Model on-disk size ≠ VRAM, and a model that doesn't fit spills to CPU: a 25B MoE runs ~0.8 tok/s on a 24 GB L4 vs ~12 tok/s on a 48 GB L40S. Size for capacity first, bandwidth second. Pricing details · GPU guide

Patterns

GPU function (simplest)

import modal

app = modal.App("my-app")
image = modal.Image.debian_slim(python_version="3.12").uv_pip_install("torch")


@app.function(gpu="A100", image=image, timeout=3600)
def train():
    import torch

    print(f"CUDA: {torch.cuda.is_available()}, GPU: {torch.cuda.get_device_name(0)}")


@app.local_entrypoint()
def main():
    train.remote()

Stateful service with model loading (@app.cls)

Use for GPU inference — load model once, serve many requests:

import modal

app = modal.App("inference")
image = modal.Image.debian_slim(python_version="3.12").uv_pip_install("vllm")


@app.cls(gpu="H100", image=image)
@modal.concurrent(max_inputs=32)
class LLM:
    @modal.enter()
    def load(self):
        import vllm

        self.engine = vllm.LLM("meta-llama/Llama-3-8B")

    @modal.method()
    def generate(self, prompt: str) -> str:
        return self.engine.generate(prompt)[0].outputs[0].text

    @modal.exit()
    def cleanup(self):
        del self.engine

Lifecycle hooks · Concurrency

Parallel map (CPU swarm or GPU fleet)

@app.function(image=image)
def process(item: str) -> str:
    return f"processed: {item}"


@app.local_entrypoint()
def main():
    results = list(process.map(["a", "b", "c", "d"]))  # parallel, ordered
    print(results)

.map() for ordered results, .for_each() for fire-and-forget, .starmap() for multi-arg. Scaling out

Async spawn with gather

@app.local_entrypoint()
def main():
    job_a = step1.spawn("foo")
    job_b = step2.spawn(42)
    results = modal.FunctionCall.gather(job_a, job_b)

.spawn() returns immediately. .get() blocks for result. .spawn_map() for async fan-out.

Volumes for persistent storage

vol = modal.Volume.from_name("my-data", create_if_missing=True)


@app.function(volumes={"/data": vol})
def save():
    Path("/data/output.txt").write_text("done")
    vol.commit()  # required for writes to be visible to other containers

Volumes guide

Web endpoint

@app.function(gpu="A100", image=image)
@modal.fastapi_endpoint()
def predict(prompt: str):
    return {"result": "..."}

Deploy: uv run modal deploy script.py — returns a URL. For full apps: @modal.asgi_app() (FastAPI), @modal.wsgi_app() (Flask/Django). Put the ASGI factory on an @app.cls method, not a standalone @app.function() — the standalone form can return None at container start on modal 1.4.x, 500ing every route. Web endpoints · ASGI gotchas

Scheduled function

@app.function(schedule=modal.Cron("0 */6 * * *"))
def periodic_job():
    print("Runs every 6 hours")

Only active when deployed (uv run modal deploy). Also: modal.Period(hours=1). Scheduling

Dynamic batching (high-throughput inference)

@app.cls(gpu="H100", image=image)
class Embedder:
    @modal.enter()
    def load(self):
        self.model = load_model()

    @modal.batched(max_batch_size=64, wait_ms=1000)
    async def embed(self, texts: list[str]) -> list[list[float]]:
        return self.model.encode(texts).tolist()

Callers send single inputs; Modal accumulates into batches transparently. Dynamic batching

Scaling Config

Parameter	Purpose
max_containers	Hard cap on container count
min_containers	Warm floor (prevents scale-to-zero; costs money when idle)
buffer_containers	Extra warm containers while function is active (burst handling)
scaledown_window	Seconds before idle container shuts down (default 60, max 3600¹). Keep it low on web-serving containers — deploys don't restart warm ones
single_use_containers=True	Fresh container per input (no reuse)
timeout	Max execution time per input (max 24h)
retries	Int or modal.Retries(initial_delay=0.0, max_retries=10)

¹ Docs say max 1200; the deploy validator accepts 3600 ("must be between 2 and 3600 inclusive", live-verified 2026-06-23).

Image Building

image = (
    modal.Image.debian_slim(python_version="3.12")
    .apt_install("ffmpeg", "git")
    .uv_pip_install("torch==2.8.0", "transformers")  # pin versions
    .env({"MY_VAR": "value"})
    .add_local_python_source("my_package")
)

Also: Image.from_registry(...), Image.from_dockerfile(...), Image.uv_sync() for uv projects. Images guide · Image reference

For advanced patterns (sandboxes, dicts/queues, secrets, 1.0 migration) and field-verified production gotchas (billing gate, stale warm containers, readiness signals, third-party images), see reference.md.

Troubleshooting

• Auth failed: Run uv run modal token set or set MODAL_TOKEN_ID + MODAL_TOKEN_SECRET
• Please add a payment method: GPU deploys need a card on file — a credit balance doesn't count. Settings → Usage & Billing. Already-deployed GPU functions keep serving; only new deploys are gated
• Deploy didn't take effect: modal deploy does not restart warm containers — they serve the old code until scaledown_window expires. See reference.md
• Every route 500s with 'NoneType' object is not callable: standalone @modal.asgi_app() factory returned None (modal 1.4.x) — move the factory onto an @app.cls method
• OOM on GPU: Use a larger GPU or reduce batch size
• Timeout: Increase timeout= (default 300s, max 24h). Use --detach for long jobs
• Import errors in container: Add packages to .uv_pip_install() in the image definition
• Cold starts slow: Use min_containers, or bake model into image with Image.run_function()