Coreweave performance tuning

Skill jeremylongshore/claude-code-plugins-plus-skills/plugins/saas-packs/coreweave-pack/skills/coreweave-performance-tuning

'Optimize CoreWeave GPU inference latency and throughput.From its SKILL.md

Install

npx -y skills add jeremylongshore/claude-code-plugins-plus-skills --skill coreweave-performance-tuning

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its file declares

Copied from the file, not written here

The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

2.3 KB, 538 tokens by cl100k_base, as published. Nobody here has run it

CoreWeave Performance Tuning

Community-contributed. Not affiliated with, endorsed by, or sponsored by CoreWeave, Inc. CoreWeave is a registered trademark of CoreWeave, Inc.

GPU Selection by Workload

Workload	Recommended GPU	Why
LLM inference (7-13B)	A100 80GB	Good balance of memory and cost
LLM inference (70B+)	8xH100	NVLink for tensor parallelism
Image generation	L40	Good for diffusion models
Training (large models)	8xH100 SXM5	Fastest interconnect
Batch processing	A100 40GB	Cost-effective

Inference Optimization

# Continuous batching with vLLM
containers:
  - name: vllm
    args:
      - "--model=meta-llama/Llama-3.1-8B-Instruct"
      - "--max-num-batched-tokens=8192"
      - "--max-num-seqs=256"
      - "--gpu-memory-utilization=0.90"
      - "--enable-prefix-caching"
      - "--dtype=float16"

Autoscaling Tuning

# HPA based on GPU utilization
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: inference-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: inference-server
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Pods
      pods:
        metric:
          name: DCGM_FI_DEV_GPU_UTIL
        target:
          type: AverageValue
          averageValue: "70"

Performance Benchmarks

Metric	A100-80GB	H100-80GB
Llama-8B tokens/sec	~2,000	~4,500
Llama-70B tokens/sec	~200 (4x)	~500 (4x)
Cold start (vLLM)	30-60s	20-40s

Resources

Next Steps

For cost optimization, see coreweave-cost-tuning.

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.