agentsclimarketplace
Sign in

Ai inference optimization

Skill ebarti/skills/ai-inference-optimization

Practical knowledge for optimizing inference for foundation models. Covers inference fundamentals (computational bottlenecks, online vs batch APIs, latency/throughput metrics), AI accelerators (GPUs, TPUs, computational/memory characteristics), model optimization (compression, speculative decoding, attention optimization, kernels/compilers), and service optimization (batching, prefill/decode separation, prompt caching, parallelism). Use this skill when: - Optimizing inference latency or throughput - Choosing AI accelerators for inference - Implementing prompt caching - Setting up batching strategies - Reducing model size (quantization, pruning, distillation) - Diagnosing inference performance bottlenecksFrom its SKILL.md

Install
npx -y skills add ebarti/skills --skill ai-inference-optimization

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.8 KB, 247 tokens by cl100k_base, as published. Nobody here has run it

AI Inference Optimization

Knowledge from "AI Engineering" by Chip Huyen (Chapter 9). Practical techniques for making inference faster and cheaper.

Quick Start

  1. Check guidelines.md to find which files to load for your task
  2. Load only relevant files (each topic has knowledge.md, rules.md, examples.md)
  3. Apply guidance to your work

Contents

References

CategoryPurpose
inference-fundamentalsComputational bottlenecks, online/batch APIs, latency/throughput/utilization metrics
ai-acceleratorsGPUs/TPUs, computational capabilities, memory size/bandwidth, power
model-optimizationCompression, speculative decoding, attention optimization, kernels/compilers
service-optimizationBatching, prefill/decode decoupling, prompt caching, parallelism

Workflows

TaskWorkflow
Diagnose and fix inference bottlenecksworkflows/diagnose-bottleneck.md

Guidelines

See guidelines.md for task-based file selection.

What ships with it: 17 files

95.6 KB alongside SKILL.md

references/

workflows/

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.