agentsclimarketplace

Ai inference optimization

Skill ebarti/skills/ai-inference-optimization

📚 Agent skills distilled from technical books — AI Engineering, Context Engineering, Designing Data-Intensive Applications, and more. Agent-agnostic, plain Markdown. Give your AI agent a bookshelf.

Install
npx -y skills add ebarti/skills --skill ai-inference-optimization

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Practical knowledge for optimizing inference for foundation models. Covers inference fundamentals (computational bottlenecks, online vs batch APIs, latency/throughput metrics), AI accelerators (GPUs, TPUs, computational/memory characteristics), model optimization (compression, speculative decoding, attention optimization, kernels/compilers), and service optimization (batching, prefill/decode separation, prompt caching, parallelism). Use this skill when: - Optimizing inference latency or throughput - Choosing AI accelerators for inference - Implementing prompt caching - Setting up batching strategies - Reducing model size (quantization, pruning, distillation) - Diagnosing inference performance bottlenecks

SKILL.md

1.8 KB, as published. Nobody here has run it

AI Inference Optimization

Knowledge from "AI Engineering" by Chip Huyen (Chapter 9). Practical techniques for making inference faster and cheaper.

Quick Start

  1. Check guidelines.md to find which files to load for your task
  2. Load only relevant files (each topic has knowledge.md, rules.md, examples.md)
  3. Apply guidance to your work

Contents

References

CategoryPurpose
inference-fundamentalsComputational bottlenecks, online/batch APIs, latency/throughput/utilization metrics
ai-acceleratorsGPUs/TPUs, computational capabilities, memory size/bandwidth, power
model-optimizationCompression, speculative decoding, attention optimization, kernels/compilers
service-optimizationBatching, prefill/decode decoupling, prompt caching, parallelism

Workflows

TaskWorkflow
Diagnose and fix inference bottlenecksworkflows/diagnose-bottleneck.md

Guidelines

See guidelines.md for task-based file selection.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.