LLM Inference Optimization Techniques: The Map by Bottleneck

LLM inference optimization advice usually arrives as a pile of technique names — quantize, batch, cache, parallelize — with no map of which technique treats which problem. Applied blind, that pile produces the most common waste in this work: fixing the wrong bottleneck with the right technique, efficiently. This page is the map: six technique families, each keyed to the bottleneck it treats, the rules for combining them, and the application order that starts with diagnosis.

The Technique Families and What Each Treats

Six families cover the optimization landscape — quantization (memory footprint and bandwidth), batching (throughput), KV cache management including paged and prefix caching (memory and its bandwidth), attention kernels like Flash Attention classes (compute efficiency), speculative decoding (exposed latency), and parallelism (scale-out) — and each family treats a specific bottleneck, which is the only sane way to organize the map.

The catalog's members evolve monthly; its families are stable, which is why the map organizes by family. Vendor engineering guidance catalogs the same territory — MQA and GQA reducing KV cache memory, kernel choices, batching strategies — and educational roadmaps organize by the same axes the families key on: latency, throughput, memory, cost. The organizing principle is the map's whole value: a technique is only "an optimization" relative to a bottleneck it treats.

How the Techniques Combine

Most families compose: quantization shrinks weights while batching packs requests and cache management cuts redundant computation — production stacks routinely run all three — while parallelism is orthogonal scale-out and speculative decoding interacts with batching (speculation favors lower concurrency), so combinations need measured verification rather than assumed addition.

The composition table's last rule is the one teams skip: interactions are engine-specific hypotheses until benchmarked on your stack, so add techniques one at a time with the benchmark watching — the same manifest discipline the site's benchmark-methodology page defines. The stack that measured five percent from its last addition is a stack that knows what each family contributed; the stack that applied everything at once knows only the total.

Application Order: Diagnose, Then Treat

Start with the diagnosis — measure which bottleneck binds (memory, bandwidth, compute, or exposed latency) — then apply the family that treats it, in the site's per-technique depth: fixing the wrong bottleneck with the right technique is the most common waste in inference optimization work.

1. Diagnose: measure where serving binds — the site's bottleneck-diagnosis method (LLM inference optimization starts with the bottleneck) walks this step; the map assumes its output.
2. Treat: apply the family keyed to the measured bottleneck — memory-bound to quantization and cache management, underutilized GPUs to batching, single-request latency to speculation, single-device ceilings to parallelism.
3. Go deep: within the family, the site's per-technique pages carry the implementation detail — the KV cache explainer, the quantization comparison, the speculative decoding guide, the batching series.
4. Re-measure: after each addition, the benchmark confirms the gain and re-identifies the next binding constraint — the loop continues until the next treatment costs more than it returns.

The map and the diagnosis are complementary halves of one method: the diagnosis tells you where you hurt, the map tells you what treats it, and the deep dives tell you how. Teams that internalize the pairing stop asking "what should we optimize" and start asking "what is binding" — which is the question the serving stack can actually answer.

FAQ

Which optimization should we try first?

Whichever family treats your measured bottleneck — memory-bound serving points at quantization and cache management, underutilized GPUs at batching, single-request latency at speculative decoding — which is why diagnosis precedes treatment: the first technique is chosen by measurement, not by fashion.

Can we apply multiple techniques together?

Mostly yes: quantization, batching, and cache management compose routinely and production stacks run all three — but interactions (speculative decoding with heavy batching, aggressive quantization with cache compression) need measured verification, so add techniques one at a time with the benchmark watching.

What does 'fastest inference' actually mean?

It must be defined before optimizing: lowest single-request latency and highest batch throughput are different targets with different technique mixes — the fastest engine for one is rarely fastest for the other, so name the metric (TTFT, per-token latency, tokens per second) before choosing treatments.