GPUscale.net

v4.7

LLM capacity & dimensioning studio: model, precision, workload and hardware in; memory fit, latency, throughput and resilient topology out.

Scenario

Model

Architecture geometry drives every estimate: total params size the weights, active params drive compute and decode reads, and the effective KV heads × head dim encode each model's real cache cost (MLA, sparse and SSM hybrids use sheet-compatible effective values).

Custom model · enter geometry manually

Precision

Weights

KV cache

Bytes per weight. FP8 is the 2026 serving default (half of BF16, minimal loss). NVFP4/MXFP4 are Blackwell-native 4-bit floats; Q-tiers are GGUF/llama.cpp formats.Typical production: FP8 · edge/workstation: Q4_K_M or MXFP4

Bytes per KV element. FP8 KV halves cache cost with negligible quality impact and is the vLLM production default.Typical: FP8 · long-context on tight VRAM: INT4

Workload

Use-case preset

Presets fill resident sequence, reasoning mode, visible output and the three SLO targets with realistic values for the workload. Everything stays editable afterwards.

Reasoning mode

Thinking tokens generated before the visible answer. They add decode time and, if the toggle below is on, extend the KV footprint too.Light ≈ 2,000 tok · Heavy ≈ 8,000 tok

Reasoning tokens extend the KV cache

On for standard serving (thinking stays in context). Off if the engine discards reasoning traces from the cache after each call.

KV residency policy

Running batch: only admitted sequences hold KV and the rest queue (the modern default). All sessions: every concurrent call keeps its cache resident (lowest latency variance, far more memory).

Concurrency estimator · Little's law

Peak sessions

Turns / session / hr

% turns hitting LLM

LLM calls / turn

Burst factor

Call duration s

→ 0 concurrent calls

Hardware & resilience

GPU

Bandwidth sets decode speed, dense FP16 Tensor-Core TFLOPS sets prefill speed, VRAM sets what fits. 2026 parts (VR200, MI455X, Rubin CPX) carry pre-launch estimates.

Resilience model

Performance is sized on the N load-bearing workers; redundancy adds procurement, not throughput. N+1 survives one node failure. N+N duplicates the system in-site. DR mirrors it to a standby site. N+N + DR, a popular enterprise pattern, runs N+N in each of two active/active sites, surviving a full site loss while both normally share traffic.

Tuning & SLOs

SLO targets

Pass/fail thresholds for the compliance panel. A target of 0 disables that check. Presets refill these with workload-appropriate values.

TTFT ≤ms

TPS ≥tok/s

P95 ≤s

—

Memory ledger

Time to first token

0ms

Per-user speed

0tok/s

Aggregate throughput

0tok/s

Mean user latency

SLO compliance

Deployment topology · workers & GPU utilization

Throughput vs admitted sequencesbatch-aware

per-user tok/s aggregate tok/s memory limit operating point

Per-user speed vs context lengthat current batch

per-user tok/s current context

Request latency anatomy

Reading of this configuration