Measuring the token-count-dependent SM frequency droop in sustained tensor-core GEMM, and reconstructing throughput from the measured clock on Hopper.
Internal status page — current state of the GPU-frequency workshop measurement note
This page is the advisor-facing status snapshot of the GPU-frequency measurement note.
The reviewer's verdict trajectory on the true Pro model was 5.5 → 6.0 → 6.5.
current; switching to select gave the intended gpt-5.5-pro reviewer for the first time.Roofline models and learned predictors in serving simulators share a quiet assumption: the SM clock is a fixed constant, usually the datasheet boost.
The assumption breaks under the very workload these models target: a sustained GEMM saturates the power cap and the controller pulls the clock down, so the realized clock becomes a decreasing function of M.
The contribution is narrow and empirical: measure the realized clock vs M on five GPUs, isolate it as an explicit factor, and show it removes most of the throughput-model error.
The compute-bound efficiency is a product of a run-state clock factor and a clock-independent geometric factor.
The geometric factor transfers; the clock factor is a deployment property set by the power cap and thermal state.
The point estimate is now the per-cell mean throughout; droop is defined once as 1 − mean clock(M=16384) / rated boost.
| GPU | rated boost (MHz) | sustained @16384, mean ± CI | qkv droop | large-M throughput |
|---|---|---|---|---|
| A100-80GB | 1410 | 1128 ± 20.4 | 20.0% | 0% (225 TFLOP/s) |
| H200-SXM | 1980 | 1461 ± 10.2 | 26.2% | −0.5% (659) |
| H100-SXM | 1980 | 1383 ± 8.3 | 30.2% | 0% (676) |
| L40S | 2520 | 1443 ± 33.3 | 42.7% | −15% (196) |
| Max-Q Blackwell RTX PRO 6000 | 2280 (rated) | 1414 ± 16 | 38.0% | −2–4% (o/gate_up/down) |
The L40S is the discriminating case — a 15% throughput decline a constant-clock model cannot produce. Blackwell must always be read against a stated reference (38% vs 2280 rated, 54% vs 3090 silicon max).
On datacenter SKUs the evidence is observational; only the consumer RTX 5090 can be power-controlled, and there the full chain is measured.
The 5090 shows the cap→clock→throughput links but, being well-provisioned, not the cap→clock→decline chain; proving mediation needs a power-controllable tight-cap part.
Each SKU was re-run on additional physical cards (GPU-UUID-confirmed). The droop reproduces across distinct cards with a small card-to-card spread.
| SKU | distinct cards (by UUID) | qkv droop across cards | spread |
|---|---|---|---|
| H100-SXM | 3 ✓ | 28.6% · 28.6% · 29.1% | 0.5 pp |
| H200-SXM | 3 ✓ | 25.5% · 25.8% · 25.8% | 0.3 pp |
| A100-80GB | 2 ✓ | 18.3% · 19.8% | 1.5 pp |
| L40S | 1 ✗ 2nd | 41.8% | same physical L40S returned |
Spread ≤1.5 pp; the headline card sits at the high end (paper quotes the conservative end). Distinct silicon confirmed by UUID. A 2nd distinct L40S could not be obtained — three submissions returned the same physical card.
Supplying the measured clock cuts reconstruction MAPE from 29% to ~10% (37%→3.5% at M≥2048), over the 20 qkv pinned-clock cells. A reconstruction, not a predictor.
Per-SKU: A100 4.2%, H200 9.0%, L40S 9.6%, H100 15.6%. The clock factor is necessary.
H100 still ~14% at high M (cross-instrument residual). Honest as reconstruction-with-residuals, not a closed model.
The remaining objections at 6.5 are inherent limits of a single-cluster study, and they define the realistic ceiling.