The Clock Is Not a Constant

Measuring the token-count-dependent SM frequency droop in sustained tensor-core GEMM, and reconstructing throughput from the measured clock on Hopper.

Measurement note Cold Pro-gate: 6.5/10 5 GPUs · n=5 CIs Llama-3.1-8B GEMMs

Internal status page — current state of the GPU-frequency workshop measurement note

0. Where the paper stands today

This page is the advisor-facing status snapshot of the GPU-frequency measurement note.

6.5/10
cold gpt-5.5-pro gate
5 GPUs
H100·H200·A100·L40S·Blackwell
29%→10%
recon MAPE with measured clock
2–3×
distinct cards per SKU

The reviewer's verdict trajectory on the true Pro model was 5.5 → 6.0 → 6.5.

A tooling lesson worth recording. The cold-review wrapper had hardcoded the model strategy to current; switching to select gave the intended gpt-5.5-pro reviewer for the first time.

1. Why this measurement is needed

Roofline models and learned predictors in serving simulators share a quiet assumption: the SM clock is a fixed constant, usually the datasheet boost.

The assumption breaks under the very workload these models target: a sustained GEMM saturates the power cap and the controller pulls the clock down, so the realized clock becomes a decreasing function of M.

The contribution is narrow and empirical: measure the realized clock vs M on five GPUs, isolate it as an explicit factor, and show it removes most of the throughput-model error.

2. The decomposition

The compute-bound efficiency is a product of a run-state clock factor and a clock-independent geometric factor.

(1) $$\eta_c \;=\; f_{\mathrm{clk}}\cdot\eta_{\mathrm{noclk}},\qquad f_{\mathrm{clk}}=\frac{f_{\mathrm{sustained}}}{f_{\mathrm{boost}}}$$

The geometric factor transfers; the clock factor is a deployment property set by the power cap and thermal state.

3. The headline droop — five GPUs, n=5 with CIs

The point estimate is now the per-cell mean throughout; droop is defined once as 1 − mean clock(M=16384) / rated boost.

GPU rated boost (MHz) sustained @16384, mean ± CI qkv droop large-M throughput
A100-80GB14101128 ± 20.420.0%0% (225 TFLOP/s)
H200-SXM19801461 ± 10.226.2%−0.5% (659)
H100-SXM19801383 ± 8.330.2%0% (676)
L40S25201443 ± 33.342.7%−15% (196)
Max-Q Blackwell RTX PRO 60002280 (rated)1414 ± 1638.0%−2–4% (o/gate_up/down)

The L40S is the discriminating case — a 15% throughput decline a constant-clock model cannot produce. Blackwell must always be read against a stated reference (38% vs 2280 rated, 54% vs 3090 silicon max).

4. The controlled causal chain — RTX 5090 only

On datacenter SKUs the evidence is observational; only the consumer RTX 5090 can be power-controlled, and there the full chain is measured.

Cap → clock

Caps 400/450/500/575 W → mean 2400/2554/2647/2731 MHz; r = 0.97–0.99.

Clock → throughput

Achieved throughput tracks the clock with r ≈ 1.00.

Clock-lock control

Locking the clock collapses the 390 MHz M-dependent swing to 15 MHz.

The 5090 shows the cap→clock→throughput links but, being well-provisioned, not the cap→clock→decline chain; proving mediation needs a power-controllable tight-cap part.

5. Cross-card reproducibility — new multi-device evidence partial

Each SKU was re-run on additional physical cards (GPU-UUID-confirmed). The droop reproduces across distinct cards with a small card-to-card spread.

SKU distinct cards (by UUID) qkv droop across cards spread
H100-SXM3 28.6% · 28.6% · 29.1%0.5 pp
H200-SXM3 25.5% · 25.8% · 25.8%0.3 pp
A100-80GB2 18.3% · 19.8%1.5 pp
L40S1 ✗ 2nd41.8%same physical L40S returned

Spread ≤1.5 pp; the headline card sits at the high end (paper quotes the conservative end). Distinct silicon confirmed by UUID. A 2nd distinct L40S could not be obtained — three submissions returned the same physical card.

6. Reconstruction: what the measured clock buys

Supplying the measured clock cuts reconstruction MAPE from 29% to ~10% (37%→3.5% at M≥2048), over the 20 qkv pinned-clock cells. A reconstruction, not a predictor.

What it shows

Per-SKU: A100 4.2%, H200 9.0%, L40S 9.6%, H100 15.6%. The clock factor is necessary.

Where it does not close

H100 still ~14% at high M (cross-instrument residual). Honest as reconstruction-with-residuals, not a closed model.

7. Honest limits — the distance to a clear accept

The remaining objections at 6.5 are inherent limits of a single-cluster study, and they define the realistic ceiling.

Bottom line for the advisor. The result is solid and verified at 6.5/10 on the harshest reviewer; the in-flight cross-card runs answer the single-device objection; the realistic ceiling is the 6.5–7 band, and the gate itself varies by ~1 point run-to-run.