Guided reading 8 of 9

論文導讀 8 / 9

Congestion Control for Large-Scale RDMA Deployments (DCQCN)Congestion Control for Large-Scale RDMA Deployments（DCQCN）

Yibo Zhu, Haggai Eran, Daniel Firestone, Chuanxiong Guo, Marina Lipshteyn, Yehonatan Liron, Jitendra Padhye, Shachar Raindel, Mohamad Haj Yahia, Ming Zhang · ACM SIGCOMM 2015

≈ 40 min · self-contained digest≈ 40 分鐘 · 不必打開論文也能讀懂Primer chapters: 07對應入門頁章節：07Jump to the quiz ↓跳到測驗 ↓

If you read nothing else

只讀這一段也可以

DCQCN is the congestion-control algorithm that has shipped in nearly every RoCE NIC since 2015. When Meta writes "we turned DCQCN off" or ByteDance writes "we combined Swift and DCQCN", this is the algorithm they mean. Microsoft and Mellanox built it because priority flow control (PFC) alone, while it keeps the network lossless, is unfair and spreads congestion to flows that are not even on the congested path. DCQCN adds per-flow rate control that runs in NIC hardware: switches mark packets with ECN as a queue grows (the congestion point), the receiving NIC sends a congestion notification packet back at most every 50 µs (the notification point), and the sending NIC cuts its rate in proportion to how often marks arrive and then recovers in three phases without ever doing a TCP-style slow start (the reaction point). The paper also derives the one inequality operators must respect: the PFC threshold must exceed the ECN threshold times the number of ports, so that marking happens before pausing. Results: fair sharing, a 3× better tail for incast, PAUSE frames at the spine down from 6 million to 3,000, and 16× more user traffic. The warning for later readers: without PFC, the 10th-percentile incast throughput is zero, because RoCE's go-back-N recovery collapses under loss.

DCQCN 是 2015 年以來幾乎每張 RoCE NIC 都內建的壅塞控制演算法。Meta 寫「我們關掉了 DCQCN」、字節跳動寫「我們結合了 Swift 與 DCQCN」，指的就是它。微軟與 Mellanox 建造它，是因為優先權流量控制（PFC）雖然能讓網路無損，卻不公平，而且會把壅塞擴散到根本不在壅塞路徑上的流。DCQCN 加入在 NIC 硬體中運行的逐流速率控制：交換器在佇列增長時以 ECN 標記封包（壅塞點），接收端 NIC 最多每 50 µs 送回一個壅塞通知封包（通知點），發送端 NIC 依標記抵達的頻率按比例削減速率，再以三個階段恢復，從不做 TCP 式的慢啟動（反應點）。論文也推導出營運者必須遵守的那一條不等式：PFC 門檻必須大於 ECN 門檻乘以埠數，標記才會發生在暫停之前。結果：公平分享、incast 尾端改善 3 倍、spine 的 PAUSE 幀從六百萬降到三千、使用者流量多 16 倍。給後來讀者的警告：沒有 PFC 時 incast 的第 10 百分位吞吐是零，因為 RoCE 的 go-back-N 恢復在丟包下會崩潰。

Reading plan (40 minutes)閱讀計畫（40 分鐘）

§1–§2 state the problem with PFC and why existing algorithms could not be used; §3 is the algorithm, with three roles and a handful of equations; §4 is the buffer analysis, the most practical part; §5–§6 are results. The algorithm looks intimidating on the page but is three rules: mark, notify, react.

§1–§2 陳述 PFC 的問題與為何既有演算法不能用；§3 是演算法，三個角色加幾條方程式；§4 是緩衝分析，最實用的部分；§5–§6 是結果。演算法在紙上看起來嚇人，其實只有三條規則：標記、通知、反應。

1. Why PFC alone is not enough1. 為何 PFC 本身不夠

RoCEv2 needs a lossless fabric and gets it from PFC: a switch whose ingress queue for a priority class passes a threshold pauses the upstream port. PFC works per port and per priority, never per flow, and two problems follow directly from that granularity.

RoCEv2 需要無損網路，由 PFC 提供：某個優先權類別的入口佇列超過門檻的交換器會暫停上游埠。PFC 以埠與優先權為單位運作，永遠不以流為單位，兩個問題直接來自這種粒度。

Unfairness, the parking-lot problem (Figure 3). Four senders H1–H4 write to one receiver on the same priority. H4 arrives at the bottleneck switch on its own ingress port, while H1–H3 share a port. When the bottleneck pauses, it pauses ports, so the three flows sharing one port split that port's share while H4 keeps a whole port. H4 reached up to 20 Gb/s, and its minimum throughput exceeded the maximum of the other three. Victim flows (Figure 4). An unrelated flow from VS to VR, whose path shares no congested link with anyone, dropped from 20 Gb/s to 10 Gb/s because PAUSE frames from an incast elsewhere cascaded back through a switch it happened to traverse, and fell to 4.5 Gb/s when two more incast senders joined. A flow can be hurt by congestion that is not on its path.

不公平，停車場問題（Figure 3）。四個發送端 H1–H4 在同一優先權寫往一個接收端。H4 以自己的入口埠到達瓶頸交換器，H1–H3 則共用一個埠。瓶頸暫停時暫停的是埠，所以共用一埠的三條流分那個埠的份額，H4 獨占一整個埠。H4 達到最高 20 Gb/s，其最低吞吐超過其他三者的最高值。受害者流（Figure 4）。一條從 VS 到 VR、路徑與任何人都不共用壅塞鏈路的無關流，因為別處 incast 的 PAUSE 幀串聯回它碰巧經過的交換器，從 20 Gb/s 掉到 10 Gb/s，再有兩個 incast 發送端加入時掉到 4.5 Gb/s。一條流可能被不在它路徑上的壅塞傷害。

Why not an existing algorithm. Ethernet's own congestion control, QCN, identifies flows by layer-2 addresses, so it cannot work across the IP-routed subnets of a data center, and extending it would require new switch ASICs. TCP-style schemes (DCTCP, iWARP, TCP-Bolt) start slowly, which is wrong for bursty storage traffic that needs full rate immediately; they run in software with CPU cost, and iWARP's end-to-end TCP recovery adds latency. DCQCN is therefore rate-based rather than window-based (simpler to implement in a NIC and finer-grained, which keeps queues shorter), per-flow, and starts at line rate.

為何不用既有演算法。Ethernet 自己的壅塞控制 QCN 以第二層位址識別流，無法跨越資料中心的 IP 路由子網，擴充它又需要新的交換器 ASIC。TCP 類的方案（DCTCP、iWARP、TCP-Bolt）慢啟動，對需要立刻全速的突發儲存流量是錯的；它們在軟體中運行、有 CPU 成本，iWARP 的端到端 TCP 恢復還增加延遲。因此 DCQCN 以速率而非視窗為基礎（在 NIC 中更容易實作、粒度更細，佇列更短）、逐流、並從線速開始。

2. Three roles2. 三個角色

Drawn from §3: the switch marks, the receiver notifies, the sender reacts. Everything at the endpoints runs in NIC hardware.依 §3 繪製：交換器標記、接收端通知、發送端反應。端點上的一切都在 NIC 硬體中運行。Drawn for this primer from Zhu et al., SIGCOMM 2015.

Congestion point (CP): the switch. At each egress queue the switch marks packets with the ECN bit using RED-style probabilistic marking: no marks below a queue length $K_{\min}$ , a probability rising linearly to $P_{\max}$ at $K_{\max}$ , and all packets marked above that. The deployed values are $K_{\min} = 5\,\text{KB}$ , $K_{\max} = 200\,\text{KB}$ and $P_{\max} = 1\%$ . Setting $K_{\min} = K_{\max}$ and $P_{\max} = 1$ would reproduce DCTCP's step marking; DCQCN deliberately marks gently so that a flow seeing a small queue is slowed only a little.

壅塞點（CP）：交換器。在每個出口佇列，交換器以 RED 式的機率標記為封包設 ECN 位元：佇列長度低於 $K_{\min}$ 不標記，機率線性上升到 $K_{\max}$ 時的 $P_{\max}$ ，超過則全部標記。部署值為 $K_{\min} = 5\,\text{KB}$ 、 $K_{\max} = 200\,\text{KB}$ 、 $P_{\max} = 1\%$ 。把 $K_{\min} = K_{\max}$ 且 $P_{\max} = 1$ 就重現 DCTCP 的階梯式標記；DCQCN 刻意溫和地標記，讓看到小佇列的流只被稍微減速。

Notification point (NP): the receiving NIC. When a marked packet arrives for a flow, the NIC sends a congestion notification packet (CNP) back to the sender, but at most one per flow every $N = 50\,\mu\text{s}$ , however many marked packets arrive in that window. The CNP is a tiny packet in a high-priority class, so it is not delayed behind data.

通知點（NP）：接收端 NIC。當某條流收到標記的封包，NIC 向發送端送回一個壅塞通知封包（CNP），但每條流每 $N = 50\,\mu\text{s}$ 最多一個，不論該窗口內到達多少標記封包。CNP 是高優先權類別中的極小封包，不會被資料拖延。

Reaction point (RP): the sending NIC. The sender keeps a current rate $R_C$ , a target rate $R_T$ , and an estimate $\alpha$ of how congested the path is. On receiving a CNP it remembers the current rate as the target, cuts the current rate, and raises $\alpha$ toward 1:

反應點（RP）：發送端 NIC。發送端維護目前速率 $R_C$ 、目標速率 $R_T$ ，以及路徑壅塞程度的估計 $\alpha$ 。收到 CNP 時，它把目前速率記為目標、削減目前速率，並把 $\alpha$ 推向 1：

\begin{aligned} R_T &\leftarrow R_C \\ R_C &\leftarrow R_C\,(1 - \alpha/2) \\ \alpha &\leftarrow (1 - g)\,\alpha + g \end{aligned}

If no CNP arrives within a timer $K = 55\,\mu\text{s}$ (which must exceed the NP's 50 µs), $\alpha$ decays: $\alpha \leftarrow (1 - g)\,\alpha$ . So $\alpha$ tracks how often marks arrive, and a flow that sees only occasional marks is cut only slightly rather than halved. The gain $g$ is 1/256 in the deployed configuration; the QCN and DCTCP value of 1/16 was tested and rejected as too jumpy. Rate increase is driven by a timer (55 µs) and a byte counter (10 MB), whichever fires, in three phases: fast recovery, five steps of $R_C \leftarrow (R_T + R_C)/2$ that climb back toward the remembered target; then additive increase, $R_T \leftarrow R_T + R_{\text{AI}}$ with $R_{\text{AI}} = 40\,\text{Mb/s}$ per step, probing above the old target; then hyper increase, larger steps once the path has clearly been free for a while. There is no slow start: a new flow begins at line rate.

若在計時器 $K = 55\,\mu\text{s}$ （必須大於 NP 的 50 µs）內沒有 CNP， $\alpha$ 衰減： $\alpha \leftarrow (1 - g)\,\alpha$ 。所以 $\alpha$ 追蹤標記抵達的頻率，只偶爾看到標記的流只被稍微削減而非減半。部署設定中增益 $g$ 為 1/256；QCN 與 DCTCP 的 1/16 經測試後因太跳動而被否決。速率增加由計時器（55 µs）與位元組計數器（10 MB）中先到者驅動，分三個階段：快速恢復，五步 $R_C \leftarrow (R_T + R_C)/2$ 爬回記住的目標；接著加性增加， $R_T \leftarrow R_T + R_{\text{AI}}$ ，每步 $R_{\text{AI}} = 40\,\text{Mb/s}$ ，探測舊目標之上；然後超速增加，當路徑明顯空閒一段時間後用更大的步伐。沒有慢啟動：新的流從線速開始。

Why

\alpha/2

and not

1/2

為何是

\alpha/2

而非

1/2

A TCP sender halves on any loss. DCQCN has a richer signal: with probabilistic marking and one CNP per 50 µs, the fraction of windows that carry a CNP measures how deep the queue is. $\alpha$ is a smoothed version of that fraction, so the cut is proportional to the congestion actually observed. The price is a slow estimator: with $g = 1/256$ , $\alpha$ needs hundreds of CNP intervals to converge, which is why sudden incast still relies on PFC as the backstop.

TCP 發送端遇到任何丟包就減半。DCQCN 有更豐富的訊號：機率標記加每 50 µs 一個 CNP，帶有 CNP 的窗口比例量測了佇列有多深。 $\alpha$ 是該比例的平滑版本，所以削減與實際觀察到的壅塞成比例。代價是估計器很慢： $g = 1/256$ 時 $\alpha$ 需要數百個 CNP 間隔才收斂，這就是為什麼突發的 incast 仍依賴 PFC 作為後盾。

3. Making ECN fire before PFC3. 讓 ECN 在 PFC 之前觸發

DCQCN only helps if the switch marks packets before it has to pause; if PFC fires first, the senders never learn to slow down and all the PFC problems return. §4 works out the thresholds on their switch, an Arista 7050QX32 with a Broadcom Trident II chip: 32 ports of 40 Gb/s and a 12 MB shared buffer. Two quantities compete for that buffer. Headroom. After a switch sends PAUSE, packets already on the wire keep arriving, about 22.4 KB per port per priority for a 1,500-byte MTU; that space must be reserved for every lossless class on every port. The PFC threshold $t_{\text{PFC}}$ . This is the ingress queue length at which a port pauses its upstream, and it can be at most what remains of the buffer $B$ after all headroom is reserved: for $n$ ports and 8 priorities,

DCQCN 只有在交換器必須暫停之前就標記封包時才有幫助；若 PFC 先觸發，發送端永遠學不會減速，PFC 的所有問題又回來了。§4 在他們的交換器上算出門檻：Arista 7050QX32，Broadcom Trident II 晶片，32 個 40 Gb/s 埠與 12 MB 共享緩衝。兩個量在爭奪那塊緩衝。Headroom。交換器送出 PAUSE 後，已在線上的封包繼續抵達，以 1,500 位元組 MTU 計算每埠每優先權約 22.4 KB；這些空間必須為每個埠的每個無損類別保留。PFC 門檻 $t_{\text{PFC}}$ 。這是埠暫停其上游時的入口佇列長度，最多只能是保留所有 headroom 後剩下的緩衝 $B$ ： $n$ 個埠、8 個優先權時，

t_{\text{PFC}} \le \frac{B - 8n \cdot t_{\text{flight}}}{8n}

which on their hardware is about 24.47 KB. The safety condition. ECN marking happens at the egress queue, which is fed by up to $n$ ingress ports. If every one of the $n$ ports fills to the ECN threshold $t_{\text{ECN}}$ at once, the egress queue holds $n \cdot t_{\text{ECN}}$ bytes, and no port may yet have paused, so

在他們的硬體上約 24.47 KB。安全條件。ECN 標記發生在出口佇列，它由最多 $n$ 個入口埠餵入。若 $n$ 個埠同時都填到 ECN 門檻 $t_{\text{ECN}}$ ，出口佇列有 $n \cdot t_{\text{ECN}}$ 位元組，而此時不應有任何埠已暫停，所以

t_{\text{PFC}} \gt n \cdot t_{\text{ECN}}

With the dynamic PFC thresholds the switch actually uses ( $\beta = 8$ ), the bound works out to $t_{\text{ECN}} \lt 21.75\,\text{KB}$ . The lesson that carries to the AI papers: the ECN threshold is bounded by buffer size divided by port count. Deep buffers make congestion control easy; shallow buffers with many ports make it nearly impossible.

用交換器實際使用的動態 PFC 門檻（ $\beta = 8$ ），上界算出 $t_{\text{ECN}} \lt 21.75\,\text{KB}$ 。帶到 AI 論文的教訓：ECN 門檻受緩衝大小除以埠數限制。深緩衝讓壅塞控制容易；多埠的淺緩衝讓它幾乎不可能。

4. Results4. 結果

Fairness. With DCQCN, the four parking-lot senders get equal shares with little variance (Figure 8), and the victim flow's throughput no longer drops as incast senders are added (Figure 9). Incast. On a 20-machine testbed, $K$ -to-1 incast for $K$ from 1 to 19 keeps total throughput above 39 Gb/s on a 40 Gb/s link, and the switch queue never exceeds 100 KB. At incast degree 10, the 10th-percentile flow throughput rises from 1.12 Gb/s without DCQCN to 3.43 Gb/s with it, a 3× better tail, because PFC no longer starves the flows on shared ports. PAUSE frames. In the same incast run the two spine switches receive more than 6 million PAUSE frames without DCQCN and about 3,000 with it; congestion is handled at the sender instead of propagating through the fabric. Capacity. In a larger experiment, the network supported 80 communicating pairs with DCQCN where it had supported 5 without degradation before: 16× more user traffic. Latency. The 95th-percentile queue is 76.6 KB with DCQCN versus 162.9 KB with DCTCP-style marking. Two warnings. With misconfigured thresholds (PFC firing before ECN), tail performance is better than PFC alone but worse than correctly configured DCQCN. And with PFC disabled entirely, the 10th-percentile incast throughput is zero: some flows never recover from persistent losses, because go-back-N cannot cope. DCQCN reduces how often PFC fires; in 2015 it does not remove the need for it.

公平性。有 DCQCN 時，停車場的四個發送端得到相等的份額、變異很小（Figure 8），受害者流的吞吐也不再隨 incast 發送端增加而下降（Figure 9）。Incast。在 20 台機器的測試環境中， $K$ 從 1 到 19 的 $K$ 對 1 incast 在 40 Gb/s 鏈路上維持總吞吐高於 39 Gb/s，交換器佇列從未超過 100 KB。incast 度為 10 時，第 10 百分位的流吞吐從沒有 DCQCN 的 1.12 Gb/s 升到有 DCQCN 的 3.43 Gb/s，尾端改善 3 倍，因為 PFC 不再餓死共用埠上的流。PAUSE 幀。同一次 incast 中兩台 spine 交換器在沒有 DCQCN 時收到超過六百萬個 PAUSE 幀，有 DCQCN 時約三千個；壅塞在發送端處理，而非在網路中傳播。容量。更大的實驗中，有 DCQCN 的網路支援 80 對通訊、而之前無劣化只能支援 5 對：使用者流量多 16 倍。延遲。第 95 百分位的佇列在 DCQCN 下是 76.6 KB，DCTCP 式標記下是 162.9 KB。兩個警告。門檻設錯（PFC 先於 ECN 觸發）時，尾端效能比只有 PFC 好、但比正確設定的 DCQCN 差。而完全關閉 PFC 時，incast 的第 10 百分位吞吐是零：有些流永遠無法從持續的丟包恢復，因為 go-back-N 應付不了。DCQCN 減少 PFC 觸發的頻率；在 2015 年它並未消除對 PFC 的需求。

5. Critique and context5. 評論與脈絡

DCQCN's signal is coarse and slow: a probabilistic one-bit mark, summarized by one CNP per 50 µs, smoothed by $g = 1/256$ . That was the right trade for 40 Gb/s links with millisecond-scale storage flows. At 400 Gb/s a 50 µs interval is 2.5 MB of data, and a training burst can fill a switch buffer before the first CNP returns, which is what Meta found: at 200 Gb/s, tightening the thresholds gained 3% and made PFC 2–3× worse, and at 400 Gb/s with new NIC firmware they removed DCQCN and relied on deep buffers plus NCCL's receiver-driven admission, which bounds bytes in flight before congestion forms instead of reacting after. The research line since then has tried to make the signal faster or more exact: Alibaba's HPCC (SIGCOMM 2019) replaces the mark with precise queue sizes carried by in-band telemetry; Google's Swift uses measured RTT; ByteDance's MegaScale combines Swift's RTT with DCQCN's ECN; and the Ultra Ethernet transport and Meta's 2026 design move to receiver-driven credit with multipath spraying. Read DCQCN as the baseline every one of these is measured against, and read §4 as the reason buffer depth appears in every AI-network paper.

DCQCN 的訊號粗糙又慢：一個機率性的單位元標記，由每 50 µs 一個 CNP 總結，再以 $g = 1/256$ 平滑。對 40 Gb/s 鏈路與毫秒級的儲存流，那是正確的取捨。在 400 Gb/s，50 µs 的間隔就是 2.5 MB 的資料，一次訓練突發可以在第一個 CNP 返回之前就塞滿交換器緩衝，Meta 發現的正是如此：在 200 Gb/s 調緊門檻只多 3% 卻讓 PFC 惡化 2–3 倍，在 400 Gb/s 加上新的 NIC 韌體後他們移除 DCQCN，改靠深緩衝與 NCCL 的接收端驅動許可——它在壅塞形成之前就限制在途位元組，而非事後反應。此後的研究路線都在試圖讓訊號更快或更精確：阿里的 HPCC（SIGCOMM 2019）用帶內遙測攜帶的精確佇列大小取代標記；Google 的 Swift 使用量測的 RTT；字節跳動的 MegaScale 結合 Swift 的 RTT 與 DCQCN 的 ECN；Ultra Ethernet 傳輸層與 Meta 2026 年的設計則走向接收端驅動的 credit 加多路徑噴灑。把 DCQCN 當成所有這些方案的比較基線來讀，並把 §4 當成緩衝深度出現在每篇 AI 網路論文中的原因來讀。

Key numbers關鍵數字

Number	數字	Meaning
20 → 10 → 4.5 Gb/s	Victim flow throughput as unrelated incast senders are added (PFC only)	無關的 incast 發送端增加時受害者流的吞吐（僅 PFC）
5 KB / 200 KB / 1%	$K_{\min}$ / $K_{\max}$ / $P_{\max}$ for ECN marking (deployed)	ECN 標記的 $K_{\min}$ / $K_{\max}$ / $P_{\max}$ （部署值）
50 µs / 55 µs	Minimum CNP interval per flow / RP timer (must exceed the CNP interval)	每條流的最小 CNP 間隔 / RP 計時器（必須大於 CNP 間隔）
$1 - \alpha/2$ , $g = 1/256$	Rate cut on a CNP; $\alpha$ 's smoothing gain (1/16 tested and rejected)	收到 CNP 時的速率削減； $\alpha$ 的平滑增益（1/16 經測試後否決）
5 steps / 40 Mb/s / 10 MB	Fast-recovery iterations / additive increase per step / byte counter	快速恢復步數 / 每步加性增加 / 位元組計數器
12 MB / 32 × 40G	Shared buffer and radix of the Trident II switch analyzed in §4	§4 分析的 Trident II 交換器的共享緩衝與埠數
22.4 KB / 24.47 KB	Headroom per port per priority / static upper bound on $t_{\text{PFC}}$	每埠每優先權的 headroom / $t_{\text{PFC}}$ 的靜態上界
$t_{\text{PFC}} \gt n \cdot t_{\text{ECN}}$ ; $t_{\text{ECN}} \lt 21.75\,\text{KB}$	Safety condition so ECN marks before PFC pauses; resulting ECN bound with $\beta = 8$	ECN 先於 PFC 暫停標記的安全條件； $\beta = 8$ 時的 ECN 上界
>39 Gb/s, $K \le 19$	Total incast throughput held on a 40G link; queue under 100 KB	40G 鏈路上維持的 incast 總吞吐；佇列低於 100 KB
1.12 → 3.43 Gb/s	10th-percentile flow throughput at incast 10, without → with DCQCN	incast 10 時第 10 百分位流吞吐，無 → 有 DCQCN
6M → ~3,000	PAUSE frames at the two spines in one incast run	一次 incast 中兩台 spine 的 PAUSE 幀數
5 → 80 pairs (16×)	Communicating pairs supported without degradation	無劣化可支援的通訊對數
0	10th-percentile incast throughput with PFC disabled	關閉 PFC 時 incast 的第 10 百分位吞吐

Terms used in this paper本篇用到的術語

CP / NP / RP: Congestion point (switch), notification point (receiving NIC), reaction point (sending NIC).; 壅塞點（交換器）、通知點（接收端 NIC）、反應點（發送端 NIC）。
ECN: Explicit congestion notification: a bit in the IP header a switch sets to say "this queue is growing" without dropping the packet.; 明確壅塞通知：IP 標頭中交換器設定的位元，表示「這個佇列在增長」而不丟棄封包。
CNP: Congestion notification packet, sent by the receiver NIC to the sender at most once per flow per 50 µs.; 壅塞通知封包，由接收端 NIC 送給發送端，每條流每 50 µs 最多一個。
Parking-lot problem: PFC unfairness in which flows arriving on a shared ingress port get less than a flow with its own port.; PFC 的不公平：從共用入口埠到達的流得到的比獨占一埠的流少。
Victim flow: A flow slowed by PAUSE frames caused by congestion on a path it does not share.; 被不在其路徑上的壅塞所引起的 PAUSE 幀拖慢的流。
Headroom ( $t_{\text{flight}}$ ): Buffer reserved for packets that arrive after a PAUSE is sent, about 22.4 KB per port per priority at 40G.; 為送出 PAUSE 後抵達的封包保留的緩衝，40G 下每埠每優先權約 22.4 KB。
Rate-based control: Limiting a sender's bits per second directly, rather than its bytes in flight (window-based); simpler in a NIC and finer-grained.; 直接限制發送端的每秒位元數，而非其在途位元組（視窗式）；在 NIC 中更簡單、粒度更細。