Research digest 14 of 16 · Chapter 4: Collective communication

研究導讀 14 / 16 · 第 4 章：集體通訊

EPIC: Abstraction and Polymorphism of In-Network Collectives on EthernetEPIC：乙太網路上網內集體通訊的抽象化與多型實現

SIGCOMM 2026Yitao Yuan, Jianglong Nie, Tianyu Bai et al., Wenfei Wu (Peking University, USTC, NUDT, ICT CAS, BUPT, Infrawaves, Lenovo Research, Centec, Alibaba Cloud) · paper論文

After reading, you can explain how EPIC lets switches from different vendors add gradients for unmodified RoCE hosts: one tree abstraction, three switch modes, and what an 8-rank testbed measured.讀完後，你能說明 EPIC 如何讓不同廠商的交換器為未修改的 RoCE 主機相加梯度：一個樹狀抽象、三種交換器模式，以及 8 個 rank 的測試平台量到了什麼。

≈ 11 min · 2 interactive panels · 5-question quiz≈ 11 分鐘 · 2 個互動面板 · 5 題測驗Chapter 4 ↗第 4 章 ↗Jump to the quiz ↓跳到測驗 ↓

If you read nothing else

只讀這一段也可以

A switch that adds data in flight cuts the bytes each rank sends and the steps it waits for. Open Ethernet lacks a common form of this, because its parts come from different vendors. EPIC defines one abstraction, the IncTree, for six collectives over unmodified RoCE NICs, and three switch modes built from shared modules. With 8 ranks on one Tofino switch, AllReduce reached 65.3 Gb/s at 1 GB: 1.59× NCCL forced to 256-byte packets (41.1 Gb/s), 1.36× NCCL with its default 1 KB packets (48.0 Gb/s). Tensor-parallel training ran up to 1.38× faster, data-parallel about 1.2%.

會在傳輸途中相加資料的交換器，能減少每個 rank 要送的位元組與要等的步驟。開放的乙太網路缺少共通的做法，因為各部分來自不同廠商。EPIC 定義一個抽象 IncTree，在未修改的 RoCE NIC 上描述六種集體操作，並提供由共用模組組成的三種交換器模式。在一台 Tofino 交換器接 8 個 rank 時，AllReduce 在 1 GB 達到 65.3 Gb/s：是被迫使用 256 位元組封包的 NCCL（41.1 Gb/s）的 1.59 倍，是使用預設 1 KB 封包的 NCCL（48.0 Gb/s）的 1.36 倍。張量平行訓練最多快 1.38 倍，資料平行約快 1.2%。

1. Where this paper sits1. 這篇論文的位置

Section 4.4 ends with in-network reduction: fewer bytes and no chain of dependent steps, at the cost of switch state for each collective. Section 9.5 summarizes EPIC; this page says what each of its headline numbers measures.

第 4.4 節以網內歸約作結：位元組更少、沒有相依步驟串成的鏈，代價是交換器要為每個集體操作保存狀態。第 9.5 節簡介了 EPIC；本頁說明它每個主要數字量的是什麼。

Earlier systems solved parts of the problem: SHARP with its own InfiniBand headers, SwitchML and ATP with a host-software transport (DPDK) that costs CPU cores, NetReduce with a header that the host CPU adds. Most support only AllReduce. EPIC wants one specification that each vendor can implement alone, over standard IP and RoCE.

先前的系統各自只解決一部分：SHARP 使用自己的 InfiniBand 標頭，SwitchML 與 ATP 以主機軟體（DPDK）做傳輸、耗費 CPU 核心，NetReduce 則由主機 CPU 加上標頭。它們多半只支援 AllReduce。EPIC 要的是一份每家廠商都能獨立實作、建立在標準 IP 與 RoCE 上的規格。

A note on the name: Alibaba Cloud's SIGCOMM 2026 paper "Balancing and Beyond" also calls its expert-parallel library EPIC. The names coincide; the two systems are unrelated.

關於名稱：阿里雲的 SIGCOMM 2026 論文〈Balancing and Beyond〉也把它的專家平行函式庫命名為 EPIC。兩者只是名稱相同，系統彼此無關。

2. The problem: bytes, steps and vendors2. 問題：位元組、步驟與廠商

The paper opens with no measurement. Its case is a byte count and a market problem. In a ring all-reduce, each rank sends $2(N-1)/N$ of its data in $2(N-1)$ dependent steps; with a switch that adds, it sends its data once (Figure 1). For large messages, the ideal algorithm throughput $A$ (application bytes over completion time) rises from the ring's value to the switch's:

這篇論文不以量測開場。它的論證是一筆位元組的計算與一個市場問題。在環形 all-reduce 中，每個 rank 在 $2(N-1)$ 個相依步驟中送出資料的 $2(N-1)/N$ ；若交換器會做加法，每個 rank 只送一次資料（圖 1）。對大訊息而言，理想的演算法吞吐 $A$ （應用資料量除以完成時間）從環的值升到交換器的值：

A = \frac{N}{2(N-1)}\,B \;\longrightarrow\; A = B

Here $B$ is the link bandwidth; the ratio is 1.75 at the testbed's $N = 8$ . The market problem: a switch vendor gains nothing until libraries and controllers use its feature, and chips differ. So EPIC keeps the industry's division of work, uses the NIC's RoCE hardware, and lets each vendor pick a design that fits its chip.

$B$ 是鏈路頻寬；在測試平台的 $N = 8$ 時，比值為 1.75。市場問題則是：在函式庫與控制器採用之前，交換器廠商的功能毫無收益，而各家晶片又各不相同。因此 EPIC 維持產業既有的分工、使用 NIC 的 RoCE 硬體，並讓每家廠商選擇適合自家晶片的設計。

Figure 1. Ideal bytes per rank: a ring of four ranks takes

2(N-1) = 6

steps and sends

1.5\,S

per rank; with a switch, each rank sends

S

once. The values come from the ideal-bytes model.圖 1。每個 rank 的理想位元組數：四個 rank 的環需要

2(N-1) = 6

步，每個 rank 送出

1.5\,S

；有交換器時，每個 rank 只送

S

一次。數值來自理想位元組模型。Drawn for this primer from Chapter 4's sections 2 and 4 and Yuan et al., SIGCOMM 2026, App. I.3 (analysis).

Panel 1 adds the speed of the switch's adder and the measured points.

面板 1 加入交換器加法器的速度與實測點。

Panel 1面板 1

Ring or switch: bytes, steps and the adder's speed環或交換器：位元組、步驟與加法器的速度

Ranks

N

rank 數

N

Link bandwidth

B

鏈路頻寬

B

Adder speed, share of

B

加法速度占

B

的比例

Preset預設情境

Bytes each rank sends, ring → switch

每個 rank 送出的位元組，環 → 交換器

Dependent steps, ring → switch

相依步驟，環 → 交換器

Ideal algorithm throughput

理想的演算法吞吐

Switch ÷ ring

交換器 ÷ 環

Adder speed needed to beat the ring (per port)

勝過環所需的加法速度（每埠）

Lines: the ideal model of section 2 for large messages. The ring sends $2(N-1)/N$ of the data per rank, so its algorithm throughput is $NB/(2(N-1))$ . With a switch, each rank sends its data once, at the slower of $B$ and the switch's adder. The switch wins only if its adder runs faster than $N/(2(N-1))$ of $B$ per port, just over half for large $N$ . Diamonds: AllReduce at 1 GB, measured on the paper's testbed (8 ranks, 100 Gb/s ports), shown only when $B = 100$ Gb/s. "NP switch" is the Huawei NetEngine 8000 in Mode-II; "add removed" is the same switch with the payload addition commented out. The NP preset sets the adder to the measured 11.7 Gb/s per rank; treating it as a fixed speed per port for other $N$ is our simplification. Measured values stay below the ideal lines: EPIC's 65.3 Gb/s against 100, NCCL's 48.0 against 57.1. Defaults are the testbed: $N = 8$ , $B = 100$ Gb/s.

線：第 2 節針對大訊息的理想模型。環的每個 rank 送出資料的 $2(N-1)/N$ ，所以演算法吞吐是 $NB/(2(N-1))$ 。有交換器時，每個 rank 只送一次資料，速度取 $B$ 與交換器加法器兩者中較慢的。只有當加法器每埠的速度超過 $B$ 的 $N/(2(N-1))$ （ $N$ 大時略高於一半），交換器才會勝出。菱形：論文測試平台（8 個 rank、100 Gb/s 埠）上 1 GB AllReduce 的實測值，只在 $B = 100$ Gb/s 時顯示。「NP 交換器」是 Mode-II 的華為 NetEngine 8000；「移除加法」是同一台交換器把酬載加法註解掉後的結果。NP 預設把加法器設為實測的每 rank 11.7 Gb/s；在其他 $N$ 下把它當成固定的每埠速度，是我們的簡化。實測值都低於理想線：EPIC 的 65.3 Gb/s 對 100，NCCL 的 48.0 對 57.1。預設值即測試平台： $N = 8$ 、 $B = 100$ Gb/s。

3. The design3. 設計

3.1 One tree for six collectives3.1 一棵樹承載六種集體操作

For each group, a controller (the IncManager) maps an IncTree onto the fabric (Figure 2). Ranks are leaves; switches are inner nodes that add data going up and copy it going down. Each edge has an endpoint at each end, an IP address and a RoCE queue pair (QP), so a switch port looks like an ordinary RoCE peer.

對每個群組，控制器（IncManager）把一棵 IncTree 對應到網路上（圖 2）。rank 是葉；交換器是內部節點，資料上行時相加、下行時複製。每條邊兩端各有一個端點，由 IP 位址與 RoCE 佇列對（QP）組成，所以交換器埠看起來就像普通的 RoCE 對端。

Figure 2. The paper's Tree-3-2 as an IncTree. Dots are endpoints; for AllReduce, data is added going up and copied going down.圖 2。以 IncTree 表示的論文 Tree-3-2。圓點是端點；做 AllReduce 時，資料上行時相加、下行時複製。Drawn for this primer from Yuan et al., SIGCOMM 2026, §3.1, Figures 1–2 and App. A.

AllReduce, Reduce and Broadcast are designed directly; Barrier, ReduceScatter and AllGather are built from them. Hosts run CommLib over standard RDMA verbs; switches run the IncEngine. Before each collective, CommLib sends one RDMA Send-with-Immediate message that names the operation, root and size; switches recognize its opcode, so no custom header is needed.

AllReduce、Reduce 與 Broadcast 是直接設計的；Barrier、ReduceScatter 與 AllGather 由它們組成。主機在標準 RDMA verbs 上執行 CommLib；交換器執行 IncEngine。每次集體操作前，CommLib 送一則 RDMA Send-with-Immediate 訊息，寫明操作、根與大小；交換器辨認它的 opcode，所以不需要自訂標頭。

3.2 Three ways a switch can hold the tree3.2 交換器承載這棵樹的三種方式

The modes differ in where the RoCE connection ends, and so in who repairs a lost packet (Figure 3).

各模式的差別在於 RoCE 連線在哪裡終結，因此也在於由誰修復遺失的封包（圖 3）。

Figure 3. The three modes on one hop: where the connection ends, what the switch keeps, and who repairs a loss.圖 3。三種模式在同一跳上的樣子：連線在哪裡終結、交換器保存什麼、由誰修復遺失。Drawn for this primer from Yuan et al., SIGCOMM 2026, §4.2–§4.4, Algorithms 1–2 and Table 1.

Mode-I, connection terminated. The switch runs a full RoCE stack, like a parameter server: it reassembles whole messages, adds them, and recovers losses hop by hop. The paper puts SHARP-class products here.

Mode-I，終結連線。交換器執行完整的 RoCE 協定堆疊，角色像參數伺服器：它重組整則訊息、相加，並逐跳恢復遺失。論文把 SHARP 類的產品歸在這一類。

Mode-II, connection translated. The switch keeps no transport state; connections run end to end, and the hosts recover losses. All NICs of a group start at the same packet sequence number (PSN), so the $k$ -th packet of every rank has the same PSN. For each upward packet, the switch:

Mode-II，轉譯連線。交換器不保存傳輸狀態；連線端到端相連，由主機恢復遺失。群組中所有 NIC 都從相同的封包序號（PSN）開始，所以每個 rank 的第 $k$ 個封包帶相同的 PSN。對每個上行封包，交換器：

takes slot $i = \mathrm{PSN} \bmod N$ of its payload and degree arrays;取出酬載與 degree 陣列的槽 $i = \mathrm{PSN} \bmod N$ ；
adds the payload and counts it, unless an arrival bit marks a retransmission;加入酬載並計數，除非到達位元顯示這是重傳；
drops the packet until all inputs have arrived, then writes the sum into it;在所有輸入到齊前丟棄封包，到齊後把總和寫進封包；
copies it to each output, rewrites the destination IP and QP, and forwards it.複製到每個輸出，改寫目的 IP 與 QP，然後轉送。

The first-hop switch reflects the receiver's ACKs and NAKs to the sender, so standard RoCE retransmission recovers losses. The arrays have $2MW$ slots (two windows of $W$ messages of $M$ packets); when slot $p$ completes, the switch clears slot $(p+MW) \bmod 2MW$ . This is safe because a rank gets its ACK only after the sum is done, so all ranks stay within one window (Figure 4, top). PSN drift after Reduce and Broadcast forces one group per primitive: $2N+1$ EPIC groups per $N$ ranks.

第一跳交換器把接收端的 ACK 與 NAK 反射回發送端，所以由標準的 RoCE 重傳恢復遺失。陣列有 $2MW$ 個槽（兩個視窗，每個視窗 $W$ 則訊息、每則 $M$ 個封包）；槽 $p$ 完成時，交換器清除槽 $(p+MW) \bmod 2MW$ 。這是安全的，因為 rank 要等總和完成才收到 ACK，所有 rank 因此都在同一個視窗內（圖 4 上）。Reduce 與 Broadcast 之後 PSN 會錯開，迫使每種基本操作各用一個群組：每 $N$ 個 rank 需要 $2N+1$ 個 EPIC 群組。

Mode-III, connection augmented. This mode adds per-hop reliability to per-packet work. A pipe holds $N$ slots with the writable range $[\mathrm{psnStart}, \mathrm{psnStart}+N)$ . The switch ACKs each packet it accepts, NAKs a packet beyond the range, and resends from its own buffer. The range moves when the next hop acknowledges:

Mode-III，增強連線。這個模式在逐封包處理上加入逐跳可靠性。pipe 保存 $N$ 個槽，可寫入的範圍是 $[\mathrm{psnStart}, \mathrm{psnStart}+N)$ 。交換器對接受的封包回 ACK，對超出範圍的封包回 NAK，並從自己的緩衝區重送。下一跳確認之後，範圍才前移：

\mathrm{psnStart} = \min_{e}\, \mathrm{lastAcked}_e + 1

over the pipe's outputs $e$ . Ranks now advance at different speeds, so the first-hop switch sends a fast rank a congestion notification packet (CNP) and DCQCN slows it: rate synchronization. Tofino ran Mode-II, the NP switch Modes II and III; Mode-I ran only in VM emulation.

（ $e$ 為 pipe 的各個輸出）。這時各 rank 前進的速度不同，所以第一跳交換器對跑得快的 rank 送出壅塞通知封包（CNP），由 DCQCN 讓它減速：這就是速率同步。Tofino 執行 Mode-II，NP 交換器執行 Mode-II 與 Mode-III；Mode-I 只在 VM 仿真中執行過。

3.3 What each mode costs3.3 各模式的代價

Mode-I stores and forwards whole messages of $M$ MTUs ( $U$ bytes each) at each of the $2H-3$ switches of a depth- $H$ tree; Modes II and III pipeline packets. With link latency $L$ and data size $S$ (section 5 of the paper):

Mode-I 在深度 $H$ 的樹上，於 $2H-3$ 台交換器各對 $M$ 個 MTU（每個 $U$ 位元組）的整則訊息做儲存轉送；Mode-II 與 Mode-III 則逐封包管線化。設鏈路延遲為 $L$ 、資料量為 $S$ （論文第 5 節）：

\begin{aligned} T_{\text{I}} &= (2H-2)L + (2H-3)\frac{MU}{B} + \frac{S}{B} \\ T_{\text{II/III}} &= (2H-2)L + (2H-3)\frac{U}{B} + \frac{S}{B} \end{aligned}

Buffers follow the owner of reliability: one-hop bandwidth-delay products for per-hop recovery (about $(D+1) \times 2BL$ for Mode-I at node degree $D$ , $4BL$ for Mode-III), the whole path for Mode-II ( $4(H-1)BL$ ). Loss tolerance follows too, with success rate $r$ per hop and throughput in proportion to $r$ :

緩衝跟著可靠性的負責者走：逐跳恢復只需單跳的頻寬延遲乘積（節點的度為 $D$ 時 Mode-I 約 $(D+1) \times 2BL$ ，Mode-III 為 $4BL$ ），Mode-II 則要涵蓋整條路徑（ $4(H-1)BL$ ）。遺失容忍度也一樣；設每跳成功率為 $r$ ，吞吐與 $r$ 成比例：

X_{\text{I/III}} \approx B\, r^{2(H-1)}, \qquad X_{\text{II}} \approx B\, r^{2(H-1)(D-1)}

In Mode-II one loss makes every rank resend that PSN, so losses multiply; in Modes I and III each hop repairs its own. Under PFC, $r \approx 1$ and the modes perform alike. The paper tests loss only in simulation, so this page gives the model alone.

在 Mode-II 中，一次遺失讓每個 rank 重送該 PSN，所以遺失會相乘；在 Mode-I 與 Mode-III 中，每一跳修復自己的遺失。在 PFC 之下 $r \approx 1$ ，各模式表現相近。論文只以模擬測試遺失，所以本頁只給模型。

Panel 2面板 2

Three modes on one tree: buffer, delay, loss同一棵樹上的三種模式：緩衝、延遲、遺失

Tree depth

H

樹的深度

H

Node degree

D

節點的度

D

Message size

M

, in MTUs訊息大小

M

（以 MTU 計）

Loss rate per hop,

1-r

每跳遺失率

1-r

Summation order加總順序

Buffer per switch, I / II / III

每台交換器的緩衝，I / II / III

Mode-I store-and-forward cost

Mode-I 儲存轉送的代價

Throughput under loss, I and III / II

遺失下的吞吐，I 與 III / II

All three quantities are the paper's analysis (its section 5; this page's section 3.3), not measurements. Buffer, in one-hop bandwidth-delay products $2BL$ : Mode-I $D+1$ , Mode-II $2(H-1)$ , Mode-III 2; reproducible summation multiplies Modes II and III by $D+1$ , and Mode-I always pays it. Delay: Mode-I stores whole messages of $M$ MTUs at each of the $2H-3$ switches; we fix $U = 1$ KB and $B = 100$ Gb/s. Loss: every hop has the same success rate $r$ , and throughput falls in proportion to $r$ ; the paper notes that it falls faster when a loss also triggers congestion control. Defaults: one switch ( $H = 2$ ) as on the testbed, $D = 8$ , and $M = 64$ , the 64 KB messages of the Mode-I emulation at a 1 KB MTU. Try $H = 4$ with reproducible order: Mode-II's buffer grows with depth and degree, while Mode-III's stays at one hop.

三個量都是論文的分析（論文第 5 節；本頁第 3.3 節），不是量測。緩衝以單跳頻寬延遲乘積 $2BL$ 為單位：Mode-I 為 $D+1$ ，Mode-II 為 $2(H-1)$ ，Mode-III 為 2；可重現的加總讓 Mode-II 與 Mode-III 乘上 $D+1$ ，Mode-I 則一定要付這個代價。延遲：Mode-I 在 $2H-3$ 台交換器上各儲存一整則 $M$ 個 MTU 的訊息；我們固定 $U = 1$ KB、 $B = 100$ Gb/s。遺失：每一跳的成功率都是 $r$ ，吞吐與 $r$ 成比例下降；論文指出，若遺失也觸發壅塞控制，吞吐會降得更快。預設值：一台交換器（ $H = 2$ ），與測試平台相同； $D = 8$ ； $M = 64$ ，即 Mode-I 仿真在 1 KB MTU 下的 64 KB 訊息。試試 $H = 4$ 並選可重現順序：Mode-II 的緩衝隨深度與度增加，Mode-III 的則維持在一跳。

3.4 Model checking and the bug it found3.4 模型檢查，以及它找到的錯誤

The IncEngine is ten modules over shared per-group state, for routing, transport and payload work. A vendor can start with Mode-II and add the transport modules to reach Mode-III. The paper's module diagram (below) shows which modules each mode composes.

IncEngine 由十個作用於共用群組狀態的模組組成，分別負責轉送、傳輸與酬載處理。廠商可以從 Mode-II 開始，加上傳輸模組而得到 Mode-III。論文的模組圖（下圖）標出每種模式組合了哪些模組。

Three block diagrams of switch modules for Mode-I, Mode-II and Mode-III, with arrows for up data, down data and ACKs. — Figure 4 of the paper: the modules and data flow in the three modes (a design diagram). From (a) to (c) the switch keeps less connection state: Mode-I terminates the transport, Mode-II translates headers so end hosts still own reliability, Mode-III makes each link reliable on its own.論文圖 4：三種模式的模組與資料流（設計示意圖）。從 (a) 到 (c)，交換器保存的連線狀態越來越少：Mode-I 直接終結傳輸層，Mode-II 轉換標頭、讓終端主機仍負責可靠傳輸，Mode-III 則讓每條鏈路各自可靠。Yuan et al., SIGCOMM 2026, Figure 4 (arXiv 2605.18683 HTML, vector figure).

The authors model-checked the designs in TLA+, letting every packet be delivered, lost or reordered. On a Tree-3-2 with three packets per rank and at most one loss at a time, Modes II and III gave correct sums and terminated. Mode-III AllReduce took 146 minutes and 2.24 billion states on a 75-core Xeon.

作者以 TLA+ 對設計做模型檢查，讓每個封包都可能送達、遺失或亂序。在 Tree-3-2、每個 rank 三個封包、任一時刻最多遺失一個的條件下，Mode-II 與 Mode-III 都算出正確的總和並且終止。Mode-III 的 AllReduce 在 75 核心的 Xeon 上花了 146 分鐘、22.4 億個狀態。

The checker caught a real bug (Figure 4). The first Mode-III draft kept Mode-II's recycling rule, but per-hop ACKs let a fast rank run ahead, so clearing slot $p+MW$ could erase its data. The pipe fixed this: a slot is cleared only after its outputs acknowledge it. The checker also found design risks in ATP, SwitchML and NetReduce.

檢查器抓到了一個真正的錯誤（圖 4）。Mode-III 的第一版草稿沿用 Mode-II 的回收規則，但逐跳 ACK 讓快的 rank 跑在前面，所以清除槽 $p+MW$ 可能抹掉它的資料。pipe 修正了這點：槽只有在輸出都確認之後才清除。檢查器也在 ATP、SwitchML 與 NetReduce 中找到設計風險。

Figure 4. Slot recycling with

MW = 4

: safe in Mode-II (top), wrong in a Mode-III draft that let rank A run ahead (middle), and the fix (bottom).圖 4。

MW = 4

時的槽回收：在 Mode-II 中安全（上），在讓 rank A 跑在前面的 Mode-III 草稿中出錯（中），以及修正方式（下）。Drawn for this primer from Yuan et al., SIGCOMM 2026, §4.3, §5.1, Figure 5 and App. F.4.

The controller also lets groups share switch SRAM (see the Chapter 11), for example tensor- and data-parallel groups in turn; this was evaluated only in simulation.

控制器也讓多個群組共用交換器 SRAM（見共用第 4 章），例如張量平行與資料平行群組輪流使用；這只以模擬評估過。

4. Implementation4. 實作

The code is open (github.com/In-Net/EPIC). Tofino runs Mode-II with integers only: floats are scaled by a fixed factor and saturate on overflow, and aggregation payloads are capped at 256 bytes. A Huawei NetEngine 8000 NP switch runs Modes II and III. VM emulation runs all three modes, for interoperability; Modes II and III share 1,557 of Mode-III's 2,741 lines (57%). A Xilinx VU13P FPGA ran at 100 Gb/s line rate: 96 ns per 1 KB packet when forwarding, 244 ns with FP32 aggregation. The often-quoted 53 ns (the paper's Table 1, our chapter) is from cycle-accurate RTL simulation of an Agilex 7 adder core, not a measurement.

程式碼已開源（github.com/In-Net/EPIC）。Tofino 只以整數執行 Mode-II：浮點數以固定比例縮放、溢位時取飽和值，聚合酬載上限為 256 位元組。華為 NetEngine 8000 NP 交換器執行 Mode-II 與 Mode-III。VM 仿真執行三種模式，用來驗證互通性；Mode-II 與 Mode-III 共用 Mode-III 2,741 行中的 1,557 行（57%）。Xilinx VU13P FPGA 以 100 Gb/s 線速運作：1 KB 封包只轉送時 96 ns，做 FP32 聚合時 244 ns。常被引用的 53 ns（論文 Table 1、本站第 4 章）來自 Agilex 7 加法核心的週期精確 RTL 模擬，不是量測。

5. What was measured5. 量到了什麼

All hardware results come from one star: four servers, each with two RTX 3090 GPUs and two 100 GbE NICs, on one switch, for 8 ranks. AllReduce algorithm throughput on Tofino, in Gb/s (the paper's Table 9):

所有硬體結果都來自同一個星狀拓樸：四台伺服器，每台兩顆 RTX 3090 GPU 與兩張 100 GbE NIC，接在一台交換器上，共 8 個 rank。Tofino 上的 AllReduce 演算法吞吐，單位 Gb/s（論文 Table 9）：

System系統	4 KB	1 GB
EPIC-II	5.42	65.3
SwitchML	3.85	65.4
ATP, dedicated serverATP，專用伺服器	1.98	60.4
NCCL Ring, 1 KBNCCL Ring，1 KB	0.015	48.0
NCCL Ring, 256 BNCCL Ring，256 B	0.015	41.1

The headline 1.59× uses NCCL at 256-byte packets, the size Tofino forces on EPIC; against NCCL's default 1 KB, the gain is 1.36×. SwitchML matches EPIC but needs six CPU cores to EPIC's one, because EPIC's NIC runs the transport.

標題數字 1.59 倍用的是 256 位元組封包的 NCCL，那正是 Tofino 強加在 EPIC 身上的大小；與 NCCL 預設的 1 KB 相比，增益是 1.36 倍。SwitchML 與 EPIC 打平，但需要六個 CPU 核心，EPIC 只要一個，因為 EPIC 由 NIC 執行傳輸。

Question問題	Calculation算式	Result結果
EPIC-II vs NCCL Ring, 256 BEPIC-II 對 NCCL Ring，256 B	$65.3 \div 41.1$	$\approx 1.59\times$
EPIC-II vs NCCL Ring, 1 KB (default)EPIC-II 對 NCCL Ring，1 KB（預設）	$65.3 \div 48.0$	$\approx 1.36\times$
Ideal ratio at $N = 8$ $N = 8$ 時的理想比值	$2 \times 7 \div 8$	$1.75\times$ analysis分析值

The paper's Figure 13 (below) shows the whole curve. Read off the plot, EPIC-II is more than ten times faster than NCCL at 1 MB and more than a hundred times faster at 4 KB (note the log axis on the left); from 64 MB up the gap narrows to the ratios in the table.

論文的 Figure 13（下圖）畫出整條曲線。從圖上讀，EPIC-II 在 1 MB 時比 NCCL 快十倍以上，在 4 KB 時快一百倍以上（注意左圖是對數軸）；從 64 MB 起，差距縮小到上表的比值。

Two line charts of AllReduce throughput versus message size for EPIC-II, SwitchML, ATP and NCCL variants. — Figure 13 of the paper: AllReduce throughput on the Tofino testbed (8 ranks behind one switch), measured. In-network aggregation wins most at small and medium messages; at 1 GB, EPIC-II's about 65 Gb/s compares with about 48 Gb/s for NCCL Ring at its default 1 KB packets and about 41 Gb/s when NCCL is forced to 256-byte packets.論文圖 13：Tofino 測試平台（一台交換器下 8 個 rank）上的 AllReduce 吞吐量，實測。網內聚合在中小訊息時優勢最大；1 GB 時 EPIC-II 約 65 Gb/s，NCCL Ring 在預設 1 KB 封包下約 48 Gb/s，被限制成 256 位元組封包時約 41 Gb/s。Yuan et al., SIGCOMM 2026, Figure 13 (arXiv 2605.18683 HTML, vector figure).

At 1 GB, Broadcast beat NCCL (89.0 against 66.6 Gb/s) and Reduce tied. ReduceScatter and AllGather lost (66.3 and 90.6 against 95.6): EPIC runs them as sequences, NCCL as concurrent rings. A Barrier took about 5 µs against about 2 ms. Training iteration time, in seconds (context 2K, batch 256):

在 1 GB 時，Broadcast 勝過 NCCL（89.0 對 66.6 Gb/s），Reduce 打平。ReduceScatter 與 AllGather 落後（66.3 與 90.6 對 95.6）：EPIC 以序列方式執行它們，NCCL 則同時跑多個環。一次 Barrier 約 5 µs，NCCL 約 2 ms。訓練迭代時間，單位秒（上下文 2K、批次 256）：

Model模型	TP = 8 EPIC	TP = 8 NCCL	DP = 8 EPIC	DP = 8 NCCL
GPT-3 Large	28.3	38.1	16.7	16.9
OPT-350m	20.0	27.5	12.2	12.4
Llama-3.2-1B	35.1	46.1	24.5	24.8

Question問題	Calculation算式	Result結果
OPT-350m, TP = 8OPT-350m，TP = 8	$27.5 \div 20.0$	$\approx 1.38\times$
Llama-3.2-1B, TP = 8 (the paper's "31.3%")Llama-3.2-1B，TP = 8（論文的「31.3%」）	$46.1 \div 35.1$	$\approx 1.31\times$ iteration 24% shorter迭代縮短 24%
Llama-3.2-1B, DP = 8Llama-3.2-1B，DP = 8	$24.8 \div 24.5$	$\approx 1.012\times$

Gains follow the share of communication in the iteration (Amdahl's law): TP = 8 across servers communicates far more than DP = 8.

增益跟著通訊在迭代中的占比走（Amdahl 定律）：跨伺服器的 TP = 8 比 DP = 8 的通訊多得多。

The SRAM wall. On the NP switch, AllReduce reached only 11.7 Gb/s (Mode-II) and 12.6 Gb/s (Mode-III). With the payload addition commented out it reached 72.4 Gb/s, close to Broadcast's 72. The cores contend for shared SRAM when they add vectors; future switches need faster SRAM access for summation.

SRAM 之牆。在 NP 交換器上，AllReduce 只有 11.7 Gb/s（Mode-II）與 12.6 Gb/s（Mode-III）。把酬載加法註解掉之後達到 72.4 Gb/s，接近 Broadcast 的 72。各核心相加向量時爭用共用 SRAM；未來的交換器需要更快的 SRAM 存取來做加總。

6. Limits and open questions6. 限制與未解的問題

The paper lists its own limits: no MoE all-to-allv or scale-up fabrics yet, 256-byte payloads on Tofino, sequential ReduceScatter and AllGather, and SRAM-bound NP switches. A careful reader should add:

論文列出自己的限制：尚不支援 MoE 的 all-to-allv 與 scale-up 網路、Tofino 的 256 位元組酬載、序列化的 ReduceScatter 與 AllGather，以及受 SRAM 限制的 NP 交換器。細心的讀者還應補上：

One small switch. All hardware results use 8 ranks at 100 Gb/s on one switch. Mode-III, deeper trees and SRAM sharing rest on emulation and simulation, and at 400–800 Gb/s the adder must be four to eight times faster.一台小交換器。所有硬體結果都是一台交換器上、100 Gb/s 的 8 個 rank。Mode-III、更深的樹與 SRAM 共用都只建立在仿真與模擬上；在 400–800 Gb/s 下，加法器必須快四到八倍。
Integer sums. Tofino adds scaled integers; the claim of no accuracy loss comes without data.整數加總。Tofino 相加的是縮放後的整數；「精度沒有損失」的說法沒有附資料。
An unusual workload. The big gain is TP = 8 across servers; production keeps TP inside NVLink (Llama 3, MegaScale).不常見的工作負載。大增益來自跨伺服器的 TP = 8；生產環境把 TP 留在 NVLink 內（Llama 3、MegaScale）。
A small proof. The model check covered a Tree-3-2 and one loss at a time; Mode-I was not checked.小規模的證明。模型檢查只涵蓋 Tree-3-2 與一次一個遺失；Mode-I 沒有被檢查。
Unreviewed appendices. The NP and FPGA results sit in appendices that the paper marks as not peer-reviewed.未經審查的附錄。NP 與 FPGA 的結果都在附錄，而論文註明附錄未經同儕審查。

7. How it connects7. 與其他內容的關聯

Chapter 7, sections 1, 2 and 4: PSN, go-back-N and DCQCN's CNP carry Modes II and III.第 7 章第 1、2、4 節：PSN、go-back-N 與 DCQCN 的 CNP 支撐著 Mode-II 與 Mode-III。
Guided readings: under PFC (Microsoft RoCE) the modes converge; Mode-III adds per hop the loss tolerance that IRN asks of NICs; Meta's fabric turned off the DCQCN reaction that rate synchronization needs; MegaScale and Llama 3 keep TP in NVLink.論文導讀：在 PFC 之下（Microsoft RoCE）各模式趨於一致；IRN 要求 NIC 具備的遺失容忍，Mode-III 逐跳加入；速率同步需要的 DCQCN 反應，被 Meta 的網路關掉了；MegaScale 與 Llama 3 把 TP 留在 NVLink。
Other digests: Theseus swaps schedules at run time; an in-network path is one more schedule that needs a host fallback.其他導讀：Theseus 在執行期切換排程；網內路徑就是又一個需要主機備援的排程。

Key numbers關鍵數字

Number	數字	Meaning (Tofino testbed unless noted)
65.3 Gb/s	EPIC-II AllReduce at 1 GB, 8 ranks, 100 Gb/s ports	EPIC-II 在 1 GB 的 AllReduce，8 個 rank、100 Gb/s 埠
1.59× · 1.36×	Against NCCL at 256-byte · default 1 KB packets	對 256 位元組 · 預設 1 KB 封包的 NCCL
1.38× · ≈ 1.2%	Best TP = 8 · DP = 8 training speedup	TP = 8 最佳 · DP = 8 的訓練加速
11.7 vs 72.4 Gb/s	NP-switch AllReduce with vs without the addition	NP 交換器 AllReduce，有與沒有加法
96–244 ns	VU13P FPGA per 1 KB packet, at 100 Gb/s	VU13P FPGA 每個 1 KB 封包，100 Gb/s
146 min	Mode-III AllReduce model check, 2.24 billion states	Mode-III AllReduce 模型檢查，22.4 億個狀態

Terms used on this page本頁用到的術語

IncTree: A group's logical tree: ranks are leaves, switches inner nodes, and edges end in ⟨IP, QP⟩ endpoints.; 一個群組的邏輯樹：rank 是葉，交換器是內部節點，邊的兩端是 ⟨IP, QP⟩ 端點。
Mode-I, II, III: The switch ends RoCE connections; only rewrites headers; or adds per-hop ACKs to per-packet work.; 交換器終結 RoCE 連線；只改寫標頭；或在逐封包處理上加入逐跳 ACK。
PSN: Packet sequence number; Mode-II uses it to find each packet's slot.; 封包序號；Mode-II 用它找出每個封包的槽。
Pipe: Mode-III's $N$ slots with a writable PSN range.; Mode-III 的 $N$ 個槽與一段可寫入的 PSN 範圍。
Rate synchronization: A switch-sent CNP that slows a rank running past the pipe's range.; 由交換器送出的 CNP，讓超出 pipe 範圍的 rank 減速。