AI Training Network Primer v2
AI Training Network PrimerAI 訓練網路入門v2
Second edition第二版

How the networks behind AI training workAI 訓練背後的網路如何運作

For readers with no background in networking or machine learning. Read the thirteen chapters in order: each term is defined where it first appears.寫給沒有網路或機器學習背景的讀者。請依序讀這十三章：每個名詞都在它第一次出現的地方定義。

13 chapters · ≈11 h13 章 · 約 11 小時Start with Chapter 1 →從第 1 章開始 →What one training step computes一個訓練步驟算什麼 53 papers, 2024–202653 篇，2024–2026Research digests →研究導讀 →Recent papers in 11 topics, about 10 minutes each近期論文，分 11 個主題，每篇約 10 分鐘 162 terms162 個名詞Glossary →名詞表 →Every term, linked to where it is defined每個名詞都連到它的定義處

The thirteen chapters十三章

Part I · Foundations第一部 · 基礎

  1. 01What training computes, and why it needs thousands of GPUs訓練在算什麼，又為何需要上千顆 GPU30′
  2. 02The hardware: GPU, server, network card, switch and cable硬體：GPU、伺服器、網路卡、交換器與線纜45′
  3. 03Splitting the model: five parallelisms and the traffic they make切分模型：五種平行化與它們產生的流量60′
  4. 04Collective communication: algorithms, cost and the library that runs them集體通訊：演算法、成本與執行它們的函式庫55′
  5. 05The step-time budget: why the network decides how far training scales步驟時間帳：為何網路決定訓練能擴展到多大45′

Part II · The network第二部 · 網路

  1. 06Topology: how switches are wired into a network拓樸：交換器如何連成一個網路70′
  2. 07Transport: RDMA over Ethernet, lossless networks and congestion control傳輸層：Ethernet 上的 RDMA、無損網路與壅塞控制60′
  3. 08Routing and load balancing: why hashing fails and what replaces it路由與負載平衡：為何雜湊會失效，以及取代它的做法50′
  4. 09The front end: data, checkpoints and control前端網路：資料、檢查點與控制45′
  5. 10Failures and stragglers: the arithmetic of staying up故障與落後者：保持運轉的算術50′
  6. 11Sharing the cluster: isolation, placement and spare capacity共用叢集：隔離、擺放與備用容量45′
  7. 12Models and simulation: predicting step time before the hardware exists模型與模擬：在硬體存在之前預測步驟時間30′

Part III · Putting it together第三部 · 綜合

  1. 13Designs compared, and how to choose設計比較，以及如何選擇50′

Beyond the chapters章節之外

Research digests研究導讀53

One recent paper per page, grouped by topic. Each topic says which chapter to finish first.每頁一篇近期論文，依主題分組。每個主題都標明要先讀完哪一章。

See all 53 digests →看全部 53 篇 →

Guided readings論文導讀9

The production papers this primer is built on, read one at a time.這份入門所依據的生產環境論文，一次讀一篇。