性能

利特尔法则Little's Law

利特尔法则（Little's Law）确定了通过吞吐量完全隐藏延迟 (hide latency) 所需的并发量。

并发量 (操作数) = 延迟 (秒) * 吞吐量 (操作数/秒)

在 Lazowska等人所著的经典定量系统教科书中，利特尔法则描述为分析领域 "最重要的基本定律" 之一。

利特尔法则决定了 GPU 需要有多少指令处于 "执行中" 状态，才能通过线程束调度器 (warp schedulers) 进行线程束 (warp) 切换（也称为细粒度线程级并行，类似于 CPU 中的同步多线程）来隐藏延迟 (hide latency) 。

如果某个 GPU 的峰值吞吐量为每周期 1 条指令，内存访问延迟为 400 个周期，那么程序中所有活跃线程束 (active warps) 需要 400 个并发内存操作。如果吞吐量增加到每周期 10 条指令，那么程序需要 4000 个并发内存操作才能充分利用这种增长。更多细节请参阅关于延迟隐藏 (latency hiding) 的文章。

对于利特尔法则的一个重要应用，请考虑 Vasily Volkov博士论文第4.3节中关于延迟隐藏 (latency hiding) 的观察结果：隐藏纯内存访问延迟所需的线程束数量并不比隐藏纯算术延迟所需的线程束数量多太多（在他的实验中是30 vs 24）。直观上看，内存访问的较长延迟似乎需要更多的并发。但并发量不仅由延迟决定，还受吞吐量影响。由于内存带宽 (memory bandwidth) 远低于算术带宽 (arithmetic bandwidth)，所需的并发量结果大致相同——这对于面向延迟隐藏 (latency hiding) 且需要混合算术和内存操作的系统而言，是一种有用的平衡形式。

英文原文(Modal GPU Glossary)

Little's Law establishes the amount of concurrency required to fully hide latency with throughput.

concurrency (ops) = latency (s) * throughput (ops/s)

Little's Law is described as "the most important of the fundamental laws" of analysis in the classic quantitative systems textbook by Lazowska and others.

Little's Law determines how many instructions must be "in flight" for GPUs to hide latency through warp switching by warp schedulers (aka fine-grained thread-level parallelism, like simultaneous multi-threading in CPUs).

If a GPU has a peak throughput of 1 instruction per cycle and a memory access latency of 400 cycles, then 400 concurrent memory operations are needed across all active warps in a program. If the throughput goes up to 10 instructions per cycle, then the program needs 4000 concurrent memory operations to properly take advantage of the increase. For more detail, see the article on latency hiding.

For a non-trivial application of Little's Law, consider the following observation, from Section 4.3 of Vasily Volkov's PhD thesis on latency hiding: the number of warps required to hide pure memory access latency is not much higher than that required to hide pure arithmetic latency (30 vs 24, in his experiment). Intuitively, the longer latency of memory accesses would seem to require more concurrency. But the concurrency is determined not just by latency but also by throughput. And because memory bandwidth is so much lower than arithmetic bandwidth, the required concurrency turns out to be roughly the same — a useful form of balance for a latency hiding-oriented system that will mix arithmetic and memory operations.

相关词条