性能

性能瓶颈performance bottleneck

瓶子的实际瓶颈会限制液体倒出的速度；系统中类似的性能瓶颈则会限制任务完成的效率。

像这样的屋顶线图用于快速识别吞吐量导向型系统中的性能瓶颈。改编自 Williams, Waterman, and Patterson (2008)。

瓶颈是性能优化的目标。教科书式的优化方法是：

确定瓶颈，
提升瓶颈直到它不再是瓶颈，然后
对新的瓶颈重复此过程。

这种方法已被正式化，例如 Eliyahu Goldratt 的"约束理论" ，该理论帮助将丰田制造方法传播给全球的制造商，进而影响了软件工程和运营领域。

在为 Jane Street 做的这个演讲中，Horace He 将在 GPU 上运行的程序内核所做的工作分解为三类：

计算（在 CUDA 核心或张量核心上运行浮点运算）
内存（在系统的内存层次结构中搬运数据）
开销（其他所有操作）

因此，对于 GPU 内核，性能瓶颈主要分为三类：

计算受限内核，受限于计算单元（如 CUDA 核心或张量核心）的算术带宽，例如大型矩阵-矩阵乘法，
内存受限内核，受限于内存子系统的带宽，例如大型向量-向量乘法，以及
开销受限内核，受限于延迟，例如小型数组操作。

屋顶线模型分析有助于快速确定程序的性能是受限于计算/算术带宽还是内存带宽。

当然，任何资源都可能成为瓶颈。例如，功率输入和热量散发可能会使某些 GPU 的性能低于其理论最大值。参见 NVIDIA 的这篇文章，其中提到通过将 L2 缓存的功率重新分配给流式多处理器 (Streaming Multiprocessor) 实现了 4% 的端到端性能提升；或者参考 Horace He 的这篇文章，指出矩阵乘法性能会因输入数据通过晶体管开关所需的功率量而变化。但计算和内存是最重要的资源，也是最常见的瓶颈。

英文原文(Modal GPU Glossary)

The literal neck of a bottle limits the rate at which liquid can be poured; a metaphorical performance bottleneck in a system limits the rate at which tasks can be completed.

like this one are used to quickly identify performance bottlenecks in throughput-oriented systems. Adapted from Williams, Waterman, and Patterson (2008).](themed-image://roofline-model.svg)

Bottlenecks are the target of performance optimization. The textbook approach to optimization is to

determine the bottleneck,
elevate the bottleneck until it is no longer such, and
repeat on the new bottleneck.

This approach is formalized in, for instance, the "Theory of Constraints" by Eliyahu Goldratt that helped transmit the Toyota approach to manufacturing to manufacturers worldwide, thence to software engineering and operations.

In this talk for Jane Street, Horace He broke down the work done by the kernels of programs run on GPUs into three categories:

Compute (running floating point operations on
CUDA Cores or
Tensor Cores)
Memory (moving data in the system's
memory hierarchy)
Overhead (everything else)

And so for GPU kernels, performance bottlenecks fall into three main\* categories:

compute-bound
kernels, bottlenecked by the
arithmetic bandwidth of compute
units, like large matrix-matrix multiplication,
memory-bound
kernels, bottlenecked by the
bandwidth of memory subsystems, like
large vector-vector multiplication, and
overhead-bound
kernels bottlenecked by latency, like
small array operations.

Roofline model analysis helps quickly identify whether a program's performance is bottlenecked by compute/arithmetic bandwidth or memory bandwidth.

<small>Of course, _any_ resource can become a bottleneck. For instance, power ingress and heat egress can and do bottleneck some GPUs below their theoretical maximum performance. See this article from NVIDIA explaining a 4% end-to-end performance improvement by redirecting power from the L2 cache to the Streaming Multiprocessors or this article from Horace He indicating that matrix multiplication performance varies depending on the input data via the amount of power demanded by transistor switching. But compute and memory are the most important resources and the most common bottlenecks.</small>

相关词条