性能

峰值速率peak rate

峰值速率是指硬件系统能够完成工作的理论最大速率。

峰值速率代表了当所有执行单元都以最高效率满负荷运行时，GPU 性能的绝对上限。它假设在理想运行状态下，即没有任何资源限制（如寄存器、内存带宽、同步屏障等）造成性能瓶颈。

峰值速率是衡量所有实际性能的基准。它在屋顶线模型中设定了计算受限的 "屋顶"。它是在流水线利用率指标中利用率分数的分母，也是 GPU 利用率的最终评判标准。

富有诗意的是，NVIDIA 工程师通常称之为"光速"——这是由物理定律所决定的程序速度极限。

峰值速率是直接根据每个 GPU 架构的固定硬件规格计算得出的。

例如，一个具有 132 个流式多处理器 (SM) 的 NVIDIA H100 GPU，每个 SM 包含 128 个 FP32 核心，每个核心可以发出 1 个单精度融合乘加 (FMA) 操作，该操作包含 2 个浮点运算。这相当于每时钟周期可执行 33,792 条指令。当使用 FP32 核心时，H100 可以使其计算子系统时钟以最高 1980 MHz（每秒百万时钟周期）的速率运行，因此峰值速率为 66,908 亿 FLOPS，即 66.9 TFLOPS。

这与 NVIDIA H100 白皮书中宣传的峰值 FP32 TFLOPS（非 Tensor）速率完全吻合。

英文原文(Modal GPU Glossary)

Peak rate is the theoretical maximum rate at which a hardware system can complete work.

Peak rate represents the absolute upper bound of GPU performance when every execution unit operates at maximum capacity with perfect efficiency. It assumes ideal operation, where no resource constraints (registers, memory bandwidth, synchronization barriers, etc.) create bottlenecks.

Peak rate is the yardstick against which all achieved performance is measured. It sets the compute-bound "roof" in a roofline analysis. It is the denominator in the utilization fraction reported in pipe utilization metrics and the ultimate arbiter of GPU utilization.

Poetically, NVIDIA engineers often call it the "speed of light" — the limit on program speed imposed by physics.

Peak rate is computed directly from the fixed hardware specifications of each GPU architecture.

For example, an NVIDIA H100 GPU with 132 SMs, each containing 128 FP32 cores, can issue 1 single precision Fused Multiply Add (FMA) operation, which comprises 2 floating point operations per core. That's 33,792 instructions per clock. The H100 can operate its compute subsystem clock at a maximum rate of 1980 MHz (million clocks per second) when using the FP32 cores, and so the peak rate is 66,908 billion FLOPS, or 66.9 TFLOPS.

This precisely matches the Peak FP32 TFLOPS (non-Tensor) rate advertised in NVIDIA's H100 whitepaper.

相关词条