性能

占用率occupancy

占用率（Occupancy）是指设备上活跃线程束数量与最大活跃线程束数量的比值。

每个周期有4个线程束槽位，共4个时钟周期，因此总共有16（4×4）个线程束槽位，其中15个槽位存在活跃线程束，占用率约为94%。图表灵感来自GTC 2025的 *CUDA Techniques to Maximize Compute and Instruction Throughput* 演讲。

占用率测量有两种类型：

理论占用率（Theoretical Occupancy）：由内核启动配置和设备能力决定的占用率上限。
实际占用率（Achieved Occupancy）：内核执行期间（即活动周期）的实际占用率。

作为 CUDA编程模型的一部分，线程块中的所有线程都被调度到同一个流式多处理器 (SM) 上。每个 SM 都有资源（如共享内存中的空间），这些资源必须在线程块之间进行分配，因此限制了可以在 SM 上调度的线程块数量。

让我们来看一个例子。考虑 NVIDIA H100 GPU，它具有以下规格：

最大线程束/SM：64
最大块/SM：32
（32位）寄存器：65536
共享内存：228 KB

对于一个每线程块 32 个线程、每线程 8 个寄存器和每线程块 12 KB 共享内存的内核，我们最终会受到共享内存的限制：

64 > 1   = warps/block = 32 threads/block ÷ 32 threads/warp
32 < 256 = blocks/register-file = 65,536 registers/register-file ÷ (32 threads/block × 8 registers/thread)
32       = blocks/SM
19       = blocks/smem = 228 KB/smem ÷ 12 KB/block

尽管寄存器文件足够大，可以同时支持 256 个线程块，但共享内存却不够，因此每个 SM 只能运行 19 个线程块，对应 19 个线程束。这种情况很常见，存储在寄存器中的程序中间结果的大小，远小于需要保留在共享内存中的程序工作集元素的大小。

当没有足够的就绪线程束来隐藏指令延迟时，低占用率会损害性能，具体表现为指令发射效率低和流水线利用率不足。然而，一旦占用率足以进行延迟隐藏，进一步增加占用率实际上可能会降低性能。更高的占用率会减少每个线程的资源，可能导致内核在寄存器上出现瓶颈或者降低现代GPU架构旨在利用的算术强度。

更一般地说，占用率衡量的是 GPU 同时处理其最大并行任务的比例，这在大多数内核中并非优化的固有目标。相反，如果是计算受限，我们希望最大化计算资源的利用率；如果是内存受限，我们希望最大化内存资源的利用率。

特别是，在 Hopper 和 Blackwell 架构的 GPU 上，高性能 GEMM 内核通常以个位数的百分比占用率运行，因为它们不需要太多的线程束就能使张量核心完全饱和。

英文原文(Modal GPU Glossary)

Occupancy is the ratio of the active warps to the maximum number of active warps on a device.

talk at GTC 2025.](themed-image://cycles.svg)

There are two types of occupancy measurements:

_Theoretical Occupancy_ represents the upper limit for occupancy due to the
kernel launch configuration and device capabilities.
_Achieved Occupancy_ measures the actual occupancy during
kernel execution, aka on
active cycles.

As part of the CUDA programming model, all the threads in a thread block are scheduled onto the same Streaming Multiprocessor (SM). Each SM has resources (like space in shared memory) that must be partitioned across thread blocks and so limit the number of thread blocks that can be scheduled on the SM.

Let's work through an example. Consider an NVIDIA H100 GPU, which has these specifications:

Maximum warps/SM: 64
Maximum blocks/SM: 32
(32 bit) Registers: 65536
Shared memory (smem): 228 KB

For a kernel using 32 threads per thread block, 8 registers per thread, and 12 KB shared memory per thread block, we end up limited by shared memory:

64 > 1   = warps/block = 32 threads/block ÷ 32 threads/warp
32 < 256 = blocks/register-file = 65,536 registers/register-file ÷ (32 threads/block × 8 registers/thread)
32       = blocks/SM
19       = blocks/smem = 228 KB/smem ÷ 12 KB/block

Even though our register file is big enough to support 256 thread blocks concurrently, our shared memory is not, and so we can only run 19 thread blocks per SM, corresponding to 19 warps. This is the common case where the size of program intermediates stored in registers is much smaller than elements of the program's working set that need to stay in shared memory.

Low occupancy can hurt performance when there aren't enough eligible warps to hide the latency of instructions, which shows up as low instruction issue efficiency and under-utilized pipes. However, once occupancy is sufficient for latency hiding, increasing it further may actually degrade performance. Higher occupancy reduces resources per thread, potentially bottlenecking the kernel on registers or reducing the arithmetic intensity that modern GPU architectures are designed to exploit.

More generally, occupancy measures what fraction of its maximum parallel tasks the GPU is handling simultaneously, which is not inherently a target of optimization in most kernels. Instead, we want to maximize the utilization of compute resources if we are compute-bound or memory resources if we are memory-bound.

In particular, high-performance GEMM kernels on Hopper and Blackwell architecture GPUs often run at single-digit occupancy percentages because they don't need many warps to fully saturate the Tensor Cores.

相关词条