性能

记分板scoreboard stall

当某条指令由于依赖先前指令的结果而无法发射（Issue）时，就会发生记分板停顿 (Scoreboard Stall)。

记分板 (Scoreboard) 是一种硬件结构，用于追踪哪些寄存器正在等待被 “在途（正在执行中）” 的指令写入数据。当线程束 (warp)处于停顿状态时，它无法继续向前推进。

记分板停顿通常可以分为两类：短记分板停顿 (Short Scoreboard Stalls) 和长记分板停顿 (Long Scoreboard Stalls)。

短记分板停顿 (Short Scoreboard Stall) 当某条指令正在等待一个未离开流式多处理器 Streaming Multiprocessor (SM) 的可变延迟指令的结果时，就会发生短记分板停顿。例如： * 在特殊功能单元 (Special Function Unit) 上执行的慢速数学指令（如 MUFU.EX2 和 MUFU.SQRT）。 * 在张量核心 (Tensor Core) 上执行的矩阵乘法（如 MMA）。 * 共享内存 (Shared Memory) 操作（如 LDS 和 STS）。

长记分板停顿 (Long Scoreboard Stall) 当某条指令正在等待一个需要离开 Streaming Multiprocessor (SM) 的内存操作结果时，就会发生长记分板停顿。例如： * 全局内存加载 (LDG) 或存储 (STG)。 * 长记分板停顿通常在内存受限 (Memory-Bound) 的代码中占据主导地位。

一个线程束 (warp) 拥有 6 个记分板，编译器利用它们来追踪指令之间的数据依赖关系。

部分记分板信息可以在流式汇编（流式汇编器 (SASS)）中读取。例如，以下是使用带有 --dump-sass 标志的 cuobjdump 工具时可能看到的内容：

[barrier:  :  :  :  ]  /*line*/  INSTRUCTION Ri, [Rj] ; # Format: scoreboard info, line number, instruction, operands
[B------:R-:W2:-:S04]  /*00f0*/  LDG.E.SYS R0, [R2] ;   # 设置记分板 2
[B------:R-:W2:-:S01]  /*0100*/  LDG.E.SYS R5, [R4] ;   # `ptxas` 智能地复用了记分板 2 
...
[B--2---:R-:W-:Y:S08]  /*0150*/  IMAD R0, R0, c[0x0][0x160], R5 ;  # 等待记分板 2

我们可以看到，这里的 IMAD 指令在记分板 2 上有一个栅栏同步/屏障（B--2---），这表明它需要该位标志（Bit Flag）被清除后才能发射。两条 LDG 指令在发射时都会递增记分板 2（W2 写入），从而确保 IMAD 指令在执行前，寄存器 R0 和 R5 中已经写入了正确的值。

一条指令可能会受到多个记分板的屏障限制，例如 B01--4- 意味着必须等待记分板 0、1、4 全部被清除。当数据依赖关系得到满足时，对应的记分板就会递减。

记分板的复用可能会导致 Nsight Compute 的停顿分类不够准确。如果长记分板停顿和短记分板停顿使用了同一个记分板，它们可能会被混淆。

记分板是一种在动态指令调度中用于跟踪依赖关系的技术，其历史可以追溯到 “第一台超级计算机” —— Control Data Corporation 6600，其中一台机器曾在 1966 年推翻了欧拉猜想（欧拉幂和猜想）。

与 CPU 不同的是，GPU 中的记分板并不用于线程内部的乱序执行（指令级并行，ILP），而仅用于线程之间（线程级并行，TLP）；具体可参见相关的 NVIDIA 专利。

欲了解更多关于 GPU 记分板实现的详细信息，请参阅 Matthew D. Sinclair 教授的课程讲义 (Slides)。

英文原文(Modal GPU Glossary)

A scoreboard stall occurs when an instruction cannot be issued due to a dependency on the result of a prior instruction.

A scoreboard is a hardware structure that tracks which registers are waiting to be written to by an in-flight instruction. A warp cannot progress when it is in the stalled state.

Scoreboard stalls can be classified into two types: short scoreboard stalls and long scoreboard stalls.

A short scoreboard stall occurs when an instruction is waiting on the result of a variable latency instruction which does not leave the Streaming Multiprocessor (SM). Most prominently, this includes shared memory operations like LDS and STS. It also includes certain Special Function Unit (SFU) operations for reasons that are somewhat obscure, but related to their integration with the same Memory Input/Output hardware as shared memory accesses.

A long scoreboard stall occurs when an instruction is waiting on the result of a memory operation which may leave the SM, such as global memory loads (LDG) or stores (STG). Long scoreboard stalls dominate memory-bound code.

Some scoreboard information is legible in Streaming Assembler (SASS). For instance, you can see that each warp has six distinct barrier identifiers which the compiler uses to track data dependencies between instructions.

Concretely, below is what you might see from a cuobjdump with the --dump-sass flag:

[barrier:  :  :  :  ]  /*line*/  INSTRUCTION Ri, [Rj] ; # Format: scoreboard info, line number, instruction, operands
[B------:R-:W2:-:S04]  /*00f0*/  LDG.E.SYS R0, [R2] ;   # Sets scoreboard 2
[B------:R-:W2:-:S01]  /*0100*/  LDG.E.SYS R5, [R4] ;   # `ptxas` intelligently reuses scoreboard 2
...
[B--2---:R-:W-:Y:S08]  /*0150*/  IMAD R0, R0, c[0x0][0x160], R5 ;  # Waits on scoreboard 2

We can see that our IMAD instruction has a barrier (B--2---) on scoreboard 2, indicating that it requires that bit flag to be cleared before it can issue. Both LDG instructions increment (W2 write) scoreboard 2 when they are issued so that our IMAD instruction will have the correct values in registers R0 and R5 before it executes.

There may be multiple scoreboards to barrier, such as B01--4- which means wait until scoreboards 0,1,4 are all cleared. When the data dependency has been satisfied, the respective scoreboard is decremented.

Scoreboard reuse can mean that the stall classification from Nsight Compute is incorrect, as a long and short scoreboard stall may be conflated if they use the same scoreboard.

Scoreboarding for dependency tracking in dynamic instruction scheduling dates back to the "first supercomputer", the Control Data Corporation 6600, one of which disproved Euler's sum of powers conjecture in 1966. Unlike in CPUs, scoreboarding in GPUs isn't used for out-of-order execution within threads (instruction-level parallelism), only across them (thread-level parallelism); see this NVIDIA patent.

For more details about scoreboard implementation on GPUs, see Professor Matthew D. Sinclair's slides.

相关词条