设备硬件

线程束调度器Warp Scheduler

流式多处理器 (SM) 的 Warp Scheduler 会在每个时钟周期决定要执行哪一组线程。

H100 流式多处理器（SM）的内部架构图。其中，Warp 调度器和分发单元以橙色显示。改编自 NVIDIA 的 H100 白皮书。

这些被称为线程束线程组按每个时钟周期（约 1 纳秒）进行切换——这与 CPU 中同时多线程（“超线程”）的细粒度线程级并行性类似，但规模要大得多。线程束调度器能够在指令操作数就绪时快速切换大量并发任务，这是 GPU 具备延迟隐藏特性的关键。

CPU 完整线程上下文切换需要数百到数千个时钟周期（更接近微秒级而非纳秒级），因为需要保存一个线程的上下文并恢复另一个线程的上下文。此外，CPU 上的上下文切换会降低局部性，通过增加缓存未命中率进一步影响性能（参见 Mogul and Borg, 1991）。

由于每个线程都拥有从 SM 的寄存器文件分配的私有寄存器能力，GPU 的上下文切换无需任何数据移动来保存或恢复上下文。

而且由于 GPU 的 L1 缓存可完全由程序员管理，并在调度到同一 SM 上的线程束之间共享（参见协作线程数组），因此 GPU 的上下文切换对缓存命中率的影响要小得多。有关 GPU 中程序员管理缓存与硬件管理缓存交互的详细信息，请参阅《CUDA C 编程指南》的"最大化内存吞吐量"章节。

线程束调度器还负责管理线程束的执行状态。

英文原文(Modal GPU Glossary)

The Warp Scheduler of the Streaming Multiprocessor (SM) decides which group of threads to execute on each clock cycle.

.](themed-image://gh100-sm.svg)

These groups of threads, known as warps, are switched out on a per clock cycle basis — roughly one nanosecond - much like the fine-grained thread-level parallelism of simultaneous multi-threading ("hyper-threading") in CPUs, but at a much larger scale. The ability of the Warp Schedulers to switch rapidly between a large number of concurrent tasks as soon as their instructions' operands are available is key to the latency hiding capabilities of GPUs.

Full CPU thread context switches take a few hundred to a few thousand clock cycles (more like a microsecond than a nanosecond) due to the need to save the context of one thread and restore the context of another. Additionally, context switches on CPUs lead to reduced locality, further reducing performance by increasing cache miss rates (see Mogul and Borg, 1991).

Because each thread has its own private registers allocated from the register file of the SM, context switches on the GPU do not require any data movement to save or restore contexts.

And because the L1 caches on GPUs can be entirely programmer-managed and are shared between the warps scheduled together onto an SM (see cooperative thread array), context switches on the GPU have much less impact on cache hit rates. For details on the interaction between programmer-managed caches and hardware-managed caches in GPUs, see the "Maximize Memory Throughput" section of the CUDA C Programming Guide.

The Warp Schedulers also manage the execution state of warps.

相关词条