设备软件

CUDA 编程模型CUDA Programming Model

CUDA 编程模型是一种用于对大规模并行处理器进行编程的编程模型。

CUDA 全称是 _Compute Unified Device Architecture_（统一计算设备架构）。根据上下文的不同，"CUDA" 可以指代多个不同的概念：一种设备硬件架构，或是一种适用于该架构设计的并行编程模型，或是一种扩展高级语言（如 C 语言）以支持该编程模型的软件平台。

CUDA 的愿景在 Lindholm 等人于 2008 年发布的白皮书中有所阐述。我们强烈推荐阅读这份论文，它是 NVIDIA 文档中许多观点、图表乃至特定表述的原始出处。

本文，重点介绍 CUDA _编程模型_。

根据英伟达 CUDA C++ 编程指南， CUDA 编程模型包含三个关键抽象：

**线程组层次结构**。
程序以线程方式执行，但可按嵌套层次引用线程组，从线程块到线程块网格。
**存储器层次结构**。
层次结构中每个级别的线程组都能访问特定的内存资源，以便在组内进行通信。访问存储器层次结构中的最底层时，其速度应几乎与执行指令一样快。
屏障同步。线程组可通过屏障实现执行协调。

线程组和存储器的层次结构及其到设备硬件的映射总结在下图中。

左图：CUDA 编程模型的抽象线程组和存储器层次结构。右图：实现这些抽象概念的匹配硬件。修改自英伟达的 CUDA Refresher: The CUDA Programming Model 和英伟达 CUDA C++ Programming Guide 中的图表。

这三个抽象概念共同鼓励以一种能够随着 GPU 设备并行执行资源的扩展而透明扩展的方式来表达程序。

更具挑战性地说：这种编程模型可防止程序员为 NVIDIA CUDA 架构 GPU 编写的程序在用户购买新NVIDIA GPU 时无法获得加速的程序。

例如，CUDA 程序中的每个线程块都可以进行紧密协调，但块间协调受到限制。这确保线程块能够捕捉程序的可并行化组件，并且可以按任意顺序调度——用计算机体系结构的术语来说，程序员将这种并行性"暴露"给了编译器和硬件。当程序在具有更多调度单元（具体而言是更多流式多处理器 (Streaming Multiprocessor)）的新 GPU 上执行时，更多此类线程块可以并行执行。

一个包含八个线程块的 CUDA 程序在两个流式多处理器 (SM) 的 GPU 上分四个顺序步骤（波次）运行，但在拥有两倍数量 SM 的 GPU 上，只需一半步骤即可完成。修改自 CUDA 编程指南。

CUDA 编程模型的抽象概念通过扩展高级 CPU 编程语言（例如C++ 的 CUDA C++ 扩展）的方式提供给程序员。该编程模型在软件层面通过指令集架构（并行线程执行，即 PTX）和低级汇编语言（流式汇编器，即 SASS）来实现。例如，线程层次结构中的线程块级别通过这些语言中的协作线程数组 (cooperative thread array) 来实现的。

英文原文(Modal GPU Glossary)

The CUDA programming model is a programming model for programming massively parallel processors.

CUDA stands for _Compute Unified Device Architecture_. Depending on the context, "CUDA" can refer to multiple distinct things: a high-level device architecture, a parallel programming model for architectures with that design, or a software platform that extends high-level languages like C to add that programming model.

The vision for CUDA is laid out in the Lindholm et al., 2008 white paper. We highly recommend this paper, which is the original source for many claims, diagrams, and even specific turns of phrase in NVIDIA's documentation.

Here, we focus on the CUDA _programming model_.

Per the NVIDIA CUDA C++ Programming Guide, there are three key abstractions in the CUDA programming model:

**Hierarchy of thread groups**.
Programs are executed in threads but can make reference to groups of threads
in a nested hierarchy, from
blocks to
grids.
**Hierarchy of memories**.
Thread groups at each level of the hierarchy have access to a memory resource
for communication within the group. Accessing the
lowest layer of the memory
hierarchy should be
nearly as fast as executing an instruction.
Barrier synchronization. Thread groups can coordinate execution by means
of barriers.

The hierarchies of execution and memory and their mapping onto device hardware are summarized in the following diagram.

and the NVIDIA CUDA C++ Programming Guide.](themed-image://cuda-programming-model.svg)

Together, these three abstractions encourage the expression of programs in a way that scales transparently as GPU devices scale in their parallel execution resources.

Put provocatively: this programming model prevents programmers from writing programs for NVIDIA's CUDA-architected GPUs that fail to get faster when the program's user buys a new NVIDIA GPU.

For example, each thread block in a CUDA program can coordinate tightly, but coordination between blocks is limited. This ensures blocks capture parallelizable components of the program and can be scheduled in any order — in the terminology of computer architecture, the programmer "exposes" this parallelism to the compiler and hardware. When the program is executed on a new GPU that has more scheduling units (specifically, more Streaming Multiprocessors), more of these blocks can be executed in parallel.

runs in four sequential steps (waves) on a GPU with two SMs but in half as many steps on one with twice as many SMs. Modified from the CUDA Programming Guide.](themed-image://wave-scheduling.svg)

The CUDA programming model abstractions are made available to programmers as extensions to high-level CPU programming languages, like the CUDA C++ extension of C++. The programming model is implemented in software by an instruction set architecture (Parallel Thread eXecution, or PTX) and low-level assembly language (Streaming Assembler, or SASS). For example, the thread block level of the thread hierarchy is implemented via cooperative thread arrays in these languages.

相关词条