主机软件

CUTLASSCUTLASS

CUDA 线性代数子程序与求解器模板库 (CUDA Templates for Linear Algebra Subroutines and Solvers, CUTLASS) 是一个抽象库，用于在 CUDA 核函数 kernels 中实现高性能线性代数计算。

与 cuBLAS 类似，CUTLASS 的命名参考了基础线性代数子程序 Basic Linear Algebra Subprograms (BLAS) 标准，该标准定义了用于线性代数计算的底层例程。但与 cuBLAS 不同的是，CUTLASS 是一个用于构建核函数的工具包，而不是一个包含现成可调用例程的库。CUTLASS 主要用于 BLAS 层次结构中的第三层：通用矩阵乘法（"GEMMs"）。

正如其名称所示，CUTLASS 包含了一系列 CUDA C++ 模板抽象。模板 Templates 是 C++ 对参数多态性 parametric polymorphism 的实现，您在其他语言中可能以泛型 generics 的形式接触过这一概念。多态函数只需编写一次，便可操作不同输入类型的数据。

现代 CUTLASS 的核心是 CuTe 库，它定义了 Layout 和 Tensor 类型，用于以可组合（Composable）的方式描述和操作由数据与线程构成的张量。请注意，不要将它与 CuTe DSL 混淆，后者是在 Python 中通过领域特定语言（DSL）来封装并提供 CuTe/CUTLASS 模板的功能。

在 CuTe 之上，CUTLASS 提供了一个仅包含头文件的 CUDA C++ 库，该库在三个级别上运行：整个设备（device）、单个核函数 kernel 或 threads 集合（collective，通常为一个 thread block）。在 collective 层中，矩阵与矩阵的乘法通常被分为“主循环 (Mainloops)”和“尾部处理 (Epilogues)”。主循环负责表达核心算法（例如分块策略）；尾部处理则描述后处理步骤，例如应用缩放因子或标量非线性变换（这在神经网络中非常流行）。

CUTLASS 经常被用来编写一些性能极高的核函数，尤其是针对较新 Streaming Multiprocessor architectures 硬件上的矩阵乘法。这些核函数需要对张量核心 Tensor Cores 进行精细的编程，才能达到接近峰值的性能 performance。

CUTLASS 是开源的，可以在 GitHub 上获取。该库还包含了许多使用 CUTLASS 实现的高性能开源核函数，这些函数经常作为其他开源核函数开发时的参考。我们强烈推荐 Colfax International 的 Jay Shah 撰写的热门教程，它们详细解释了如何利用 CUTLASS 的核心组件来实现最大性能。不过需要注意的是，与大多数 C++ 模板元编程一样，CUTLASS 绝非浅尝辄止者所能轻易掌握的！

英文原文(Modal GPU Glossary)

CUDA Templates for Linear Algebra Subroutines and Solvers (CUTLASS) is a library of abstractions for implementing high-performance linear algebra in CUDA kernels.

Like cuBLAS, CUTLASS is named in reference to the Basic Linear Algebra Subprograms (BLAS) standard for low-level routines for linear algebraic computations. Unlike cuBLAS, CUTLASS is a toolkit for constructing kernels, rather than a library of ready-to-call routines. CUTLASS is primarily associated with the third level of the BLAS hierarchy, general matrix multiplications ("GEMMs").

As the name suggests, CUTLASS includes a collection of CUDA C++ template abstractions. Templates are the C++ implementation of parametric polymorphism, which you may have encountered in the form of generics in other languages. Polymorphic functions are written once but can operate on inputs with different types.

The core of modern CUTLASS is the CuTe library, which defines Layout and Tensor types for composably describing and manipulating tensors of data and threads. It is not to be confused with CuTe DSL, which exposes CuTe/CUTLASS templates via a Domain-Specific Language (DSL) in Python.

Atop CuTe, CUTLASS exposes a header-only CUDA C++ library that operates at three levels: the whole device, a single `kernel`, or a collective of threads (typically a thread block). At the collective layer, matrix-matrix multiplications are typically split into "mainloops" and "epilogues". Mainloops express the core algorithm, like tiling strategies. Epilogues describe post-processing steps, like the application of scaling factors or scalar non-linearities (popular in neural networks).

CUTLASS is very commonly used to write some of the highest-performing kernels, especially matrix-matrix multiplications on hardware from more recent Streaming Multiprocessor architectures. These kernels require careful programming of the Tensor Cores to achieve anything like peak performance.

CUTLASS is open source and available on GitHub. The library also includes many implementations of high-performance open-source kernels using CUTLASS, which are regularly used as references elsewhere in open-source kernel development. We can highly recommend the popular tutorials by Jay Shah of Colfax International, which explain in detail how the key components of CUTLASS are used to achieve maximum performance. Note, however, that like most C++ template metaprogramming, CUTLASS is not for the faint of heart!

相关词条