What are Registers?

At the lowest level of the memory hierarchy are the registers, which store information manipulated by a single thread .

The values in registers are generally stored in the register file of the Streaming Multiprocessor (SM) , but they can also spill to the global memory in the GPU RAM at a substantial performance penalty.

As when programming CPUs, these registers are not directly manipulated by high-level languages like CUDA C . They are only visible to a lower-level language, here Parallel Thread Execution (PTX) . They are typically managed by a compiler like ptxas. Among the compiler's goals is to limit the register space used by each thread so that more thread blocks can be simultaneously scheduled into a single SM , increasing occupancy .

The registers used in the PTX instruction set architecture are documented here . The registers used in SASS are not, to our knowledge, documented.