Professional Documents
Culture Documents
NVIDIA CUDA Programming Guide 0.8.2
NVIDIA CUDA Programming Guide 0.8.2
Programming Guide
Version 0.8.2
4/24/2007
ii
Table of Contents
Chapter 1. Introduction to CUDA....................................................................... 1 1.1 1.2 1.3 2.1 2.2 The Graphics Processor Unit as a Data-Parallel Computing Device ...................1 CUDA: A New Architecture for Computing on the GPU ....................................3 Documents Structure ...................................................................................6 A Highly Multithreaded Coprocessor...............................................................7 Thread Batching...........................................................................................7 Thread Block .........................................................................................7 Grid of Thread Blocks.............................................................................8
Memory Model ........................................................................................... 10 A Set of SIMD Multiprocessors with On-Chip Shared Memory ........................ 13 Execution Model ......................................................................................... 14 An Extension to the C Programming Language ............................................. 17 Language Extensions .................................................................................. 17 Function Type Qualifiers....................................................................... 18 Variable Type Qualifiers ....................................................................... 19 Execution Configuration ....................................................................... 20 Built-in Variables.................................................................................. 21 Compilation with NVCC ........................................................................ 21 Built-in Vector Types............................................................................ 22 Mathematical Functions........................................................................ 22 Time Function ..................................................................................... 23 Texture Type....................................................................................... 23 Mathematical Functions........................................................................ 24
4.2.1 4.2.2 4.2.3 4.2.4 4.2.5 4.3 4.3.1 4.3.2 4.3.3 4.3.4 4.4 4.4.1
iii
Synchronization Function ..................................................................... 25 Type Casting Functions ........................................................................ 25 Texture Functions ................................................................................ 25 Common Concepts............................................................................... 26 Runtime API ........................................................................................ 27 Driver API ........................................................................................... 32
Chapter 5. GeForce 8800 Series and Quadro FX 5600/4600 Technical Specification .................................................................................... 39 5.1 5.2 6.1 General Specification .................................................................................. 39 Floating-Point Standard .............................................................................. 40 Instruction Performance ............................................................................. 43 Instruction Throughput ........................................................................ 43 Memory Bandwidth .............................................................................. 45
Chapter 6. Performance Guidelines ................................................................. 43 6.1.1 6.1.2 6.2 6.3 7.1 7.2 7.3
Number of Threads per Block...................................................................... 55 Data Transfer between Host and Device ...................................................... 56 Overview ................................................................................................... 57 Source Code Listing .................................................................................... 59 Source Code Walkthrough........................................................................... 61 Mul() ................................................................................................ 61 Muld() .............................................................................................. 61
7.3.1 7.3.2
Appendix A. Mathematics Functions................................................................ 63 Appendix B. Runtime API Reference ............................................................... 67 B.1 Device Management ................................................................................... 67 cudaGetDeviceCount() .................................................................. 67 cudaGetDeviceProperties() ........................................................ 67 cudaChooseDevice() ...................................................................... 68 cudaSetDevice() ............................................................................ 68 cudaGetDevice() ............................................................................ 68 cudaMalloc() .................................................................................. 68 B.1.1 B.1.2 B.1.3 B.1.4 B.1.5 B.2 B.2.1
iv
B.2.2 B.2.3 B.2.4 B.2.5 B.2.6 B.2.7 B.2.8 B.2.9 B.2.10 B.2.11 B.2.12 B.2.13 B.2.14 B.2.15 B.2.16 B.2.17 B.2.18 B.2.19 B.3 B.3.1
cudaMalloc2D() .............................................................................. 68 cudaFree() ...................................................................................... 69 cudaMallocArray() ........................................................................ 69 cudaFreeArray() ............................................................................ 69 cudaMemset() .................................................................................. 69 cudaMemset2D() .............................................................................. 69 cudaMemcpy() .................................................................................. 70 cudaMemcpy2D() .............................................................................. 70 cudaMemcpyToArray() .................................................................... 70 cudaMemcpy2DToArray() ................................................................ 70 cudaMemcpyFromArray() ................................................................ 71 cudaMemcpy2DFromArray() ............................................................ 71 cudaMemcpyArrayToArray() .......................................................... 71 cudaMemcpy2DArrayToArray() ...................................................... 71 cudaMemcpyToSymbol() .................................................................. 72 cudaMemcpyFromSymbol() .............................................................. 72 cudaGetSymbolAddress() .............................................................. 72 cudaGetSymbolSize() .................................................................... 73 Low-Level API ..................................................................................... 73 cudaCreateChannelDesc()...................................................... 73 cudaGetChannelDesc()............................................................ 73 cudaGetTextureReference().................................................. 73 cudaBindTexture().................................................................. 73 cudaUnbindTexture().............................................................. 74 cudaBindTexture().................................................................. 74 cudaUnbindTexture().............................................................. 74
B.3.2
High-Level API..................................................................................... 74
B.5
OpenGL Interoperability.............................................................................. 75 cudaGLRegisterBufferObject() .................................................. 75 cudaGLMapBufferObject() ............................................................ 76 cudaGLUnmapBufferObject() ........................................................ 76 cudaGLUnregisterBufferObject() .............................................. 76 cudaD3D9Begin() ............................................................................ 76 cudaD3D9End() ................................................................................ 76 cudaD3D9RegisterVertexBuffer() .............................................. 76 cudaD3D9MapVertexBuffer() ........................................................ 76 cudaD3D9UnmapVertexBuffer() .................................................... 77 cudaGetLastError() ...................................................................... 77 cudaGetErrorString() .................................................................. 77
B.5.1 B.5.2 B.5.3 B.5.4 B.6 B.6.1 B.6.2 B.6.3 B.6.4 B.6.5 B.7 B.7.1 B.7.2 C.1 C.2
Direct3D Interoperability............................................................................. 76
Error Handling............................................................................................ 77
Appendix C. Driver API Reference ................................................................... 79 Initialization ............................................................................................... 79 cuInit() .......................................................................................... 79 cuDeviceGetCount() ...................................................................... 79 cuDeviceGet() ................................................................................ 79 cuDeviceGetName() ........................................................................ 79 cuDeviceTotalMem() ...................................................................... 80 cuDeviceComputeCapability() .................................................... 80 cuCtxCreate() ................................................................................ 80 cuCtxAttach() ................................................................................ 80 cuCtxDetach() ................................................................................ 80 cuModuleLoad() .............................................................................. 80 cuModuleLoadData() ...................................................................... 81 cuModuleUnload() .......................................................................... 81 cuModuleGetFunction() ................................................................ 81
CUDA Programming Guide Version 0.8.2
C.1.1 C.2.1 C.2.2 C.2.3 C.2.4 C.2.5 C.3 C.3.1 C.3.2 C.3.3 C.4 C.4.1 C.4.2 C.4.3 C.4.4
Context Management.................................................................................. 80
vi
C.4.5 C.4.6 C.5 C.5.1 C.5.2 C.5.3 C.5.4 C.5.5 C.5.6 C.5.7 C.5.8 C.5.9 C.6 C.6.1 C.6.2 C.6.3 C.6.4 C.6.5 C.6.6 C.6.7 C.6.8 C.6.9 C.6.10 C.6.11 C.6.12 C.6.13 C.6.14 C.6.15 C.6.16 C.6.17 C.6.18
cuModuleGetGlobal() .................................................................... 81 cuModuleGetTexRef() .................................................................... 81 cuFuncSetBlockShape() ................................................................ 82 cuFuncSetSharedSize() ................................................................ 82 cuParamSetSize() .......................................................................... 82 cuParamSeti() ................................................................................ 82 cuParamSetf() ................................................................................ 82 cuParamSetv() ................................................................................ 82 cuParamSetArray() ........................................................................ 83 cuLaunch() ...................................................................................... 83 cuLaunchGrid() .............................................................................. 83 cuMemAlloc() .................................................................................. 83 cuMemAlloc2D() .............................................................................. 83 cuMemFree() .................................................................................... 84 cuMemAllocSystem() ...................................................................... 84 cuMemFreeSystem() ........................................................................ 84 cuMemGetAddressRange() .............................................................. 84 cuArrayCreate() ............................................................................ 85 cuArrayGetDescriptor() .............................................................. 86 cuArrayDestroy() .......................................................................... 86 cuMemset() ...................................................................................... 86 cuMemcpyStoD() .............................................................................. 86 cuMemcpyDtoS() .............................................................................. 87 cuMemcpyDtoD() .............................................................................. 87 cuMemcpyDtoA() .............................................................................. 87 cuMemcpyAtoD() .............................................................................. 87 cuMemcpyAtoS() .............................................................................. 87 cuMemcpyStoA() .............................................................................. 88 cuMemcpyAtoA() .............................................................................. 88
vii
C.6.19 C.7 C.7.1 C.7.2 C.7.3 C.7.4 C.7.5 C.7.6 C.7.7 C.7.8 C.7.9 C.7.10 C.7.11 C.7.12 C.7.13 C.7.14 C.7.15 C.8 C.8.1 C.8.2 C.8.3 C.8.4 C.8.5 C.9 C.9.1 C.9.2 C.9.3 C.9.4 C.9.5
cuMemcpy2D() .................................................................................. 88 cuModuleGetTexRef() .................................................................... 90 cuTexRefCreate() .......................................................................... 90 cuTexRefDestroy() ........................................................................ 90 cuTexRefSetArray() ...................................................................... 90 cuTexRefSetAddress() .................................................................. 91 cuTexRefSetFormat() .................................................................... 91 cuTexRefSetAddressMode() .......................................................... 91 cuTexRefSetFilterMode() ............................................................ 91 cuTexRefSetFlags() ...................................................................... 92 cuTexRefGetAddress() .................................................................. 92 cuTexRefGetArray() ...................................................................... 92 cuTexRefGetAddressMode() .......................................................... 92 cuTexRefGetFilterMode() ............................................................ 92 cuTexRefGetFormat() .................................................................... 92 cuTexRefGetFlags() ...................................................................... 93 cuGLInit() ...................................................................................... 93 cuGLRegisterBufferObject() ...................................................... 93 cuGLMapBufferObject() ................................................................ 93 cuGLUnmapBufferObject() ............................................................ 93 cuGLUnregisterBufferObject() .................................................. 93 cuD3D9Begin() ................................................................................ 94 cuD3D9End() .................................................................................... 94 cuD3D9RegisterVertexBuffer() .................................................. 94 cuD3D9MapVertexBuffer() ............................................................ 94 cuD3D9UnmapVertexBuffer() ........................................................ 94
OpenGL Interoperability.............................................................................. 93
Direct3D Interoperability............................................................................. 94
viii
List of Figures
Figure 1-1. Figure 1-2. Figure 1-3. Figure 1-4. Figure 1-5. Figure 2-1. Figure 2-2. Figure 3-1. Figure 6-1. Figure 6-2. Figure 6-3. Figure 7-1.
Floating-Point Operations per Second for the CPU and GPU.....................1 The GPU Devotes More Transistors to Data Processing ............................2 Compute Unified Device Architecture Block Diagram ................................3 The Gather and Scatter Memory Operations ............................................4 Shared Memory Brings Data Closer to the ALUs .......................................5 Thread Batching ....................................................................................9 Memory Model..................................................................................... 11 Hardware Model .................................................................................. 14 Examples of Shared Memory Access Patterns Without any Bank Conflict 51 Examples of Shared Memory Access Patterns Without any Bank Conflict 52 Examples of Shared Memory Access Patterns With Bank Conflicts........... 53 Matrix Multiplication ............................................................................. 58
ix
1.1
Figure 1-1.
The main reason behind such an evolution is that the GPU is specialized for compute-intensive, highly parallel computation exactly what graphics rendering is about and therefore is designed such that more transistors are devoted to data processing rather than data caching and flow control, as schematically illustrated by Figure 1-2.
CUDA Programming Guide Version 0.8.2
1
Control
ALU ALU
ALU ALU
Cache
DRAM
DRAM
CPU
GPU
Figure 1-2.
More specifically, the GPU is especially well-suited to address problems that can be expressed as data-parallel computations the same program is executed on many data elements in parallel with high arithmetic intensity the ratio of arithmetic operations to memory operations. Because the same program is executed for each data element, there is a lower requirement for sophisticated flow control; and because it is executed on many data elements and has high arithmetic intensity, the memory access latency can be hidden with calculations instead of big data caches. Data-parallel processing maps data elements to parallel processing threads. Many applications that process large data sets such as arrays can use a data-parallel programming model to speed up the computations. In 3D rendering large sets of pixels and vertices are mapped to parallel threads. Similarly, image and media processing applications such as post-processing of rendered images, video encoding and decoding, image scaling, stereo vision, and pattern recognition can map image blocks and pixels to parallel processing threads. In fact, many algorithms outside the field of image rendering and processing are accelerated by data-parallel processing, from general signal processing or physics simulation to computational finance or computational biology. Up until now, however, accessing all that computational power packed into the GPU and efficiently leveraging it for non-graphics applications remained tricky: The GPU could only be programmed through a graphics API, imposing a high learning curve to the novice and the overhead of an inadequate API to the nongraphics application. The GPU DRAM could be read in a general way GPU programs can gather data elements from any part of DRAM but could not be written in a general way GPU programs cannot scatter information to any part of DRAM , removing a lot of the programming flexibility readily available on the CPU. Some applications were bottlenecked by the DRAM memory bandwidth, underutilizing the GPUs computational power. This document describes a novel hardware and programming model that is a direct answer to these problems and exposes the GPU as a truly generic data-parallel computing device.