Roofline Lab: Predict Then Measure

Updated

October 28, 2026

Roofline Lab: Predict Then Measure

Every high-performance kernel is bounded by one of two physical constraints: 1. Compute Bound: The execution units (ALUs, FPUs, Tensor Cores) are saturated executing instructions; the memory interface has idle bandwidth. 2. Memory Bound: The execution units starve waiting for data to arrive from off-chip DRAM (HBM or GDDR); arithmetic throughput is a fraction of hardware peak.

Before writing a single line of CUDA, a kernel engineer must predict which regime the workload inhabits. Writing faster assembly or unrolling loops inside a memory-bound kernel achieves zero speedup. Conversely, optimizing memory tiling for a kernel already compute-bound yields diminishing returns.

1. The Roofline Model Formulation

The Roofline Model, formulated by Williams et al. (Berkeley), visualizes maximum achievable performance as a function of Operational Intensity (also called Arithmetic Intensity).

Definitions

\[\text{Operational Intensity } (I) = \frac{\text{Total Arithmetic Operations (FLOPs)}}{\text{Total Bytes Transferred from DRAM (Bytes)}}\]

\[\text{Attainable Performance } (P) = \min\left(\text{Peak Hardware Compute } (P_{\text{peak}}), \text{Peak Memory Bandwidth } (B) \times I\right)\]

Performance (TFLOP/s)
  ^
  |                           Peak Hardware Compute (P_peak)
  |                       +--------------------------------------- (Compute Bound)
  |                      /
  |                     /
  |                    /  Memory Bandwidth Slope
  |                   /   (P = Bandwidth * I)
  |                  /
  |                 /
  |                /      (Memory Bound)
  |               /
  +--------------+------------------------------------------------>
  0             I* (Ridge Point)         Operational Intensity (FLOPs/Byte)

The Ridge Point (\(I^*\))

The intersection between the memory bandwidth slope and the peak compute roof defines the Ridge Point (\(I^*\)):

\[I^* = \frac{P_{\text{peak}}}{B_{\text{peak}}}\]

  • If a workload’s intensity \(I < I^*\): The workload is strictly memory-bound. Maximum possible throughput is capped at \(B_{\text{peak}} \times I\). To go faster, you must reduce memory traffic (e.g. through kernel fusion or caching).
  • If a workload’s intensity \(I \ge I^*\): The workload is compute-bound. Maximum possible throughput is capped at \(P_{\text{peak}}\). To go faster, you must optimize instruction throughput, eliminate warp divergence, or switch to lower-precision numeric formats (e.g. FP16, BF16, FP8).

2. Hardware Ridge Points Across Generations

Notice how rapidly the Ridge Point has shifted over successive GPU generations. Compute capability has grown far faster than memory bandwidth:

GPU Model FP32 Peak Compute HBM / GDDR Bandwidth Vector Ridge Point (\(I^*_{\text{FP32}}\)) FP16 Tensor Compute Tensor Ridge Point (\(I^*_{\text{Tensor}}\))
Tesla V100 (PCIe) 14.0 TFLOP/s 900 GB/s 15.5 FLOPs/B 112 TFLOP/s 124.4 FLOPs/B
Ampere A100 (SXM4) 19.5 TFLOP/s 2,039 GB/s 9.6 FLOPs/B 312 TFLOP/s 153.0 FLOPs/B
Hopper H100 (SXM5) 67.0 TFLOP/s 3,350 GB/s 20.0 FLOPs/B 989 TFLOP/s 295.2 FLOPs/B
Blackwell B200 90.0 TFLOP/s 8,000 GB/s 11.25 FLOPs/B 2,250 TFLOP/s (FP16) 281.2 FLOPs/B

Key Takeaway: On an NVIDIA H100 running Tensor Core operations, an algorithm must execute nearly 300 floating-point operations for every single byte loaded from HBM before it can even theoretically achieve peak compute saturation.

3. Dissecting Common Machine Learning Operations

Let us calculate the operational intensity for four fundamental deep learning operations executed on an NVIDIA A100 (SXM4: \(B = 2,039 \text{ GB/s}\), \(P_{\text{vector}} = 19.5 \text{ TFLOP/s}\)).

Case 1: Vector Addition (\(y = x + z\))

Given two vectors of length \(N = 10^7\) FP32 elements: - Operations: \(N\) additions = \(10^7\) FLOPs. - Memory Traffic: - Read \(x\): \(10^7 \times 4 \text{ B} = 40 \text{ MB}\). - Read \(z\): \(10^7 \times 4 \text{ B} = 40 \text{ MB}\). - Write \(y\): \(10^7 \times 4 \text{ B} = 40 \text{ MB}\). - Total Memory = \(120 \text{ MB} = 1.2 \times 10^8 \text{ Bytes}\). - Operational Intensity: \[I = \frac{10^7 \text{ FLOPs}}{1.2 \times 10^8 \text{ Bytes}} \approx 0.0833 \text{ FLOPs/Byte}\] - Analysis: \(I = 0.0833 \ll I^* = 9.6\). Strictly memory bound. - Maximum Theoretical Throughput: \[\text{Throughput} = 2,039 \text{ GB/s} \times 0.0833 \text{ FLOPs/Byte} \approx 169.8 \text{ GFLOP/s}\] The GPU achieves less than 0.9% of its FP32 compute capability. The ALU cores spend 99% of their time stalled waiting for memory controllers.

Case 2: LayerNorm / RMSNorm

RMSNorm computes: \(y_i = \frac{x_i}{\sqrt{\frac{1}{d}\sum_{j=1}^d x_j^2 + \epsilon}} \cdot \gamma_i\). For a hidden dimension \(d = 4096\): - Operations per token: - Square: \(d\) FLOPs. - Sum reduction: \(d - 1\) FLOPs. - Mean, add \(\epsilon\), reciprocal square root (rsqrt): \(\sim 4\) FLOPs. - Scale by \(\gamma\) and write: \(2d\) FLOPs. - Total \(\approx 4d \approx 16,384\) FLOPs. - Memory per token (unfused naive PyTorch): - Read input \(x\): \(d \times 4 \text{ B} = 16 \text{ KB}\). - Intermediates written and re-read from global memory: \(\sim 3 \times 16 \text{ KB} = 48 \text{ KB}\). - Write output \(y\): \(16 \text{ KB}\). - Total \(\sim 80 \text{ KB}\). - Operational Intensity: \[I = \frac{16,384 \text{ FLOPs}}{81,920 \text{ Bytes}} \approx 0.20 \text{ FLOPs/Byte}\] - Analysis: Completely memory bound. This is why fused normalization kernels (which keep intermediates in registers and shared memory) deliver massive 5x to 10x speedups.

Case 3: Dense Matrix Multiplication (GEMM)

Compute \(C = A \times B\) where \(A, B, C \in \mathbb{R}^{M \times N}\), with \(M = K = N = 4096\): - Operations: \(2 \times M \times N \times K = 2 \times 4096^3 \approx 137.4 \times 10^9 \text{ FLOPs}\) (137.4 GFLOPs). - Minimum Algorithmic Memory (I/O): - Read \(A\): \(4096^2 \times 4 \text{ B} = 67.1 \text{ MB}\). - Read \(B\): \(4096^2 \times 4 \text{ B} = 67.1 \text{ MB}\). - Write \(C\): \(4096^2 \times 4 \text{ B} = 67.1 \text{ MB}\). - Total \(\approx 201.3 \text{ MB}\). - Operational Intensity (with perfect reuse): \[I_{\text{optimal}} = \frac{1.374 \times 10^{11} \text{ FLOPs}}{2.013 \times 10^8 \text{ Bytes}} \approx 682.6 \text{ FLOPs/Byte}\] - Analysis: \(I = 682.6 > I^* = 9.6\) (and even \(> I^*_{\text{Tensor}} = 153.0\)). Dense GEMM is thoroughly compute-bound when memory reuse is leveraged via shared memory tiling!

4. The Multi-Tier Roofline (Hierarchy of Limits)

Modern GPUs have multiple memory tiers. A kernel that exhausts L2 cache bandwidth or shared memory bandwidth hits an intermediate roofline before reaching DRAM:

Throughput (TFLOP/s)
  ^
  |                                      Tensor Core Peak (989 TFLOP/s)
  |                       +--------------------------------------------
  |                      / (SM SRAM Bandwidth: ~19 TB/s)
  |                     /
  |                    /  (L2 Cache Bandwidth: ~5.2 TB/s)
  |                   /
  |                  /   (HBM3 DRAM Bandwidth: ~3.35 TB/s)
  |                 /
  +----------------+--------------------------------------------------->
  0                                        Operational Intensity (FLOPs/B)
  1. HBM / DRAM Bandwidth: The slowest and largest layer. Optimization goal: load each byte from DRAM exactly once into on-chip memory.
  2. L2 Cache Bandwidth: Over 2x to 3x faster than DRAM. Shared across all SMs.
  3. Shared Memory / L1 SRAM Bandwidth: Over 5x to 8x faster than DRAM (~19 TB/s aggregate on H100). Serves warps inside a single SM.
  4. Register File Bandwidth: Over 30 TB/s aggregate bandwidth. The only memory tier that can feed execution units at full single-cycle latency.

5. Profiling Roofline Performance with NVIDIA Nsight Compute

To verify where a kernel lands on the physical Roofline curve, run NVIDIA Nsight Compute (ncu):

# Profile Speed of Light and Roofline metrics
ncu --set full \
    --section SpeedOfLight \
    --section SpeedOfLight_HierarchicalTensorRooflineChart \
    -o profile_result \
    ./my_cuda_binary

Key NCU Metrics to Inspect:

  • dram__throughput.avg.pct_of_peak_sustained_elapsed: High percentage (>80%) indicates saturation of off-chip DRAM bandwidth (Memory Bound).
  • sm__throughput.avg.pct_of_peak_sustained_elapsed: High percentage (>80%) indicates saturation of SM computation pipelines (Compute Bound).
  • gpu__time_duration.sum: Total kernel wall clock time.

By comparing these two percentages, you instantly know whether to optimize memory access patterns or computational instructions.