Verification, Profiling, and Benchmarking Harness

Updated

October 28, 2026

Verification, Profiling, and Benchmarking Harness

Every kernel submission must pass a three-tiered validation suite:

1. Correctness Verification

Compare kernel output tensors against double-precision PyTorch / FP64 reference outputs across edge-case dimensions, non-multiple-of-32 matrix sizes, and extreme numerical scales (\(10^{-5}\) to \(10^5\)). Maximum allowable relative error:

\[\frac{\|Y_{\text{kernel}} - Y_{\text{ref}}\|_2}{\|Y_{\text{ref}}\|_2} < 10^{-3}\]

2. CUDA Event Benchmarking

Measure kernel execution time using cudaEventRecord over 1,000 warm iterations, discarding outliers:

cudaEvent_t start, stop;
cudaEventCreate(&start);
cudaEventCreate(&stop);

cudaEventRecord(start);
for (int i = 0; i < 1000; ++i) {
    launch_kernel<<<grid, block>>>(...);
}
cudaEventRecord(stop);
cudaEventSynchronize(stop);

float ms = 0;
cudaEventElapsedTime(&ms, start, stop);
printf("Average execution time: %f ms\n", ms / 1000.0f);