LLM-Generated Kernels: The Agent + Profiler Loop

LLM-Generated Kernels: The Agent + Profiler Loop

We are entering a transformative era where AI models generate, optimize, and verify GPU kernels autonomously.

The Agentic Optimization Loop

[Prompt / Specification]
        │
        ▼
┌──────────────┐      Correctness Check       ┌──────────────────┐
│   LLM Agent  │ ───> [Compile & Test] ────>  │  Passes Tests?   │
└──────────────┘                              └─────────┬────────┘
        ▲                                               │ Yes
        │ NCU Metric Feedback                           ▼
        └───────────────────────────────────  ┌──────────────────┐
                                              │ Profiler (NCU)   │
                                              │ Measure TFLOP/s  │
                                              └──────────────────┘

Google DeepMind AlphaEvolve & KernelBench

• KernelBench: An automated evaluation benchmark testing whether language models can generate numerically correct and high-performance CUDA/Triton kernels across realistic neural network primitives.
• DeepMind AlphaEvolve: An evolutionary coding agent combining LLM code mutations with real hardware profiler feedback. AlphaEvolve discovered non-trivial memory layout schedules that sped up production attention kernels by ~32%.

The Role of the Human Kernel Engineer

The engineer’s job is not becoming obsolete; it is elevating: - Defining invariants, numerical bounds, and correctness testbenches. - Providing architectural guidance and identifying hardware bottlenecks. - Interpreting profiler telemetry to steer the agent’s exploration away from local minima.