Blackwell & NVFP4
Blackwell & NVFP4
NVIDIA’s Blackwell (B200 / GB200) microarchitecture represents the state of the art in frontier hardware (2025-2026).
Blackwell Architectural Highlights
tcgen05Tensor Cores: 5th generation Tensor Cores delivering 4x higher FP4 training and inference density compared to Hopper FP8.- Tensor Memory (TMEM): A dedicated ultra-fast on-chip memory structure decoupled from shared memory, specifically optimized for multi-tensor staging and GEMM accumulation.
- NVFP4 Microscopic Scaling: 4-bit floating point format with micro-block scaling factors that preserves accuracy across large foundation models while doubling token throughput.