Crusoe Capstone Specifications & Problem Tracks

Crusoe Capstone Specifications & Problem Tracks

The capstone project requires building a production-ready GPU kernel addressing genuine infrastructure challenges.

Problem Track 1: Fused Sparse Attention

Implement a fused attention kernel supporting block-sparse patterns (e.g. BigBird or sliding window + global tokens). The kernel must: - Skip computation of zero-weight blocks entirely. - Maintain online softmax correctness across sparse blocks. - Outperform standard dense PyTorch attention by at least 2x on sequences >16k.

Problem Track 2: Quantized FP4/FP8 GEMM with Dynamic Dequantization

Build a high-throughput matrix multiply kernel that ingests FP8 or INT4 quantized weights and dequantizes on-the-fly inside SM registers before execution.

Problem Track 3: High-Throughput Speculative Decoding Verification Kernel

Design a batched verification kernel for speculative decoding that verifies \(K\) draft tokens concurrently against a target model distribution in a single fused pass.