Where a hand-written GPU kernel beats the library, and where it can't

Show HN: Where a hand-written GPU kernel beats the library, and where it can't

Where a hand-written GPU kernel beats the library, and where it can't is a business in show hn launch.

Profile

I've been building streaming inference kernels for Zonos, the open SSM-based (Mamba2) TTS model, on an A10G. This post is two hand-written CuTeDSL kernels in two regimes, and the contrast is the point.The first is a skinny bf16 GEMV for the batch-1 decode projections. It beats cuBLAS by up to 1.36x, not because it's clever, but because cuBLAS pads the tiny M dimension out to a tensor-core GEMM tile and leaves memory bandwidth on the floor at exactly that shape. It's memory-bound, so I measured the actual achievable read bandwidth (520 GB/s, not the 600 GB/s spec) and tuned to that roofline.The second is a from-scratch port of Mamba2's four-stage SSD chunked scan. It loses to Tri Dao's Triton

Its activity falls under show hn launch; the profile below is assembled from public directory data.

Its current recorded stage is Show HN launch.

Website: kyrieblunders.bearblog.dev/the-kernels-that-were-there