We Beat MLPerf

Show HN: We Beat MLPerf: Modern Storage for KV Offload and LLM Training

We Beat MLPerf is a company in show hn launch.

Profile

Hi HN,We are the developers of OpenLake, an open source storage engine for KV cache offloading to remote disk and memory.Once we offloaded to local disk, we realized the bottleneck is the PCIe or NIC bandwidth. We wondered whether on GPU lossless compression is viable for fast reads and lower TTFT.BF16 is usually very hard to compress, (high entropy of sign/mantissa). What surprised us is that real world KV blocks are very different. The exponent byte has a very low entropy and barely populated. Instead of compressing the whole tensor, we compress only the exponent stream on the GPU.We see the following results: (H100, production KV snapshot):- 1.51× lossless compression - 622 GB/s median GP

The company sits in the show hn launch segment, where it competes on its core product or service.

The company is listed at the Show HN launch stage of development.

Website: www.theopenlake.com/blog/openlake-leads-mlperf-storage-v3-0