OpenGraviton – Run 500B+ parameter models on a consumer Mac Mini

Show HN: OpenGraviton – Run 500B+ parameter models on a consumer Mac Mini

OpenGraviton – Run 500B+ parameter models on a consumer Mac Mini operates in show hn launch.

Profile

Hi HN,I built OpenGraviton, an open-source AI inference engine designed to push the limits of running extremely large models on consumer hardware.The system combines several techniques to drastically reduce memory and compute requirements:• 1.58-bit ternary quantization ({-1, 0, +1}) for ~10x compression • dynamic sparsity with Top-K pruning and MoE routing • mmap-based layer streaming to load weights directly from NVMe SSDs • speculative decoding to improve generation throughputThese allow models far larger than system RAM to run locally.In early benchmarks, OpenGraviton reduced TinyLlama-1.1B from ~2.05GB (FP16) to ~0.24GB using ternary quantization. Synthetic stress tests at the 140B scal

OpenGraviton – Run 500B+ parameter models on a consumer Mac Mini is positioned in show hn launch, a segment that rewards a clear, focused offering.

Its current recorded stage is Show HN launch.

Website: opengraviton.github.io