Butter, a muscle memory cache for LLMs

Show HN: Butter, a muscle memory cache for LLMs

Based, Butter, a muscle memory cache for LLMs works in show hn launch.

Profile

Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit.Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out: https://blog.butter.dev/muscle-mem-as-a-proxyThe proxy’s killer feature is being template-aware, meaning it can reuse cache entries across structurally similar r

As a business in show hn launch, Butter, a muscle memory cache for LLMs is listed here with the public details available about it.

Its current recorded stage is Show HN launch.

Website: docs.butter.dev