An LLM response cache that's aware of dynamic data
In this blog post, we discuss how Butter automatically caches templates corresponding to the messages in LLM interactions. Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy.
Reactions (1)
Similar products
by description similarity
Butter
docs.butter.deva muscle memory cache for LLMs
Butter
butter.devA Behavior Cache for LLMs
We're building a caching solution for LLMs (ChatGPT, Claude).
After struggling with complex prompt engineering and unreliable parsing, we built L1M, a simple API that lets you extract structured data from unstructured text and images.
CasCache
github.commulti-generational cache with optimistic concurrency control