Reducing LLM input tokens by 70%
Run GPT-OSS, Qwen, Llama, and DeepSeek through one OpenAI-compatible API with streaming and clear usage-based pricing.
Reactions (1)
Similar products
by description similarity
Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
GPT Router
github.comOpen-Source API Gateway for LLMs (OpenAI, Anthropic, etc.)
This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient LLM GPU selections and cost-effective AI models. LLM provider price comparison, gpu benchmarks to price per token calculation, gpu benchmark table
80% faster
github.com50% less memory, 0% loss of accuracy Llama finetuning
Butter
docs.butter.deva muscle memory cache for LLMs