Tokenflood
simulate arbitrary loads on instruction-tuned LLMs. Tokenflood is a load testing framework for simulating arbitary loads on instruction-tuned LLMs. === Tokenflood is a load testing tool for instruction-tuned LLMs hat can simulate arbitrary LLM loads in terms of prompt, prefix, and output lengths and requests per second.
LibraryOpen source
Reactions (1)
Similar products
by description similarity
Token price calculator for 400+ LLMs
github.comEasy token price estimates for 400+ LLMs. TokenOps.
Can your GPU run this LLM?
github.comCalculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
Lowfat
github.compluggable CLI filter that saved 91.8% of my LLM tokens
LLM
github.coma Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
Sandboxes for AI agents. Securely run untrusted, LLM-generated code in isolated Firecracker microVM sandboxes that scale from one to thousands — with snapshot, suspend/resume, and durable orchestration.