Tokenflood
simulate arbitrary loads on instruction-tuned LLMs. Tokenflood is a load testing framework for simulating arbitary loads on instruction-tuned LLMs. === Tokenflood is a load testing tool for instruction-tuned LLMs hat can simulate arbitrary LLM loads in terms of prompt, prefix, and output lengths and requests per second.
라이브러리오픈소스
반응 (1)
비슷한 프로덕트
설명이 비슷한 순
Token price calculator for 400+ LLMs
github.comEasy token price estimates for 400+ LLMs. TokenOps.
Can your GPU run this LLM?
github.comCalculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
Lowfat
github.compluggable CLI filter that saved 91.8% of my LLM tokens
LLM
github.coma Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)
Sandboxes for AI agents. Securely run untrusted, LLM-generated code in isolated Firecracker microVM sandboxes that scale from one to thousands — with snapshot, suspend/resume, and durable orchestration.