← Latest products

liteLLM Proxy Server

50+ LLM Models, Error Handling, Caching. The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]. TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming What can liteLLM proxy do?

LibraryOpen source

62,465 stars · GitHubexternal site

Visit github.comexternal site

Reactions (2)

Similar products

by description similarity

  1. An open-source AI Gateway with integrated guardrails

    github.com

    A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

    Hacker News37 pts

  2. GPT Router

    github.com

    Open-Source API Gateway for LLMs (OpenAI, Anthropic, etc.)

    Hacker News15 pts

  3. I built an open-source AI data layer that connects any LLM to any data

    github.com

    Chat with your data - with memory, rules, and observability built in. Deploy in 2 minutes

    Hacker News18 pts

  4. Reducing LLM input tokens by 70%

    adola.app

    Run GPT-OSS, Qwen, Llama, and DeepSeek through one OpenAI-compatible API with streaming and clear usage-based pricing.

    Hacker News56 pts

  5. A tool to benchmark LLM APIs (OpenAI

    llmapitest.com

    Claude, local/self-hosted)

    Hacker News55 pts