← Latest products

Willow Inference Server

Optimized ASR/TTS/LLM for Willow/WebRTC/REST. Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS. Willow Inference Server (WIS) is a focused and highly optimized language inference server implementation.

Self-hostedOpen source

511 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. Smile-Serve

    github.com

    Inference Server for ML, ONNX, and LLM

    Hacker News5 pts

  2. An educational Local Qwen3 LLM Inference project written in Rust

    github.com

    An educational Rust project for exporting and running inference on Qwen3 LLM family

    Hacker News15 pts

  3. Reame

    github.com

    a CPU inference server that gets faster as it runs

    Hacker News59 pts

  4. Costco for LLM Tokens

    inference.net

    Inference infrastructure for AI-native teams. Run, monitor, and optimize production AI workloads with lower cost, faster latency, and dedicated support from Inference.net.

    Hacker News6 pts

  5. PageIndex for Reasoning-Based RAG

    vectifyai.notion.site

    Large Language Models (LLMs) are powerful tools for language understanding and generation, but they suffer from a fundamental limitation: the context window. Most LLMs can only process a limited number of tokens at a time. For instance, Claude does not allow uploading very long documents in full, which makes tasks like long document analysis or question answering particularly challenging.

    Hacker News6 pts