← Latest products

50+ LLMs on 2 GPUs with 2-Second Swapping? We built AI-Native Runtime

InferX: Inference as a Service Platform. We've built InferX, a specialized runtime environment that fundamentally changes how LLMs are served.

Open source

3 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. Costco for LLM Tokens

    inference.net

    Inference infrastructure for AI-native teams. Run, monitor, and optimize production AI workloads with lower cost, faster latency, and dedicated support from Inference.net.

    Hacker News6 pts

  2. LLMKube

    github.com

    Kubernetes for Local LLMs with GPU Acceleration

    Hacker News5 pts

  3. Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload

    github.com

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Hacker News22 pts

  4. Jinfer

    qxotic.ai

    AI inference engine for the JVM. AI in a jar

    Hacker News6 pts

  5. Distributed Llama

    github.com

    Run LLMs on multiple devices in parallel

    Hacker News12 pts