← 최신 프로덕트

50+ LLMs on 2 GPUs with 2-Second Swapping? We built AI-Native Runtime

InferX: Inference as a Service Platform. We've built InferX, a specialized runtime environment that fundamentally changes how LLMs are served.

오픈소스

3 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. Costco for LLM Tokens

    inference.net

    Inference infrastructure for AI-native teams. Run, monitor, and optimize production AI workloads with lower cost, faster latency, and dedicated support from Inference.net.

    Hacker News6점

  2. LLMKube

    github.com

    Kubernetes for Local LLMs with GPU Acceleration

    Hacker News5점

  3. Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload

    github.com

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Hacker News22점

  4. Jinfer

    qxotic.ai

    AI inference engine for the JVM. AI in a jar

    Hacker News6점

  5. Distributed Llama

    github.com

    Run LLMs on multiple devices in parallel

    Hacker News12점