← 최신 프로덕트

Serve 100 Large AI models on a single GPU with low impact to TTFT

I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm.

오픈소스

543 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. A Highly Available Distributed Router for Global Realtime AI

    cerebrium.ai

    If you haven’t noticed, GPUs are scarce right now. The global boom in inference has demand growing faster than supply can keep up, and at Cerebrium ...

    Hacker News9점

  2. InfiniteGPU

    github.com

    An open-source AI compute network,now supporting training

    Hacker News5점

  3. I made a YouTube Documentary using only open source models on one GPU [video]

    youtube.com

    I wanted to explore different open source Gen AI tools and brought them together to generate this youtube video: voice, music, image and text.

    Hacker News5점

  4. A tool that turns everyday computers into your own AI cloud

    github.com

    Aggregates compute from spare GPU capacity

    Hacker News5점

  5. Inferbench

    inferbench.com

    collect/share datapoints on GPU's inference performance

    Hacker News7점