← Latest products

Serve 100 Large AI models on a single GPU with low impact to TTFT

I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm.

Open source

543 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. A Highly Available Distributed Router for Global Realtime AI

    cerebrium.ai

    If you haven’t noticed, GPUs are scarce right now. The global boom in inference has demand growing faster than supply can keep up, and at Cerebrium ...

    Hacker News9 pts

  2. InfiniteGPU

    github.com

    An open-source AI compute network,now supporting training

    Hacker News5 pts

  3. I made a YouTube Documentary using only open source models on one GPU [video]

    youtube.com

    I wanted to explore different open source Gen AI tools and brought them together to generate this youtube video: voice, music, image and text.

    Hacker News5 pts

  4. A tool that turns everyday computers into your own AI cloud

    github.com

    Aggregates compute from spare GPU capacity

    Hacker News5 pts

  5. Inferbench

    inferbench.com

    collect/share datapoints on GPU's inference performance

    Hacker News7 pts