← Latest products

Self-host open-source LLMs on AWS with scale-to-zero

Deploy LLMs to production in one command. Puffdeploy CLI is an open-source engine that provisions optimized serverless infrastructure directly in your own AWS/GCP account.

CLISelf-hostedOpen source

6 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. LambdaPi

    github.com

    Serverless Runtime Environment for LLM-Generated Code

    Hacker News6 pts

  2. LLM CLI tool now supports self-hosted language models via plugins

    simonwillison.net

    LLM is my command-line utility and Python library for working with large language models such as GPT-4. I just released version 0.5 with a huge new feature: you can now …

    Hacker News6 pts

  3. I built an open-source answer engine that runs local LLMs

    github.com

    🔍 AI search engine - self-host with local or cloud LLMs

    Hacker News5 pts

  4. Open-source load balancer for llama.cpp

    github.com

    Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

    Hacker News151 pts

  5. A CLI that makes LLMs convenient to use for developers

    github.com

    Command-line interface for the Arcane Engine

    Hacker News26 pts