← 최신 프로덕트

Distributed Llama

Run LLMs on multiple devices in parallel. Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

오픈소스

3,124 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. An RDMA/Infiniband Distributed Cache for Fast Inference and Training

    github.com

    A high-performance RDMA distributed file system for fast LLM Inference and GPU Training.

    Hacker News13점

  2. Speeding up LLM inference 2x times (possibly)

    asciinema.org

    Here's a project I've been working on for the last few months.

    Hacker News419점

  3. LLM

    github.com

    a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)

    Hacker News45점

  4. A fully open-source (Apache 2.0)implementation of llama

    github.com

    Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

    Hacker News158점

  5. Reame

    github.com

    a CPU inference server that gets faster as it runs

    Hacker News59점