← 최신 프로덕트

Lightweight Llama3 Inference Engine

CUDA C. Lightweight Llama 3 8B Inference Engine in CUDA C. I recently wanted to explore my newly founded interest in CUDA programming and my passion for machine learning.

오픈소스

52 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

    github.com

    High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

    Hacker News395점

  2. Tiny-vLLM

    github.com

    high performance LLM inference engine in C++ and CUDA

    Hacker News205점

  3. We made our own inference engine for Apple Silicon

    github.com

    A high-performance inference engine for AI models

    Hacker News186점

  4. A fully open-source (Apache 2.0)implementation of llama

    github.com

    Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

    Hacker News158점

  5. I built a tensor library from scratch in C++/CUDA

    github.com

    Tensor library & inference framework for machine learning

    Hacker News119점