← 최신 프로덕트

Example POC to run the Dolly V2 12B model in 8bit mode

This repo loads the Dolly V2 12b model (databricks/dolly-v2-12b) using the `transformers` library. The code loads it in 8-bit quantized mode. I banged this out today because nobody seems to have figured this out yet (and I wanted to play with this model).

라이브러리오픈소스

26 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. OpenGraviton

    opengraviton.github.io

    Run 500B+ parameter models on a consumer Mac Mini

    Hacker News13점

  2. Lance

    github.com

    Alternative to Parquet for ML data

    Hacker News85점

  3. Run 500B+ Parameter LLMs Locally on a Mac Mini

    github.com

    By combining 1.58-bit ternary quantization, dynamic sparsity with Top-K pruning and MoE routing, and mmap-based layer streaming, OpenGraviton can run models far larger than your system RAM—even on a Mac Mini.

    Hacker News17점

  4. RISCY-V02

    github.com

    A 16-bit 2-cycle RISC-V-ish CPU in the 6502 footprint

    Hacker News7점

  5. I ran a language model on a PS2

    github.com

    Running a large language model on a PlayStation 2

    Hacker News46점