← 최신 프로덕트

Fast NF4 dequantization Triton kernel (1.41x faster than bitsandbytes)

An optimized, high-performance NF4 (NormalFloat 4-bit) dequantization Triton GPU kernel achieving up to 1.41x speedup over bitsandbytes. "I wanted to see if I could optimize the dequantization bottleneck during 4-bit LLM inference.

오픈소스

6 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. A 6M-token movable window on a single 46GB GPU

    arxiv.org

    Abstract page for arXiv paper 2607.23806: A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever

    Hacker News7점

  2. Cuq

    github.com

    Formal Verification of Rust GPU Kernels

    Hacker News94점

  3. Taliesin

    medium.com

    bit-exact KV-cache restore, 21x faster, cross-GPU verified

    Hacker News10점

  4. Bonsai 1.7B ternary model at 442T/s on M4 Max

    agents2agents.ai

    A drop-in optimized inference build of Bonsai 1.7B for Apple Silicon. ~42% faster decode (tg128) and ~9% faster prefill (pp512) on M-series GPUs. All Metal kernels written and tuned by ata, our autonomous engineering agent.

    Hacker News13점

  5. Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload

    github.com

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Hacker News22점