← Latest products

Fast NF4 dequantization Triton kernel (1.41x faster than bitsandbytes)

An optimized, high-performance NF4 (NormalFloat 4-bit) dequantization Triton GPU kernel achieving up to 1.41x speedup over bitsandbytes. "I wanted to see if I could optimize the dequantization bottleneck during 4-bit LLM inference.

Open source

6 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. A 6M-token movable window on a single 46GB GPU

    arxiv.org

    Abstract page for arXiv paper 2607.23806: A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever

    Hacker News7 pts

  2. Cuq

    github.com

    Formal Verification of Rust GPU Kernels

    Hacker News94 pts

  3. Taliesin

    medium.com

    bit-exact KV-cache restore, 21x faster, cross-GPU verified

    Hacker News10 pts

  4. Bonsai 1.7B ternary model at 442T/s on M4 Max

    agents2agents.ai

    A drop-in optimized inference build of Bonsai 1.7B for Apple Silicon. ~42% faster decode (tg128) and ~9% faster prefill (pp512) on M-series GPUs. All Metal kernels written and tuned by ata, our autonomous engineering agent.

    Hacker News13 pts

  5. Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload

    github.com

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Hacker News22 pts