Rust BPE tokenizer for Qwen models that's 12x faster than HuggingFace

비슷한 프로덕트

설명이 비슷한 순

1. FlashTokenizer

   github.com

   10x faster C++ tokenizer for Python

   HNHacker News5점

2. Rust Based Per-Token Late Interaction Dense Search

   github.com

   High-Performance Engine for Multi-Vector Search

   HNHacker News8점

3. I embedded 685M public texts in 32 minutes (on 8x A100

   github.com

   Rust, TensorRT)

   HNHacker News7점

4. Witchcraft and Pickbrain

   github.com

   fast multi-vector semantic search in Rust

   HNHacker News6점

5. Extractous

   github.com

   Rust based data extraction ~25x faster than unstructured-io

   HNHacker News6점