← Latest products

TokenDagger

A tokenizer faster than OpenAI's Tiktoken. High-Performance Implementation of OpenAI's TikToken. TokenDagger is a drop-in replacement for OpenAI’s Tiktoken (the tokenizer behind Llama 3, Mistral, GPT-3.*, etc.).

Open source

474 stars · GitHubexternal site

Visit github.comexternal site

Reactions (1)

Similar products

by description similarity

  1. LLaMA tokenizer that runs in browser

    github.com

    JS tokenizer for LLaMA 1 and 2

    Hacker News115 pts

  2. FlashTokenizer

    github.com

    10x faster C++ tokenizer for Python

    Hacker News5 pts

  3. Reducing LLM input tokens by 70%

    adola.app

    Run GPT-OSS, Qwen, Llama, and DeepSeek through one OpenAI-compatible API with streaming and clear usage-based pricing.

    Hacker News56 pts

  4. OnPanda

    onpanda.diyer22.com

    Steer LLMs and agents at the token level

    Hacker News5 pts

  5. tokmon

    github.com

    CLI to monitor your program's OpenAI token costs

    Hacker News11 pts