← Trending

Tangerine Index lists products launched since ChatGPT and people's reactions to them.

tiktoken

github.com

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Open source

Visit github.comexternal siteGitHub19,616 starsexternal site

Reactions (1)

Similar products

by description similarity

  1. tiktokenizer

    Online playground for OpenAPI tokenizers

    github.comOpen sourcelatest reaction

    Hacker News
    No reaction
    Product Hunt
    No reaction
    GitHub
    1,719 stars

  2. Tiktokenizer

    Track your users' AI token usage through an API

    tiktokenizer.devlatest reaction

    Hacker News
    No reaction
    Product Hunt
    61 votes
    GitHub
    No reaction

  3. TokenDagger

    A tokenizer faster than OpenAI's Tiktoken

    github.comOpen source2 reactionslatest reaction

    Hacker News
    281 pts
    Product Hunt
    No reaction
    GitHub
    474 stars

  4. I made an open-source alternative to LangChain

    Open Source Backend for AI

    itz.amlatest reaction

    Hacker News
    5 pts
    Product Hunt
    No reaction
    GitHub
    No reaction

  5. Rust BPE tokenizer for Qwen models that's 12x faster than HuggingFace

    A blazing-fast BPE tokenizer for Qwen models built with Rust

    github.comOpen source2 reactionslatest reaction

    Hacker News
    5 pts
    Product Hunt
    No reaction
    GitHub
    29 stars