← 최신 프로덕트

TokenDagger

A tokenizer faster than OpenAI's Tiktoken. High-Performance Implementation of OpenAI's TikToken. TokenDagger is a drop-in replacement for OpenAI’s Tiktoken (the tokenizer behind Llama 3, Mistral, GPT-3.*, etc.).

오픈소스

474 스타 · GitHub외부 사이트

github.com 방문외부 사이트

반응 (1)

비슷한 프로덕트

설명이 비슷한 순

  1. LLaMA tokenizer that runs in browser

    github.com

    JS tokenizer for LLaMA 1 and 2

    Hacker News115점

  2. Reducing LLM input tokens by 70%

    adola.app

    Run GPT-OSS, Qwen, Llama, and DeepSeek through one OpenAI-compatible API with streaming and clear usage-based pricing.

    Hacker News56점

  3. tokmon

    github.com

    CLI to monitor your program's OpenAI token costs

    Hacker News11점

  4. A tool to benchmark LLM APIs (OpenAI

    llmapitest.com

    Claude, local/self-hosted)

    Hacker News55점

  5. A fully open-source (Apache 2.0)implementation of llama

    github.com

    Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

    Hacker News158점