`tc` like `wc` but for LLM tokens
counts tokens! like wc, but for tokens! Little tool that I made to understand how (un)reasonable my prompts are.
Open source
Reactions (1)
Similar products
by description similarity
Tokensift
github.coman open-sourced token-efficiency linter for LLM prompts
A CLI tool for managing and comparing LLM prompts using semantic diffing instead of traditional text-based comparison.
A surprisingly effective way to predict token importance in LLM prompts
heatmap.demos.watchful.ioWe explored a novel method to gauge the significance of tokens in prompts given to large language models, without needing direct model access.
Wordllama
github.comThings you can do with the token embeddings of an LLM
A ~9M parameter LLM that talks like a small fish.