Skip to content
The Internet Compass

AI

Token

A token is the smallest unit of text a language model processes, typically a word fragment of about four characters in English. Model context limits and pricing are both denominated in tokens.

English averages roughly 0.75 words per token, so 1,000 tokens is about 750 words. Code, non-Latin scripts and unusual formatting tokenise less efficiently.

Because both input and output are billed per token, prompt design has a direct and measurable effect on unit economics at scale.