Tokens
The smallest language units the model works with.
Loading tokenizer…
Token IDs
Their numeric form inside the model’s computations.
Enter any text and see how AI turns it into tokens and IDs, the building blocks of its thinking.
The smallest language units the model works with.
Their numeric form inside the model’s computations.