gpt-tokenizer
niieani esm cjs
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
Version 2.8.1 License MIT
Keywords
BPEencoderdecodertokenizerGPTGPT-2GPT-3GPT-3.5GPT-4GPT-4oNLPNatural Language ProcessingText GenerationOpenAIMachine Learning
INSTALL
Requests 0
Bandwidth 0
Top version - 0
Full gpt-tokenizer Download Stats 
ShareKeywords
BPE
encoder
decoder
tokenizer
GPT
GPT-2
GPT-3
GPT-3.5
GPT-4
GPT-4o
NLP
Natural Language Processing
Text Generation
OpenAI
Machine Learning
ml
Get a badge for your package
