GitHub
gpt-tokenizer
v2.1.0
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
BPE
encoder
decoder
tokenizer
GPT
GPT-2
GPT-3
GPT-3.5
GPT-4
and 7 more...
Readme
Versions
Dependencies
Scripts
Dependencies (1)
rfc4648
Dev dependencies (2)
tsx
@niieani/scaffold
npm i
[email protected]
Source Code
github.com/niieani/gpt-t...
Homepage
github.com/niieani/gpt-token...
Metadata
MIT
Whatever
Bazyli Brzoska
released 6/1/2023
Downloads
Downloads Tab Here
Maintainers
niieani