gpt-tokenizer

v1.0.1
BPE Encoder Decoder for GPT-2 / GPT-3
BPE encoder decoder tokenizer GPT GPT-2 GPT-3 GPT-3.5 GPT-4 and 7 more...

Readme Versions Dependencies Scripts

Dependencies (0)

No dependencies

Dev dependencies (2)

tsx @niieani/scaffold
npm i [email protected]

Metadata

  • MIT
  • Whatever
  • Bazyli Brzoska
  • released 4/16/2023

Downloads

Maintainers