tokenizer2

v2.0.1
tokenize any text stream given some basic regex rules to match tokens
tokenizer through stream

Readme Versions Dependencies Scripts

Dependencies (1)

through2

Dev dependencies (2)

tap-min tape
npm i tokenizer2

Metadata

  • MIT
  • Whatever
  • smallhelm
  • released 4/9/2018

Downloads

Maintainers