Skip to content

Commit 00b51b2

Browse files
jgrahamjgraham
committed
Remove the states dictionary from the tokenizer to reduce lookups
1 parent e9279e8 commit 00b51b2

File tree

1 file changed

+126
-169
lines changed

1 file changed

+126
-169
lines changed

0 commit comments

Comments
 (0)