Commit 8ebff2e

committed

Move Tokenizer.state to Tokenizer._state

This is in preparation for Cython using C function pointers for _state

1 parent 0904df3 commit 8ebff2eCopy full SHA for 8ebff2e

3 files changed

+258

-237

lines changed

+258

-237

lines changed

Comments

(0)