Finish all unit testing for tokenizer.
Finish all unit testing for tokenizer.

lazywei/mockingbird,nemesiscodex/mockingbird,nemesiscodex/mockingbird,lazywei/mockingbird,lazywei/mockingbird,lazywei/mockingbird,nemesiscodex/mockingbird,lazywei/mockingbird,nemesiscodex/mockingbird,nemesiscodex/mockingbird,lazywei/mockingbird,nemesiscodex/mockingbird,lazywei/mockingbird,nemesiscodex/mockingbird,nemesiscodex/mockingbird,lazywei/mockingbird,nemesiscodex/mockingbird,lazywei/mockingbird