:zip: #2106 basic tokenizer.tokenize global speed improvement (3-8x) by simply caching added_tokens in a Set
:zip: #2106 basic tokenizer.tokenize global speed improvement (3-8x) by simply caching added_tokens in a Set

huggingface/transformers,huggingface/pytorch-transformers,huggingface/transformers,huggingface/transformers,huggingface/transformers