Every FLUSH_COUNT (default: 100) files, clear tokenizer tables.
Every FLUSH_COUNT (default: 100) files, clear tokenizer tables.

Major speedup, AND less memory used.

kaostao/glean,kaostao/glean,kaostao/glean,silentbicycle/glean,silentbicycle/glean