fix bug reported by sth: in tokenizer final \0 can be written one past the end of the buffer
fix bug reported by sth: in tokenizer final \0 can be written one past the end of the buffer

rain-1/s,rain-1/s,rain-1/s