Clean up and optimize tokenization some more.
Clean up and optimize tokenization some more.

 - Get rid of the weird strip/reindex dance and keep 's' intact
   throughout. This also fixes _tokenization_error() output, as it would
   previously report the stripped string.

 - Strip trailing whitespace in the regexes. This makes the stripping
   loop very cheap.

 - Shorten a comment that might've been more confusing than helpful.

ulfalizer/Kconfiglib,ulfalizer/Kconfiglib