Split tokenizer up into helper functions
Split tokenizer up into helper functions

In an attempt to make it more readable, and seperate parsing
logic from tokenizing logic.

noahmorrison/chevron,noahmorrison/chevron