Remove Unicode support in favor of speed.
Remove Unicode support in favor of speed.

Summary: Use strings.Split() instead of strings.Fields() if you don't
require Unicode support, it's faster.

(logcrunch-ws denotes implementations with strings.Split(), logcrunch-un
denotes strings.Fields())

The file for this test is: 4873510 lines long.

   _     live:s_storagenode
,_(')<   [   0 ] rbastic@storagenode:/home/rbastic #
\___)    time GOMAXPROCS=8 ./logcrunch-ws < access.log > ws-states

real    0m12.447s
user    1m6.969s
sys     0m10.191s

   _     live:s_storagenode
,_(')<   [   0 ] rbastic@storagenode:/home/rbastic #
\___)    time GOMAXPROCS=8 ./logcrunch-un < access.log > un-states

real    0m11.302s
user    1m17.114s
sys     0m3.719s

   _     live:s_storagenode
,_(')<   [   0 ] rbastic@storagenode:/home/rbastic #
\___)    time GOMAXPROCS=8 ./logcrunch-ws < access.log > ws-states

real    0m12.896s
user    1m10.384s
sys     0m10.230s

   _     live:s_storagenode
,_(')<   [   0 ] rbastic@storagenode:/home/rbastic #
\___)    time GOMAXPROCS=8 ./logcrunch-un < access.log > un-states

real    0m13.211s
user    1m31.489s
sys     0m4.160s

I ran a few additional tests but will spare the details; suffice it to
say it is always faster to avoid strings.Fields()

This is because strings.Fields() handles Unicode runes, which is
inherently more expensive to deal with than simple strings.

If you require Unicode parsing from your logfiles, then you will wish to
change this.

rbastic/logcrunch