parse: separate full tokens and words further
parse: separate full tokens and words further

As expected, overall performance improvement. The peek version that
supports both may well end up doing extra checks and even string
manipulation when it isn't necessary.

ok      github.com/mvdan/sh     4.825s
name                 old time/op    new time/op    delta
Parse/Empty-4          1.20µs ± 5%    1.16µs ± 7%   -2.97%         (p=0.029 n=10+9)
Parse/Whitespace-4     4.47µs ± 2%    4.27µs ± 2%   -4.34%        (p=0.000 n=10+10)
Parse/Comment-4        1.69µs ± 3%    1.73µs ± 5%   +2.35%        (p=0.022 n=10+10)
Parse/LongLit-4        8.28µs ± 2%    8.17µs ± 2%   -1.39%        (p=0.006 n=10+10)
Parse/Cmds-4           14.5µs ± 1%    14.1µs ± 1%   -2.22%         (p=0.000 n=10+9)
Parse/SglQuoted-4      2.52µs ± 3%    2.60µs ± 3%   +3.21%        (p=0.001 n=10+10)
Parse/DblQuoted-4      5.01µs ± 4%    4.94µs ± 2%     ~           (p=0.160 n=10+10)
Parse/NestedStmts-4    11.6µs ± 2%    11.2µs ± 1%   -3.84%        (p=0.000 n=10+10)
Parse/Clauses-4        9.52µs ± 2%    9.18µs ± 1%   -3.63%         (p=0.000 n=10+9)
Parse/CmdSubst-4       9.59µs ± 3%    9.25µs ± 1%   -3.60%        (p=0.000 n=10+10)
Parse/Binary-4         7.51µs ± 2%    7.54µs ± 3%     ~           (p=0.579 n=10+10)
Parse/Redirect-4       8.74µs ± 3%    8.64µs ± 1%   -1.14%         (p=0.014 n=9+10)
Parse/Arithm-4         10.9µs ± 3%     9.7µs ± 1%  -11.16%         (p=0.000 n=10+9)

mvdan/sh,mvdan/sh,mvdan/sh,mvdan/sh