examples/l3fwd: reorganise and optimize LPM code path
examples/l3fwd: reorganise and optimize LPM code path

With latest HW and optimised RX/TX path there is a huge gap between
tespmd iofwd and l3fwd performance results.
So there is an attempt to optimise l3fwd LPM code path and reduce the gap:
 - Instead of processing each input packet up to completion -
 divide packet processing into several stages and perform
 stage by stage for the whole burst.
 - Unroll things by the factor of 4 whenever possible.
 - Use SSE instincts for some operations (bswap, replace MAC addresses, etc).
 - Avoid TX packet buffering whenever possible.
 - Move some checks from RX/TX into setup phase.

Note that new(optimized) code path can be switched on/off by setting
ENABLE_MULTI_BUFFER_OPTIMIZE macro to 1/0.

Some performance data:
SUT: dual-socket board IVB 2.8GHz, 2x1GB pages.
4 ports on 4 NICs (all at socket 0) connected to the traffic generator.
kernel: 3.11.3-201.fc19.x86_64, gcc: 4.8.2.
64B packets, using the packet flooding method.
All 4 ports are managed by one logical core:
Optimised scalar PMD RX/TX was used.

                          DIFF % (NEW-OLD)
IPV4-CONT-BURST:               +23%
IPV6-CONT-BURST :              +13%
IPV4/IPV6-CONT-BURST:          +8%
IPV4-4STREAMSX8:               +7%
IPV4-4STREAMSX1:               -2%

Test cases description:
IPV4-CONT-BURST - IPV4 packets all packets from the one input port
are destined for the same output port.
IPV6-CONT-BURST - IPV6 packets all packets from the one input port
are destined for the same output port.
IPV4/IPV6-CONT-BURST - mix of the first 2 with interleave=1
(e.g: IPV4,IPV6,IPV4,IPV6, ...)
IPV4-4STREAMSX1 - 4 streams of IPV4 packets, where all packets
from same stream are destined for the same output port
(e.g: IPV4_DST_P0, IPV4_DST_P1,  IPV4_DST_P2, IPV4_DST_P3, IPV4_DST_P0, ...)
IPV4-4STREAMSX8 - same as above but packets for each stream
are coming in groups of 8
(e.g: IPV4_DST_P0 X 8, IPV4_DST_P1 X 8, IPV4_DST_P2 X 8, IPV4_DST_P3 X 8,
IPV4_DST_P0 X 8, ...)

Signed-off-by: Konstantin Ananyev <cbd5212b59ab42a210d72992d23a3aa17cfd3eaf@intel.com>
Tested-by: Waterman Cao <8e01f67a90086248c4e0cdea3a53a7cacd141398@intel.com>
Acked-by: Pablo de Lara Guarch <28355d9cfe41bb8148aea0fb24f47aae8f4217a1@intel.com>

msune/dpdk,mixja/dpdk,venkynv/dpdk-mirror,john-mcnamara-intel/dpdk,venkynv/dpdk-mirror,mixja/dpdk,tsphillips/dpdk-fork,msune/dpdk,venkynv/dpdk-mirror,mixja/dpdk,mixja/dpdk,msune/dpdk,john-mcnamara-intel/dpdk,john-mcnamara-intel/dpdk,john-mcnamara-intel/dpdk,tsphillips/dpdk-fork,venkynv/dpdk-mirror,tsphillips/dpdk-fork,msune/dpdk,tsphillips/dpdk-fork