NUTCH-2496 Speed up link inversion step in crawling script
NUTCH-2496 Speed up link inversion step in crawling script

- disable URL filtering and normalizing when calling invertlinks
  in bin/crawl

- add note that the steps invertlinks, dedup, index could also
  be done outside the loop over all segments created in the loop
  iterations

- move webgraph construction (commented out anyway) outside the
  loop because it's done over all available segments

commoncrawl/nutch,commoncrawl/nutch,commoncrawl/nutch,commoncrawl/nutch