FSFS reorg tool: complete implementation. Increase performance significantly by adding caches for combined windows as well as for directory contents.
FSFS reorg tool: complete implementation. Increase performance
significantly by adding caches for combined windows as well as for
directory contents.

Add implementation for the packing process: The initial reorganization
can only make educated guesses about the size of each fragment
to write. A pack step now (almost) eliminates the safety buffers
where they seem unnecessary. But afterwards we have to re-check
and may need to expand in a few places.

That process operates on one pack file at a time. To maximize
operational locality, we directly write the pack file after the
packing process finished.

Moreover, tune hash usage in a few places to maximize speed.

* tools/fsfs/reorg.c
  (dir_cache_entry_t, dir_cache_t,
   window_cache_entry_t, window_cache_t): new caching data structs
  (fs_fs_t): add caches to our root data structure
  (create_dir_cache, get_dir_cache_index, get_cached_dir_pool,
   get_cached_dir, set_cached_dir): implement directory cache
  (create_window_cache, get_window_cache_index,
   get_cached_window, set_cached_window): implement window cache
  (read_revisions): initialize new caches
  (parse_revnode_pos, get_combined_window,  read_dir): use caches

  (update_noderevs, get_content_length, move_fragment, pack_revisions,
   pack_and_write_revisions): implement 
  (write_revisions): write files directly after reorg'ing the respective pack

  (fs_open): support format 6 repos as well
  (direntry_t, create_content_cache, parse_dir, reorder_revisions,
   get_updated_dir): minimize hash overhead
  (main, print_progress, read_pack_file): nicer console output

jmckaskill/subversion,jmckaskill/subversion,jmckaskill/subversion,jmckaskill/subversion,jmckaskill/subversion,jmckaskill/subversion,jmckaskill/subversion,jmckaskill/subversion