- Fixed bug in concat() -- accept subtypes of known types as well - Changed read_whitespace_block and read_wordpunct_block to read   blocks of 20 lines at a time (rather than 1 line at a time).     This will cut down on space overhead by a factor of about 20x,   and might actually increase local random-access performance,   because of the cache.  Random access that's really random (e.g.,   selecting words from the document in random order), will become   a little slower.
- Fixed bug in concat() -- accept subtypes of known types as well
- Changed read_whitespace_block and read_wordpunct_block to read
  blocks of 20 lines at a time (rather than 1 line at a time).  
  This will cut down on space overhead by a factor of about 20x,
  and might actually increase local random-access performance,
  because of the cache.  Random access that's really random (e.g.,
  selecting words from the document in random order), will become
  a little slower.


svn/trunk@4901

nltk/nltk,nltk/nltk,nltk/nltk