SemiComplete Scrape - USE FOR TESTING
SemiComplete Scrape - USE FOR TESTING

2016-07-14 11:00:48 [scrapy] INFO: Dumping Scrapy stats:
{'downloader/exception_count': 2568,
 'downloader/exception_type_count/twisted.internet.error.ConnectError':
1,

'downloader/exception_type_count/twisted.internet.error.ConnectionRefuse
dError': 6,

'downloader/exception_type_count/twisted.internet.error.DNSLookupError':
 2428,
 'downloader/exception_type_count/twisted.internet.error.NoRouteError':
6,

'downloader/exception_type_count/twisted.internet.error.TCPTimedOutError
': 87,

'downloader/exception_type_count/twisted.web._newclient.ResponseNeverRec
eived': 40,
 'downloader/request_bytes': 760249,
 'downloader/request_count': 3550,
 'downloader/request_method_count/GET': 3550,
 'downloader/response_bytes': 9527868,
 'downloader/response_count': 982,
 'downloader/response_status_count/200': 432,
 'downloader/response_status_count/301': 320,
 'downloader/response_status_count/302': 202,
 'downloader/response_status_count/303': 4,
 'downloader/response_status_count/307': 2,
 'downloader/response_status_count/400': 5,
 'downloader/response_status_count/403': 3,
 'downloader/response_status_count/404': 8,
 'downloader/response_status_count/503': 6,
 'dupefilter/filtered': 57,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2016, 7, 14, 15, 0, 48, 21161),
 'item_scraped_count': 418,
 'log_count/DEBUG': 3990,
 'log_count/ERROR': 855,
 'log_count/INFO': 18,
 'log_count/WARNING': 5,
 'response_received_count': 436,
 'scheduler/dequeued': 3550,
 'scheduler/dequeued/memory': 3550,
 'scheduler/enqueued': 3550,
 'scheduler/enqueued/memory': 3550,
 'start_time': datetime.datetime(2016, 7, 14, 14, 50, 15, 927845)}
2016-07-14 11:00:48 [scrapy] INFO: Spider closed (finished)

khandelwal/fedtext