Updated crawler to talk to queue asynchronously. Merged in HTTP auth stuff. Static queue hasn't yet been updated to use async interface.
Updated crawler to talk to queue asynchronously. Merged in HTTP auth stuff. Static queue hasn't yet been updated to use async interface.

simplecrawler/simplecrawler,simplecrawler/simplecrawler,civalin/node-simplecrawler,cgiffard/node-simplecrawler