Fix for the AttributeError when calling _crawl_delay with no self.robots (#5)
Fix for the AttributeError when calling _crawl_delay with no self.robots (#5)


cyriac/hodor,CompileInc/hodor,cyriac/hodor,CompileInc/hodor