Use robotstxt rather than robotstxt.go
Use robotstxt rather than robotstxt.go

The package has been renamed, see:
https://github.com/temoto/robotstxt/blob/master/robots.txt-check/main.go

PuerkitoBio/gocrawl