stage 0.4.0
stage 0.4.0

* add support for meta 'robots' tag & robots.txt
* save states to cache store as possible
  - add saving for 'failture' status, with DenyException
* use cURL for Accept-Encoding
  entrusts decoding & encoding to cURL library

prezzemolo/analizzatore