crawler: now with error handling
crawler: now with error handling

get_url is called within a try block; to avoid issues such as unhandled exceptions leaving Postgres in an error state, transaction.rollback() is called after every request
ericholscher/django-test-utils,ericholscher/django-test-utils,acdha/django-test-utils,acdha/django-test-utils