Created test to to check if dpeth is being checked when respect robots is enabled.
Created test to to check if dpeth is being checked when respect robots
is enabled.

spatie/crawler,spatie/crawler,spatie/crawler