Improve the internal API with methods required for the crawler
Improve the internal API with methods required for the crawler

pombredanne/mirrormanager2,pombredanne/mirrormanager2,Devyani-Divs/mirrormanager2,Devyani-Divs/mirrormanager2,pombredanne/mirrormanager2,pombredanne/mirrormanager2,Devyani-Divs/mirrormanager2,Devyani-Divs/mirrormanager2