First cut refactoring
First cut refactoring

(cherry picked from commit 71afec427baca8e37cd9e10d98812bc586e9a4ac)

pytorch/vision,pytorch/vision,jrdurrant/vision,pytorch/vision,pytorch/vision,pytorch/vision,pytorch/vision