[TorchClassifierAgent]Don't share optimizer if we don't have it (#2565)
[TorchClassifierAgent]Don't share optimizer if we don't have it (#2565)

* Fix

* add test

* Revert "add test"

This reverts commit 580d9966ab0e8151a2bec65c6334b46ae3afa4c9.
facebookresearch/ParlAI,facebookresearch/ParlAI,facebookresearch/ParlAI,facebookresearch/ParlAI,facebookresearch/ParlAI