Fix an evaluation bug in fairseq-generate (#1158)
Fix an evaluation bug in fairseq-generate (#1158)

Summary:
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/1158

When using BPE in --sacrebleu mode, the scores were computed before BPE was removed (H- strings, not D- strings). This is now fixed.

In addition, added warnings that not using --sacrebleu for scoring with target-side BPE is a bad idea.

Reviewed By: myleott

Differential Revision: D21260024

fbshipit-source-id: f8cf9e3a42e501043b794c841297940ab9e2b75a

pytorch/fairseq,pytorch/fairseq,pytorch/fairseq