Fix bug with multilingual engines causing different translations
Fix bug with multilingual engines causing different translations

This bugfix relates to and closes #531, closes #527.
Both bugs were originally thought to relate to handling or rare words,
which was partially true.

The problem arises with how ModernMT handles fitting multiple models of
varying vocabulary sizes into the same instance of `TransformerModel`
from `fairseq`. MMT currently handles this by fitting all models' weights
to the size of the largest embedding matrix, padding with zeros as
necessary. This works fine when embedding the input, as the padded
tokens never occur in the input. As the embedding matrix is however
reused as the parameter for the `output_layer` in FAIRSeq, this now
creates several logits with probability 0 for models that aren't the
one with the largest vocabulary in the multilingual engine. For the most
part, this has limited impact in real life scenarios, outside if small
but non-critical changes to the softmax distribution. It can however
happen that some model will predict a full distribution of logits with
negative values, which all of a sudden makes one of the padded tokens
the most likely logit with its value of 0. This causes the errors we see
in issues #531 and #527, and used to cause exceptions in MMT 4.2.

To solve this, I suggest one workaround in this PR, although it may be
possible to solve this in a more elegant way. The solution is quite
simple, by simply overriding the `get_normalized_probs` function in the
`TransformerModel` from `fairseq` with a wrapped function that truncates
the logits to the original vocabulary size of the model, before calling
the original `get_normalized_probs` function. It then subsequently pads
the probability distribution returned by `get_normalized_probs` with
`-math.inf` for all the padded tokens (in a similar way to how the <PAD>
token is handled in fairseq's `SequenceGenerator`. This should result in
inference being treated the exact same way for engines consisting of
one/many models.

I have tested this fix and found that it works for both regular
translation and for translation with context memories, without any
noticable impacts on throughput.

ModernMT/MMT,ModernMT/MMT,ModernMT/MMT,ModernMT/MMT,ModernMT/MMT