Adversarial examples have been shown to be an effective way of assessing the robustness of neural sequence-to-sequence (seq2seq) models, by applying perturbations to the input of a model leading to large degradation in performance. However, these perturbations are only indicative of a weakness in the model if they do not change the semantics of the input in a way that would change the expected output. Using the example of machine translation (MT), we propose a new evaluation framework for adversarial attacks on seq2seq models taking meaning preservation into account and demonstrate that existing methods may not preserve meaning in general. Based on these findings, we propose new constraints for attacks on word-based MT systems and show, via human and automatic evaluation, that they produce more semantically similar adversarial inputs. Furthermore, we show that performing adversarial training with meaning-preserving attacks is beneficial to the model in terms of adversarial robustness without hurting test performance.
How you should evaluate adversarial attacks on seq2seq The authors investigate ways of generating adversarial examples, showing that adversarial training with the attack most consistent with the introduced meaning-preservation criteria results in improved robustness to this type of attack without degradation in the non-adversarial setting. The paper is about meaning-preserving adversarial perturbations in the context of Seq2Seq models