Robust Neural Machine
Translation with Doubly
Adversarial Inputs
Yong Cheng, Lu Jiang, Wolfgang Macherey
ACL 2019
2019/9/28
1
• NMT
2
- he she
•
•
–
3
Der … geladenen … Der … vorgeladenen …<BOS> … summoned …
The … continue …
→
<BOS> … invited …
The … continue …
• xi
•
xi’
4
xi gxi
→ xi
• xi
•
xi’
5
xi gxi
→ xi
•
•
–
•
–
• P(j)
• M δ
6
•
•
– 10
•
–
10
7
•
•
–
NMT
– NMT
– NMT
8
•
– - 120 LDC
– - 450 WMT14
• Transformer [Vaswani+ 17]
• BPE
– - 6
– - 3 2
•
9
10
-
-
Vaswani+ 17
•
• BackTranskation [Sennrich+ 16]
11
•
12
BLEU
BLEU
BLEU
Ablation test
•
13
→ Teacher forcing
• NMT
–
– Transformer [Vaswani+ 17]
–
•
– Teacher
forcing
14

Robust Neural Machine Translation with Doubly Adversarial Inputs