Download as PDF, PPTX
![1
DEEP LEARNING JP
[DL Papers]
http://deeplearning.jp/
Generating Wikipedia by Summarizing Long Sequences
(ICLR 2018)
Toru Fujino, scalab, UTokyo](https://image.slidesharecdn.com/slide-180223002928/75/Generating-Wikipedia-by-Summarizing-Long-Sequences-ICLR-2018-1-2048.jpg)
![• ( 2
• E !"
= [!%
"
, !'
"
, … , !)"
"
]
• E !+
= [!%
+
, !'
+
, … , !)+
+
]
• ) E
• A : D
5) M.-T. Luong et al. “Effective Approaches to Attention-based Neural Machine translation”, EMNLP 2015
5)](https://image.slidesharecdn.com/slide-180223002928/75/Generating-Wikipedia-by-Summarizing-Long-Sequences-ICLR-2018-13-2048.jpg)
More Related Content
![[DL輪読会]Generating Wikipedia by Summarizing Long Sequences](https://cdn.slidesharecdn.com/ss_thumbnails/slide-180309041103-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Paper Reading] Attention is All You Need](https://cdn.slidesharecdn.com/ss_thumbnails/reading20181228-190111054908-thumbnail.jpg?width=640&height=640&fit=bounds)
![[246]QANet: Towards Efficient and Human-Level Reading Comprehension on SQuAD](https://cdn.slidesharecdn.com/ss_thumbnails/246qanetdeview2018-181012000849-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Generating Wikipedia by Summarizing Long Sequences (ICLR 2018)
![[Introduction] Neural Network-Based Abstract Generation for Opinions and Argu...](https://cdn.slidesharecdn.com/ss_thumbnails/introductionpaper-160623020641-thumbnail.jpg?width=640&height=640&fit=bounds)
Generating Wikipedia by Summarizing Long Sequences (ICLR 2018)
- 1. 1 DEEP LEARNING JP [DLPapers] http://deeplearning.jp/ Generating Wikipedia by Summarizing Long Sequences (ICLR 2018) Toru Fujino, scalab, UTokyo
- 2. • . /0 •s ( -:: 8 2 I • : p 2>2 > goo.gl/wSuuS9 k • 1 2ILCG B iI rd i e • a Ird i I R l • G ItW ) • B W DC noI2>> > : g
- 3. • , )1 • , • • : , • • • : • • • , , ( ) • , ,
- 4. 1 • .G DCL N • • 51 3 2 : 0 6 4 1 ( • • R • // 1 2 : /1 1:1 4 1 ) • • • 1) Rush et al. “A Neural Attention Model for Sentence Summarization”, EMNLP 2015 2) Nallapati et al. “Abstractive Text Summarization using Sequence-to-Sequence RNNs and Beyond”, CoNLL 2016
- 5. • (, ,, ),() • e / • S • / • 2 A / / 2) Nallapati et al. “Abstractive Text Summarization using Sequence-to-Sequence RNNs and Beyond”, CoNLL 2016 2)
- 6.
- 7. • 1 WR a • • 2 G • 00 . c • • 1 d https://en.wikipedia.org/wiki/Deep_learning
- 8.
- 9.
- 10.
- 11. • 3 434 Y p • CD 4 M i • ac c 43 , 4 a • d ac c n nY r Ly • ot ldA CN m m 3 43 e • 4 , 3 43 m u • ) 24 : 42 3 43 m • s e 3) A. Vaswani et al. “Attention is All You Need”, NIPS 2017 4) N. Shazzer et al. “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer”, ICLR 2017
- 12. • . . •3 .) • .) A • ( 3 .) : : 3) A. Vaswani et al. “Attention is All You Need”, NIPS 2017
- 13. • ( 2 •E !" = [!% " , !' " , … , !)" " ] • E !+ = [!% + , !' + , … , !)+ + ] • ) E • A : D 5) M.-T. Luong et al. “Effective Approaches to Attention-based Neural Machine translation”, EMNLP 2015 5)
- 14. • ) :A •( , - , - • , - , - • , - , - • V E • SQ D K , , E K - E: AE
- 15. • • A ,- •
- 16. • Q a KA , • ( ) e6 • KV A i • d , A 5/ 2
- 17. • S Ka) a) eV • ) A ) • ( = = • 1 . 1 • i • d A 1, 3/
- 18. ) ( • ) • /( 4)N. Shazzer et al. “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer”, ICLR 2017 4)
- 19. • 9 L •2 / 02 2 / 02 2 / • 9 • M - - - 9 = • -1 5
- 20.
- 21. • - - •- • - • :
- 22.
- 24. )& ( • (()
- 26.
- 27. • M • // -,/ p k lsr im e • W lsr f : • n s k • a a > - - / • - / y • -2 2 - -, - / Wy • • ot C L c A • L M d L C • lsr : L