M # Module L# Topic Links

1 Course introduction
1 Revision of ML 2 Supervised machine learning (revision)
3 Introduction to neural networks
4 Convolutional layers https://ieeexplore.ieee.org/document/726791
http://openaccess.thecvf.com/content_cvpr_2017/papers/Chollet_Xception_Deep_Learning_CVPR_2017_paper.pdf
5 Variants of conv and pool https://www.saama.com/blog/different-kinds-convolutional-filters/
https://arxiv.org/pdf/1301.3557.pdf
6 CNN architectures for vision https://arxiv.org/abs/1506.02640
2 NN layer architectures
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
7 Recurrent neural networks
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.676.4320&rep=rep1&type=pdf
8 LSTM in detail
9 RNN architectures for NLP https://techblog.gumgum.com/articles/deep-learning-for-natural-language-processing-part-2-rnns
10 Some creative layer combinations
11 Loss functions https://arxiv.org/pdf/1701.03077.pdf
https://arxiv.org/pdf/1412.6980.pdf
12 Escaping saddle points
https://arxiv.org/pdf/1406.2572.pdf
13 Tuning learning rate https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7926641
http://openaccess.thecvf.com/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf
14 Residuals and other objective function http://openaccess.thecvf.com/content_cvpr_2017/papers/Huang_Densely_Connected_Convolutional_CVPR_2017_paper.pdf
3 Optimization and regularization
https://www.aaai.org/ocs/index.php/AAAI/AAAI17/paper/viewFile/14806/14311
http://papers.nips.cc/paper/563-a-simple-weight-decay-can-improve-generalization.pdf
15 Weight decay and dropout
http://www.jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf
16 L1 regularization and SCAD penalty https://www.jstor.org/stable/pdf/2346178.pdf
17 Some interesting examples
18 Some interesting examples
19 CRFs and training CRFs https://repository.upenn.edu/cgi/viewcontent.cgi?article=1162&context=cis_papers
20 Introduction to different types of models https://www.nowpublishers.com/article/Details/MAL-001
21 Message passing https://ieeexplore.ieee.org/document/910572
22 EM-Algorithm for training https://www.cs.utah.edu/~piyush/teaching/EM_algorithm.pdf
5 Probabilistic graphical models
23 Variational EM https://sccn.ucsd.edu/~jason/sp_nips.pdf
24 MCMC https://link.springer.com/article/10.3758%2Fs13423-016-1015-8
https://www.tandfonline.com/doi/10.1080/0094965031000147704
25 MCMC-EM
https://demonstrations.wolfram.com/MonteCarloExpectationMaximizationEMAlgorithm/
26 GAN http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
27 Issues with GAN training
4 Generative adverserial networks
28 Wasserstein GAN http://proceedings.mlr.press/v70/arjovsky17a/arjovsky17a.pdf
29 Some interesting examples
30 Transfer learning http://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf
31 Metric learning http://openaccess.thecvf.com/content_cvpr_2014/papers/Hu_Discriminative_Deep_Metric_2014_CVPR_paper.pdf
32 Semi-supervised learning http://papers.nips.cc/paper/5352-semi-supervised-learning-with-deep-generative-models.pdf
6 Other learning situations 33 Active learning https://arxiv.org/pdf/1708.00489.pdf
34 Weakly supervised learning http://papers.nips.cc/paper/1346-a-framework-for-multiple-instance-learning.pdf
35 Self-supervised learning http://delivery.acm.org/10.1145/1280000/1273592/p759-raina.pdf
36 Buffer