دانشکده ریاضی دانشگاه شریف، بهار 1402
This is an introduction to the theory and practice of Deep Learning using the PyTorch library. We start from the foundations (perceptron, Multi-Layer Perceptron, backpropagation, loss and optimization) and progress towards CNNs, RNNs and then to the more advanced topics such as Auto-Encoders (used for dimensionality reduction and obtaining latent representations of data), Generative Models (used in generating text and images), Transformers (used with sequence data such as text) and Graph Neural Networks (used with data that includes relations). We also cover applications of these methods to computer vision and natural language processing.
Instructor: Reza Rezazadegan (reza.rezazade[AT]sharif.edu)
Course webpage: https://dreamintelligent.com/deep-learning-2023/
Texts:
- Yann LeCun, Deep Learning, available at https://atcold.github.io/pytorch-Deep-Learning/
- Kevin P. Murphy, Probabilistic Machine Learning: Advanced Topics, available at https://github.com/probml/pml2-book/releases/latest
- Aston Zhang, et al., Dive Into Deep Learning available at https://d2l.ai/d2l-en.pdf
Other useful texts:
- Ian Goodfellow, at al., Deep Learning
- Jeremy Howard & Sylvain Gugger, Deep Learning for Coders with fastai & PyTorch
- Jeff Hawkins, On Intelligence
Course evaluation: presentation of a course topic (14pts), homework (6 pts).
Prerequisites: Linear algebra, multivariable calculus, basic familiarity with Python and Numpy
TAs: Pedram Rajaei
Syllabus:
1-Motivation, history, supervised learning, Perceptron and overview of course topics (LeCun 1)
2-Multi-Layer Perceptron, Gradient Decent and Backpropagation (LeCun 1.3, 2.1-2.3, 11; Murphy 16, Zhang 3-5)
3-Convolutional Neural Networks (LeCun 3, 4, 6.1, Zhang 7-8)
4-Optimization in Deep Learning (LeCun 5, Zhang 12)
5-Recurrent Neural Networks: RNN, LSTM, GRU (LeCun 6.2, 6.3, Zhang 9-10)
6-Energy-based Models and Autoencoders (LeCun 7; Murphey 20, 24)
7-Regularized Energy-based models and Variational Autoencoders (LeCun 8; Murphy 21)
8-Generative Adversarial Networks (LeCun 9; Murphy 25, 26, Zhang 20)
9-Self-supervised Learning (LeCun 10.1, 10.2)
10-Deep Learning for Natural Language Processing, Attention Mechanism, Transformers (LeCun 12, Zhang 11, 15)
11-Graph Neural Networks and Structured Prediction (LeCun 13, 14; Murphy 30)