CSCI-P 558 Deep Learning
3 credits
- Prerequisite(s): CSCI-B 551 Elements of Artificial Intelligence or CSCI-B 555 Machine Learning or CSCI-B 565 Data Mining or INFO-H 515 Statistical Learning
- Delivery: On-Campus
- Semesters offered: Spring (Check the schedule to confirm.)
Description
This course covers deep learning neural networks. Topics include logistic regression, feedforward networks, autoencoders, convolutional neural networks, recurrent neural networks, graph neural networks, deep generative models, adversarial and reinforcement learning, and optimization and regularization techniques. Students also delve into recent research and learn through projects to develop deep learning systems.
Topics
Mathematical foundations
- Linear algebra
- Probability and information theory
- Numerical computation
Basics of deep learning
- Introduction to neural networks
- Activation functions
- Loss functions
Feedforward neural networks
- Deep feedforward networks
- Perceptrons
- Multi-layer perceptrons
Backpropagation
- Gradient descent
- Optimization for training deep models
Convolutional networks
- Architecture
- Pooling layers
Sequence modeling
- Sequence modeling: recurrent and recursive nets
- LSTMs
- GRUs
Regularization techniques
- Regularization for deep learning
- Dropout
- L1 and L2 regularization
Unsupervised learning
- Linear factor models
- Autoencoders
- Representation learning
Generative models
- Deep generative models
- GANs
- VAEs
Structured probabilistic models
- Structured probabilistic models for deep learning
- Monte Carlo methods
- Confronting the partition function
- Approximate inference
Transfer learning
- Fine-tuning
- Pre-trained models
Natural language processing
- Word embeddings
- Transformers
Reinforcement learning
- Q-learning
- Policy gradients
Evaluation metrics
- Accuracy
- F1 score
Specialized architectures
- Attention mechanisms
- Residual networks
Advanced topics
- Neural architecture search
- Federated learning
Learning Outcomes
- Analyze the underlying mathematical foundations in the context of deep learning models. CS 1
- Decompose various neural network architectures to understand their components and functionality. CS 4
- Examine regularization techniques to diagnose overfitting or underfitting in model training. CS 4
- Evaluate the suitability of different neural network architectures for specific problems. CS 4
- Critically assess research papers and methodologies in the field of deep learning. CS 7
- Evaluate model performance using various metrics like accuracy and F1 score. CS 4
- Design a neural network model tailored to a specific real-world problem. CS 2
- Develop a structured approach to model training, validation, and testing. CS 4
- Design and conduct experiments using transfer learning or generative models to solve advanced problems. CS 4
Policies and Procedures
Please be aware of the following linked policies and procedures. Note that in individual courses instructors will have stipulations specific to their course.