CSCI-B 655 Pattern Recognition
3 credits
- Prerequisite(s): None (Statistics, probability, linear algebra, and data structures required)
- Delivery: On-Campus
Description
This course covers pattern recognition and machine learning techniques. Topics include Bayesian decision theory, clustering, component analysis, hidden Markov models, linear discriminant functions, maximum-likelihood and Bayesian parameter estimation, neural networks, nonparametric techniques, stochastic methods, and unsupervised learning.
Topics
Introduction
- Machine perception
- Pattern recognition systems
- The design cycle
- Learning and adaptation
Bayesian decision theory
- Bayesian decision theory: Continuous features
- Classifiers, discriminant functions, and decision surfaces
- The normal density
- Error probabilities and integrals
- Signal detection theory and operating characteristics
- Bayesian decision theory: Discrete features
- Missing and noisy features
- Bayesian belief networks
- Compound Bayesian decision theory and context
Maximum-likelihood and bayesian parameter estimation
- Maximum-likelihood estimation
- Bayesian estimation
- Parameter estimation
- Recursive bayes learning
- Noninformative priors and invariance
- Gibbs algorithm
- Sufficient statistics
- Problems of dimensionality
Component analysis and discriminants
- Principal component analysis (PCA)
- Fisher linear discriminant
- Multiple discriminant analysis
- Expectation-maximization (EM)
Hidden Markov models
- First-order Markov models
- First-order hidden Markov models
- Baum-Welch algorithm
- Viterbi algorithm
Nonparametric techniques
- Density estimation
- Parzen window estimation
- K-nearest-neighbor classification
- Fuzzy classification
- Reduced Coulomb energy networks
Linear discriminant functions
- Linear discriminant functions and decision surfaces
- The two-category linearly separable case
- Minimum squared-error procedures
- Stochastic approximation methods
- Linear programming algorithms
- Support vector machines
Multilayer neural networks
- Feedforward operation and classification
- Backpropagation algorithm
- Deep learning
- Practical techniques for improving backpropagation
- Additional networks and training methods
Stochastic methods
- Stochastic search
- Evolutionary methods
Nonmetric methods
- Decision trees
- Recognition with strings
Algorithm-independent machine learning
- Lack of inherent superiority of any classifier
- Minimum description length (MDL)
- Bias and variance
- Resampling for estimating statistics
- Combining classifiers
Unsupervised learning and clustering
- Mixture densities and identifiability
- K-means clustering
- Unsupervised bayesian learning
- Criterion functions for clustering
- Hierarchical clustering
Mathematical foundations
- Linear algebra
- Probability theory
- Hypothesis testing
- Information theory
Learning Outcomes
- Analyze and differentiate complex pattern recognition and machine learning algorithms to select the most appropriate methodology for specific problems.CS 1
- Evaluate and compare the performance and theoretical foundations of various machine learning models.CS 4
- Design, implement, and optimize models using advanced techniques in pattern recognition.CS 2
- Create innovative solutions for real-world problems by synthesizing knowledge from pattern recognition and machine learning.CS 2
- Critically assess and interpret the results of experiments to enhance model performance in machine learning.CS 1
- Conduct advanced statistical analyses for parameter estimation and validation of pattern recognition models.CS 4
- Develop and justify the selection of feature extraction techniques and data preprocessing methods for complex datasets.CS 4
- Implement and analyze unsupervised learning techniques and clustering algorithms to uncover patterns in data.CS 4
- Formulate and conduct research to advance the state of the art in pattern recognition and machine learning.CS 7
Policies and Procedures
Please be aware of the following linked policies and procedures. Note that in individual courses instructors will have stipulations specific to their course.