Applied Linear Algebra in AI and ML

Week 1: Vectors, operations on vectors, vector spaces and subspaces,inner product and vector norm, linear dependence and independence, Matrices, linear transformations, orthogonal matrices
Week 2: System of linear equations, existence and uniqueness, left and right inverses, pseudo inverse, triangular systems
Week 3: LU decomposition and computational complexity, rotators and reflectors, QR decomposition, Gram Schmidt Orthogonalization
Week 4: Condition number of a square matrix, geometric interpretation, norm of matrix, sensitivity analysis results for the system of linear equations
Week 5: Linear least squares, existence and uniqueness, geometrical interpretation, data fitting with least squares, feature engineering, application to Vector auto-regressive models, fitting with continuous and discontinuous piecewise linear functions
Week 6: Application of least squares to classification, two-class and multi-class least squares classifiers, Polynomial classifiers, application to MNIST data set
Week 7: Multi-objective least squares, applications to estimation and regularized inversion, regularized data fitting and application to image de-blurring, constrained least squares, application to portfolio optimization
Week 8: Eigenvalue eigenvector decomposition of square matrices,spectral theorem for symmetric matrices
Week 9:SVD, relation to condition number, sensitivity analysis of least squares problems, variation in parameter estimates in regression
Week 10: Multicollinearity problem and applications to principal component analysis (PCA) and diinensionality reduction, power method, application to Google page ranking algorithm
Week 11: Underdetermined systems of linear equations, least norm solutions, sparse solutions, applications in dictionary learning and sparse code recovery, inverse eigenvalue problem, application in construction of Markov chains from the given stationary distribution
Week 12: Low rank approximation (LRA) and structured low rank approximation problem (SLRA), application to model order selection in time series, alternating projections for computing LRA and SLRA