Applied Linear Algebra in AI and ML

By Prof.Swanand Khare | IIT Kharagpur

Learners enrolled: 2771

ABOUT THE COURSE:

Linear algebra, optimization techniques and statistical methods together form essential tools for most of the algorithms in artificial intelligence and machine learning. In this course, we propose to build some background in these mathematical foundations and prepare students to take on advanced study or research in the field of AI and ML. The objective of this course is to familiarize students with the important concepts and computational techniques in linear algebra useful for AI and ML applications. The unique objective of this course and the distinguishing point from the existing courses on the similar topics is the illustration of application of these concepts to many problems in AI and ML. Some of the key topics to be covered in this course are listed below: least squares solution, parameter estimation problems, concept of cost function and relation to parameter estimation, constrained least squares, multi-objective least squares, applications to portfolio optimization, sparse solutions to underdetermined systems of linear equations, applications to dictionary learning, eigenvalue eigenvector decomposition of square matrices, spectral theorem for symmetric matrices, SVD, multicollinearity problem and applications to principal component analysis (PCA) and dimensionality reduction, power method, application to Google page ranking algorithm, inverse eigenvalue problem, construction of Markov chains from the given stationary distribution, low rank approximation and structured low rank approximation problem (SLRA), Autoregressive model order selection using Hankel SLRA, approximate GCD computation and application to image de- blurring, tensors and CP tensor decomposition, tensor decomposition based sparse learning in deep networks, matrix completion problems, application to collaborative filtering

INTENDED AUDIENCE: Senior undergraduate and post graduate students from CSE, EE, ECE, AI, Maths

PREREQUISITES: First course in Engineering Mathematics with some exposure to linear algebra

Summary

Course Status :	Completed
Course Type :	Elective
Language for course content :	English
Duration :	12 weeks
Category :	Mathematics
Credit Points :	3
Level :	Undergraduate/Postgraduate
Start Date :	23 Jan 2023
End Date :	14 Apr 2023
Enrollment Ends :	06 Feb 2023
Exam Registration Ends :	17 Mar 2023
Exam Date :	30 Apr 2023 IST

Note: This exam date is subject to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits

Course layout

Untitled Document

Week 1: Vectors, operations on vectors, vector spaces and subspaces,inner product and vector norm, linear dependence and independence, Matrices, linear transformations, orthogonal matrices
Week 2: System of linear equations, existence and uniqueness, left and right inverses, pseudo inverse, triangular systems
Week 3: LU decomposition and computational complexity, rotators and reflectors, QR decomposition, Gram Schmidt Orthogonalization
Week 4: Condition number of a square matrix, geometric interpretation, norm of matrix, sensitivity analysis results for the system of linear equations
Week 5: Linear least squares, existence and uniqueness, geometrical interpretation, data fitting with least squares, feature engineering, application to Vector auto-regressive models, fitting with continuous and discontinuous piecewise linear functions
Week 6: Application of least squares to classification, two-class and multi-class least squares classifiers, Polynomial classifiers, application to MNIST data set
Week 7: Multi-objective least squares, applications to estimation and regularized inversion, regularized data fitting and application to image de-blurring, constrained least squares, application to portfolio optimization
Week 8: Eigenvalue eigenvector decomposition of square matrices,spectral theorem for symmetric matrices
Week 9:SVD, relation to condition number, sensitivity analysis of least squares problems, variation in parameter estimates in regression
Week 10: Multicollinearity problem and applications to principal component analysis (PCA) and diinensionality reduction, power method, application to Google page ranking algorithm
Week 11: Underdetermined systems of linear equations, least norm solutions, sparse solutions, applications in dictionary learning and sparse code recovery, inverse eigenvalue problem, application in construction of Markov chains from the given stationary distribution
Week 12: Low rank approximation (LRA) and structured low rank approximation problem (SLRA), application to model order selection in time series, alternating projections for computing LRA and SLRA

Books and references

1.Introduction to Applied Linear Algebra- Vectors, Matrices, and Least Squares, Stephen Boyd and Lieven Vandenberghe, Cambridge University Press, 2018
2.Linear Algebra and Learning from Data, Gilbert Strang, Wellesley-Cambridge Press, 2019
3.Fundamentals of Matrix Computations,David Watkins, Wiley, 2010
4.Matrix Computations, Gene Golub, C. F. Van Loan, Hindustan Book Agency, 2015

Instructor bio

Prof.Swanand Khare

IIT Kharagpur

Prof. Swanand Khare obtained M.Sc. and Ph.D. degrees from IIT Bombay in 2005 and 2011 respectively. He was a post-doctoral researcher in the University of Alberta, Canada from 2011 to 2014 and then subsequently joined the Department of Mathematics at IIT Kharagpur. He currently works as an Associate Professor in the Department of Mathematics and jointly in the Centre of Excellence in Al at IIT Kharagpur. His research interests include inverse eigenvalue problems, computational linear algebra, estimation and computational issues in applied statistics. He has been actively participating in fundamental as well as applied research in these areas. He has supervised four PhD students and several masters' students in their research work. He served as an Associate Editor for a journal named Control Engineering Practice for a period of three years from 2018 to 2021. He is a recipient of Excellent Young Teacher Award 2018 at IIT Kharagpur.

Course certificate

The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centres.

The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).

Date and Time of Exams: April 30, 2023 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.

Registration url: Announcements will be made when the registration form is open for registrations.

The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.

Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE

Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course.
Exam score = 75% of the proctored certification exam score out of 100

Final score = Average assignment score + Exam score

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75. If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.

Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IIT Kharagpur. It will be e-verifiable at nptel.ac.in/noc.

Only the e-certificate will be made available. Hard copies will not be dispatched.

Once again, thanks for your interest in our online courses and certification. Happy learning.

- NPTEL team

SWAYAM Helpline / Support