Optimization Algorithms: Theory and Software Implementation

By Prof. Thirumulanathan D | IIT Kanpur

Learners enrolled: 444 | Exam registration: 4

ABOUT THE COURSE:

The course will introduce various iterative algorithms used to numerically solve the unconstrained and constrained optimization problems. Each algorithm will be introduced with examples and a python code that implements the algorithm. The focus will be on introducing the theory behind each algorithm, and implementing it using python.Programming assignments will consist of applications of these algorithms in fields such as machine learning, econometrics, and game theory.

INTENDED AUDIENCE: PG/ PhD students from the department of mathematics/ statistics/economic sciences who wish to learn various optimization algorithms along with its software implementation.

PREREQUISITES: A basic knowledge on optimization theory is expected

INDUSTRY SUPPORT: Many firms value the knowledge of python coding. Knowledge on implementing optimization algorithms using python would be an added advantage

Summary

Course Status :	Upcoming
Course Type :	Elective
Language for course content :	English
Duration :	12 weeks
Category :	Mathematics Economics & Social Sciences
Credit Points :	3
Level :	Postgraduate
Start Date :	20 Jan 2025
End Date :	11 Apr 2025
Enrollment Ends :	27 Jan 2025
Exam Registration Ends :	14 Feb 2025
Exam Date :	26 Apr 2025 IST

Note: This exam date is subject to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits

Course layout

Week 1: Introduction to optimization. Need for iterative algorithms.

Week 2: Line Search Algorithms. Implementation of exact and backtracking line search.

Week 3: Descent Algorithms. Implementation of steepest descent algorithm.

Week 4: Need for conjugate gradient algorithm. Implementation.

Week 5: Newton’s method. Advantages. Damped Newton method. Implementation.

Week 6: Quasi-Newton methods. Rank-one correction, DFP, BFGS methods. Implementation.

Week 7: Optimization with constraints. Linear program. Simplex method. Implementation.

Week 8: Interior point methods. Karmakar’s algorithm. Implementation.

Week 9: Nonlinear optimization. Projected Gradient Descent. Implementation.

Week 10: Penalty methods. Barrier methods. Implementation.

Week 11: Augmented Lagrangian Method. Implementation.

Week 12: Applications of optimization algorithms in machine learning, econometrics, game theory.

Books and references

1. Ben-Tal, A. and Nemirovski, A. “Lecture Notes:Optimization III. Convex Analysis, Nonlinear Programming Theory, and Nonlinear Programming Algorithms”.
2. Nocedal, J. and Wright, S.J. “Numerical Optimization”. Springer, 1999.
3. Dantzig, G.B. and Thapa, M.N. “Linear Programming 1:Introduction”. New York: Springer, 2003.
4. James, G., Witten, D., Hastie, T., and Tibshirani, R. “An introduction to statistical learning”. Vol. 112. New York: springer, 2013.

Instructor bio

Prof. Thirumulanathan D

IIT Kanpur

Prof. Thirumulanathan D is currently working as an assistant professor at the department of economic sciences, Indian Institute of Technology Kanpur, India. He completed his B.E. degree at the College of Engineering Guindy, Anna University, Chennai, in 2010, and M.E. and Ph.D. at the Indian Institute of Science, Bengaluru, in 2012 and 2017 respectively. He worked as a senior engineer in Qualcomm Inc. Bengaluru, from 2017-2020. His research interests include game theory, optimization, computational economics, and mathematical economics. He has introduced courses on machine learning for economists and computational methods in economics at IIT Kanpur.

Course certificate

The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centres.

The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).

Date and Time of Exams: April 26, 2025 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.

Registration url: Announcements will be made when the registration form is open for registrations.

The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.

Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE

Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course.

Exam score = 75% of the proctored certification exam score out of 100

Final score = Average assignment score + Exam score

Please note that assignments encompass all types (including quizzes, programming tasks, and essay submissions) available in the specific week.

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75. If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.

Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IIT Kanpur .It will be e-verifiable at nptel.ac.in/noc.

Only the e-certificate will be made available. Hard copies will not be dispatched.

Once again, thanks for your interest in our online courses and certification. Happy learning.

- NPTEL team