Stochastic control and communication

By Prof. Ankur A. Kulkarni | IIT Bombay

Learners enrolled: 274

ABOUT THE COURSE:
The legacy of control is marked by a historic misconception that stochastic control problems could be solved using deterministic controllers superimposed with optimal state estimators and conventional communication protocols. Over time, we realized that this architecture is suboptimal and co-design of communication and control offers orders of magnitude of improvements.
This applies to systems such as smart cities, transportation systems, power grids, gaming and financial markets. This complex subject requires simultaneous application of control theory and communication theory. At present these elements are taught separately in courses designed either exclusively for deterministic control theory, or exclusively for communication theory. This leaves behind a gap, namely blending insights from both disciplines to arrive at approaches for networked control problems. This course seeks to fill this gap.
It will begin with centralized stochastic control, and then highlight open issues that arise in decentralization and the role of communication theory. Finally it will cover communication theory and establish viewpoints from which stochastic control and communication theory can be approached simultaneously.

PRE-REQUISITES: Comfort with probability.

INTENDED AUDIENCE: Students, researchers and practitioners of control and automation across any discipline.

INDUSTRY SUPPORT: Industries working in control and automation, decentralized multiagent systems.

Summary

Course Status :	Completed
Course Type :	Elective
Language for course content :
Duration :	12 weeks
Category :	Electrical, Electronics and Communications Engineering
Credit Points :	3
Level :	Postgraduate
Start Date :	25 Jul 2022
End Date :	14 Oct 2022
Enrollment Ends :	08 Aug 2022
Exam Date :	29 Oct 2022 IST

Note: This exam date is subject to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits

Course layout

Week 1: Markov decision process, finite horizon problem formulation, examples, principle of optimality, Bellman equation

Week 2: Infinite horizon problems, Optimality criteria (average cost, discounted cost), Bellman equation, optimality of Markov policies

Week 3: Computing optimal policies, linear programming formulation

Week 4: Partially observed Markov decision processes, reduction to the information state

Week 5: LQR problem, Kalman filter

Week 6: LQG problem, separation principle, optimality of linear policies

Week 7: Witsenhausen counterexample. information structure,

Week 8: Intrinsic model of stochastic control, LQG static teams, optimality of linear policies

Week 9: Variants of the Witsenhausen problem, Bansal Basar problem, optimizer’s approach

Week 10: Communication and decentralized control. Canonical communication problems of source coding, channel coding and rate distortion theory

Week 11: Shannon’s coding theorems

Week 12: Shannon’s coding theorems and optimizer’s approach

Books and references

Puterman, Martin L. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
Carpentier, Pierre, Guy Cohen, Jean-Philippe Chancelier, and Michel De Lara. ""Stochastic multi-stage optimization."" Probability Theory and Stochastic Modelling 75 (2015).
Witsenhausen, Hans S. ""A counterexample in stochastic optimum control."" SIAM Journal on Control 6, no. 1 (1968): 131-147.
Bertsekas, Dimitri P. Dynamic programming and optimal control: Vol. 1. Belmont: Athena scientific, 2000.
Kulkarni, Ankur A., and Todd P. Coleman. ""An optimizer's approach to stochastic control problems with nonclassical information structures."" IEEE Transactions on Automatic Control 60, no. 4 (2014): 937-949.
Yüksel, Serdar, and Tamer Ba?ar. Stochastic networked control systems: Stabilization and optimization under information constraints. Springer Science & Business Media, 2013.
Krishnamurthy, Vikram. Partially observed Markov decision processes. Cambridge university press, 2016.
Jose, Sharu Theresa, and Ankur A. Kulkarni. ""Linear programming-based converses for finite blocklength lossy joint source-channel coding."" IEEE Transactions on Information Theory 63, no. 11 (2017): 7066-7094.

Instructor bio

Prof. Ankur A. Kulkarni

IIT Bombay

Ankur is an Associate Professor with the Systems and Control Engineering group at Indian Institute of Technology Bombay (IITB). He received his B.Tech. from IITB in 2006, M.S. in 2008 and Ph.D. in 2010, both from the University of Illinois at Urbana-Champaign (UIUC). From 2010-2012 he was a post-doctoral researcher at the Coordinated Science Laboratory at UIUC. His research interests include information theory, the role of information in stochastic control, game theory, combinatorial coding theory problems, optimization and variational inequalities, and operations research. He is an Associate (from 20152018) of the Indian Academy of Sciences, Bangalore, a recipient of the INSPIRE Faculty Award of the Department of Science and Technology, Government of India, 2013, Best paper awards at the National Conference on Communications, 2017, Indian Control Conference, 2018, International Conference on Signal Processing and Communications (SPCOM) 2018, Excellence in Teaching Award 2018 at IITB and the William A. Chittenden Award, 2008 at UIUC. He was a consultant to the Securities and Exchange Board of India on regulation of high frequency trading. He has been a visiting faculty at MIT, USA, University of Cambridge, UK, NUS, Singapore and IISc Bangalore.

Course certificate

The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centres.

The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).

Date and Time of Exams: 29 October 2022 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.

Registration url: Announcements will be made when the registration form is open for registrations.

The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.

Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE

Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course.

Exam score = 75% of the proctored certification exam score out of 100

Final score = Average assignment score + Exam score

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75. If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.

Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IIT Bombay .It will be e-verifiable at nptel.ac.in/noc.

Only the e-certificate will be made available. Hard copies will not be dispatched.

Once again, thanks for your interest in our online courses and certification. Happy learning.

- NPTEL team