Courses » Stochastic Processes

Stochastic Processes


This course explanations and expositions of stochastic processes concepts which they need for their experiments and research. It also covers theoretical concepts pertaining to handling various stochastic modeling. This course provides classification and properties of stochastic processes, discrete and continuous time Markov chains, simple Markovian queueing models, applications of CTMC, martingales, Brownian motion, renewal processes, branching processes, stationary and autoregressive processes.


Under-graduate, Post-graduate and PhD students of mathematics, electrical engineering, computer engineering


A basic course on Probability


Goldman Sachs, FinMachenics, Deutsche Bank and other finance companies.


Prof.S. Dharmaraja earned his M.Sc. degree in Applied Mathematics from Anna University, Madras, India, in 1994 and Ph.D. degree in Mathematics from the Indian Institute of Technology Madras, in 1999. From 1999 to 2002, he was a post-doctoral fellow at the Department of Electrical and Computer Engineering, Duke University, USA. From 2002 to 2003, he was a research associate at the TRLabs, Winnipeg, Canada.
He has been with the Department of Mathematics, IIT Delhi, since 2003, where he is currently a Professor and Head, Department of Mathematics and joint faculty of Bharti School of Telecommunication Technology and Management. He appointed as 'Jaswinder & Tarvinder Chadha Chair Professor' for teaching and research in the area of Operations Research from May 2010 to July 2015. He has held visiting positions at the Duke University, USA, University of Calgary, Canada, University of Los Andes, Bogota, Colombia, National University of Colombia, Bogota, Colombia, University of Verona, Verona, Italy, Sungkyunkwan University, Suwon, Korea and Universita degli Studi di Salerno, Fisciano, Italy.
His research interests include applied probability, queueing theory, stochastic modeling, performance analysis of computer and communication systems and financial mathematics. He has published over 30 papers in refereed international journals and over 20 papers in refereed international conferences in these areas. He is an Associate Editor of International Journal of Communication Systems. Recently, he is co-author of a text book entitled "Introduction to Probability and Stochastic Processes with Applications" in John Wiley and co-author of a text book entitled "Financial Mathematics: An Introduction" in Narosa.

Prof.N. Selvaraju earned his Ph.D. degree in Mathematics from the Indian Institute of Technology Madras in 2001. From 2001 to 2003, he was a post-doctoral fellow at the Department of Mechanical Engineering (Division of Industrial Engineering), University of Minnesota, USA.  He has been with the Department of Mathematics, IIT Guwahati, since 2003, where he is currently a Professor. 
His research interests are applied probability and stochastic modelling, in particular in the areas of queueing theory, mathematical finance and inventory management in supply chains and has published over 15 papers in international journals and international conferences in these areas.

Week 1:Probability theory refresher
  1. Introduction to stochastic process
  2. Introduction to stochastic process (contd.)
Week 2:Probability theory refresher (contd.)
  1. Problems in random variables and distributions
  2. Problems in Sequence of random variables
Week 3:Definition and simple stochastic process 
  1. Definition, classification and Examples
  2. Simple stochastic processes
Week 4:Discrete-time Markov chains
  1. Introduction, Definition and Transition Probability Matrix
  2. Chapman-Kolmogorov Equations
  3. Classification of States and Limiting Distributions
Week 5:Discrete-time Markov chains (contd.)
  1. Limiting and Stationary Distributions
  2. Limiting Distributions, Ergodicity and stationary distributions
  3. Time Reversible Markov Chain, Application of Irreducible Markov chains in Queueing Models
  4. Reducible Markov Chains
Week 6:Continuous-time Markov chains
  1. Definition, Kolmogrov Differential Equation and Infinitesimal Generator Matrix
  2. Limiting and Stationary Distributions, Birth Death Processes
  3. Poisson processes
Week 7:Continuous-time Markov Chains (contd.)
  1. M/M/1 Queueing model
  2. Simple Markovian Queueing Models
Week 8:Applications of CTMC
  1. Queueing networks
  2. Communication systems
  3. Stochastic Petri Nets
Week 9:Martingales
  1. Conditional Expectation and filteration
  2. Definition and simple examples
Week 10:Brownian Motion
  1. Definition and Properties
  2. Processes Derived from Brownian Motion
  3. Stochastic Differential Equation
Week 11:Renewal Processes
  1. Renewal Function and Equation
  2. Generalized Renewal Processes and Renewal Limit Theorems
  3. Markov Renewal and Markov Regenerative Processes
  4. Non Markovian Queues
  5. Application of Markov Regenerative Processes
Week 12:Branching Processes, Stationary and Autoregressive Processes


  1. J Medhi, Stochastic Processes, 3rd edition, New Age International Publishers, 2009
  2. Liliana Blanco Castaneda, Viswanathan Arunachalam, Selvamuthu Dharmaraja, Introduction to Probability and Stochastic Processes with Applications, Wiley, 2012.
  3. Kishor S. Trivedi, Probability and Statistics with Reliability, Queuing, and Computer Science Applications, 2nd Edition, Wiley, 2002.

Name of the course:  Stochastic Processes
Course url: https://onlinecourses.nptel.ac.in/noc17_ma09/
Course duration : 12 weeks(In weeks)
Dates of exams : 23April 2017 
  • Time of exam : Shift 1: 9am-12 noon; Shift 2: 2pm-5pm
  • Any one shift can be chosen to write the exam for a course.
Final List of exam cities will be available in exam registration form.
Exam registration url - Will be announced shortly


  • The exam is optional for a fee. Exams will be on 23 April 2017.
  • Time: Shift 1: 9am-12 noon; Shift 2: 2pm-5pm
  • Any one shift can be chosen to write the exam for a course.
  • Registration url: Announcements will be made when the registration form is open for registrations.
  • The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published

  • Final score will be calculated as : 25% assignment score + 75% final exam score
  • 25% assignment score is calculated as 25% of average of Best 8 out of 12 assignments
  • E-Certificate will be given to those who register and write the exam and score greater than or equal to 40% final score. Certificate will have your name, photograph and the score in the final exam with the breakup. It will have the logos of NPTEL and Institute which is coordinating. It will be e-verifiable at nptel.ac.in/noc.